With a date-based automation just make sure that when you add contacts into the started automation that there is sufficient time between the date added and the date in the contact custom field and how this relates to the first action step.

e.g. the first action is to send an email 2 weeks before the custom field date, so make sure that the contact is added at least 2 weeks before that date.