> Everybody wants the job to run four hours before the work day starts

recurring events like this - ones that take place relative to humans in a given location - are the most common snag related to "just use UTC everywhere" advice. but they're not actually in conflict at all. they just require an extra layer of indirection.

let's say this is a factory that starts early in the morning, 06:30 local time, because then "four hours before" falls into the window affected by DST.

"run at 02:30 local time" is what you want to avoid, because on the "spring forward" day, 02:30 simply does not exist. the clocks jump from 01:59:59 to 03:00:00. [0]

likewise, "run at 01:30 local time" is to be avoided, because on the "fall back", that occurs twice.

ideally, you want to think of the recurrence as being associated with a given location, and that recurrence generates individual events, which happen in UTC time.

for example, if the factory is in San Francisco, the recurrence might specify "once a day, at 02:30 SF time", which would generate a series of daily events in UTC time. during the summer those events are at 09:30Z, during the winter they're at 10:30Z.

you also want the scheduler to be smart enough to understand the "once a day" constraint - so that a "once a day, at 01:30 local time" task generates only one daily event, rather than two, even though 01:30 local time happens twice in the same day.

and then, because the individual events are all in UTC, and the system clock is as well, there's no ambiguity in the actual execution time of the event. the complexity of the recurrence is handled at a higher level.

if you want an even harder version of this problem, imagine a team in SF and a team in London, that have a Zoom meeting every Monday at 8am (SF) / 4pm (London). it's necessary for one of those locations to "own" the recurrence, because the UK does their time changes 1-3 weeks offset from the US [1], creating weeks where it isn't a straightforward 8 hour time difference.

0: https://www.timeanddate.com/time/change/usa

1: https://www.timeanddate.com/time/change/uk