Shouldn't the training data be preprocessed to transform time values to a relative coordinate system? I would expect a time-series training pipeline to contain something like:

1. Define {N = context duration, M = forecast duration} upfront

2. Select some time value T

3. Extract historical data whose timestamps lie in time interval (T, T+N+M)

4. Transform timestamp values to (-N, M) interval by subtracting T+N from each timestamp

5. Append timestamp-transformed data to training data

6. Goto 2

Or are you saying that people don't want to define N and M upfront?