> I’ve been thinking recently about what makes time series forecasting problems so difficult compared to other sequence learning tasks
Whenever I teach people time series forecasting, I always point out that one of the biggest challenges is that you will always have values at prediction time that are out side the range of values observed during training (specifically the value of t).
In plenty of other machine learning and statistical modeling tasks this is not the case. You can train on every token you'll ever see and every pixel value you'll ever see, you can do regression analysis on every categorical value you include and an least an observation from within a range of every continuous and discrete value you'll observe. But with forecasting you will always have values you predict that are outside the range of anything you trained on.
You would run into similar problems if you tried to create a statistical model of the density of water given a temperature but your training data only included values between 0-100 C and you went out and started predicting values covering all the temperatures found on Earth.
For whatever reason, when time is a variable we somehow think it is immune from the obvious limitation of predicting on values outside of the range of values you trained on.
> Whenever I teach people time series forecasting, I always point out that one of the biggest challenges is that you will always have values at prediction time that are out side the range of values observed during training (specifically the value of t).
I don't get this, time is usually not a covariate in ts models, so why is it a challenge?
Shouldn't the training data be preprocessed to transform time values to a relative coordinate system? I would expect a time-series training pipeline to contain something like:
1. Define {N = context duration, M = forecast duration} upfront
2. Select some time value T
3. Extract historical data whose timestamps lie in time interval (T, T+N+M)
4. Transform timestamp values to (-N, M) interval by subtracting T+N from each timestamp
5. Append timestamp-transformed data to training data
6. Goto 2
Or are you saying that people don't want to define N and M upfront?
To ensure that your forecaat don't fall outside the range of training dataset you can use some tree based model like Random Forest for time series forecasting which averages out the values.