Lag features for tree-based forecasting

Tree estimators expect rows and columns, so a time series must be expressed as features available before each target. Common choices include the previous observation, the same period last season and rolling summaries of earlier observations. The word “earlier” is the entire safety condition: a rolling mean that includes the target turns forecasting into leakage.

For monthly target y[t], this lab creates lag1 = y[t-1], lag12 = y[t-12] and mean3 = mean(y[t-3:t]). The Python slice ends before t.

One-step rolling evaluation

The verified time-series lab trains a depth-limited decision tree on 108 feature rows before the test origin and evaluates 24 later rows. Its test RMSE is 19.22.

from timeseries_cases import lag_case

result = lag_case()
assert result["training_rows"] == 108
assert result["test_rows"] == 24
assert round(result["first_test_features"]["lag1"], 6) == round(81.84615082897504, 6)
print(round(result["test_rmse"], 2))

For the first test month, lag 1 is 81.85, lag 12 is 80.38 and the three-month trailing mean is 91.69 after rounding. The stored boundary assertion verifies that lag 1 equals the final development observation.

The evaluation is one-step rolling: when scoring each later test month, actual outcomes from earlier test months are available to construct its lags. It is not a 24-step forecast created once at the test origin. That distinction must appear beside the score.

Choose a multi-step strategy

For a fixed 24-month origin, future lags do not exist. A recursive strategy predicts the next point and feeds that prediction into later lags. A direct strategy fits a model for each horizon. A multi-output model predicts all horizons together. Each changes the training table and error behavior; compare them under the actual planning cadence.

Lag features also need group-safe construction. With many SKUs, shift and roll within each ordered SKU series. Confirm complete timestamps or encode gaps. A 12-row lag is not a 12-month lag when stored months are missing.

Trees do not understand time automatically

A random train-test split can let later regimes teach the tree about earlier targets. Shuffling row order does not change the leakage inside features already calculated on the full dataset. Construct features with explicit cutoffs, then use rolling or blocked validation.

Trees interpolate through feature regions well but may extrapolate a trend poorly. Include calendar and known-future drivers when justified, retain a seasonal-naïve baseline, and examine errors after level shifts. Feature importance does not establish a causal driver.

The Data Science course covers tree models together with feature lineage, validation and project handover.

Exercise

Add lags 2, 3 and 6 and trailing means of 3, 6 and 12 months. Use a rolling-origin feature builder that accepts an origin. Compare one-step rolling and fixed-origin recursive scores without using future actuals in the latter.

Continue learning

This article is part of the Forecasting and time-series analysis sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Regression forecasting with known future covariates.
• Continue with Intermittent demand: why ordinary averages disappoint.

Reference: scikit-learn’s lagged-feature forecasting example.