Data Leakage in Time Series: Lag Features Need Care

Why lag features feel safe but often are not

Lag features are one of the first tools anyone reaches for when modelling time series. Sales last week, temperature yesterday, the moving average of the last seven readings: these feel intuitively safe because they only look backwards. Surely nothing about the future can hide in a value that was recorded before the point you are trying to predict. That intuition is exactly why lag features are such a common source of leakage. The danger is never in the concept of a lag, it is in how the lag is computed and split relative to the rest of the pipeline.

The core problem is that a lag feature is only as trustworthy as the process that generated it. If that process, even briefly, uses information that would not have been available at prediction time, the model gets a sneak preview of the future. Because the leak is small and indirect, it rarely produces an obviously broken model. Instead it produces a model that looks slightly too good on validation and then underperforms once it meets real, live data. That gap between validation performance and production performance is the signature of leakage, and lag features are one of its favourite hiding places.

I think this matters more than people admit because time series problems are unusually forgiving of sloppy code during development and unusually punishing once deployed. A tabular classification leak often shows up as a suspiciously perfect accuracy score that gets caught quickly. A time series leak through a lag feature might only inflate an error metric by a modest margin, easy to mistake for a genuinely good model, and it will not be discovered until the forecast is running against real future data with no ability to peek.

A worked example: the rolling mean that knows too much

Suppose I am forecasting daily electricity demand and I build a feature called rolling_mean_7, the average demand over the trailing seven days. A common but wrong way to compute this is to take the full dataframe, sort by date, and apply a rolling window function with the window centred or, worse, computed after the target column has already been shifted incorrectly. If the rolling window is defined as the mean of days t-3 to t+3 instead of t-7 to t-1, the feature for day t now contains demand from three days in the future. The model trains on this, and because future demand is highly correlated with near future demand, the feature becomes a strong, apparently legitimate predictor.

Here is the numbers version. Say true next day demand has a correlation of around 0.6 with a correctly lagged seven day average, which is a reasonable and honest predictive relationship. If I accidentally centre the window, the correlation with the target might jump to something like 0.85, because I have quietly included demand from the day I am trying to predict and the two days after it. During cross-validation this looks fantastic, error drops sharply, and I might report a validation RMSE of, say, 120 megawatts instead of the honest 180 megawatts. In production, where day t+1 and t+2 genuinely do not exist yet, the feature cannot be computed the same way, so the leaked information disappears and real world error reverts towards that honest 180, or worse if the mismatch causes the model to behave erratically.

A subtler version of the same mistake happens with normalisation and imputation rather than the lag itself. If I fill missing values in a lag column using the overall mean or median of the full series, computed once across the entire dataset before splitting into train and test, I have again let future values influence a feature that is supposed to represent only the past. The lag column looks perfectly ordinary, dates align, nothing is obviously shifted, yet the imputation constant was calculated using data that had not happened yet relative to earlier rows. This kind of leak is harder to spot precisely because the feature values themselves look sensible.

Building lags and splits that respect time

The first defence is to compute every lag, rolling statistic, and derived feature strictly within a function that only ever looks backwards from the current row, and to do this before any split rather than relying on the split to somehow protect you afterwards. A rolling mean over the last seven days should use a window explicitly anchored so that the value at row t only ever touches rows t-7 through t-1. It is worth writing a small unit test that checks, for a synthetic series where each value equals its own timestamp index, that the lag feature at row t exactly equals the expected backward looking calculation and never touches t or later.

The second defence is the split itself. Ordinary k-fold cross-validation shuffles rows, which for time series is close to guaranteed leakage because a training fold can easily contain rows from after a validation row, and lag features computed globally will have absorbed that future information regardless of how carefully the lag function itself was written. Walk-forward validation, where each fold trains on all data up to a point and validates on a following block, respects the arrow of time and should be the default rather than an afterthought. Any feature engineering step involving statistics computed across the whole series, means, standard deviations, category encodings, needs to be recomputed within each walk-forward fold using only that fold's training window, not fitted once globally and then applied everywhere.

The third defence is to be suspicious of any lag feature that performs unusually well, rather than simply pleased. If a single lagged variable dominates feature importance and its correlation with the target seems too strong given the domain, that is worth investigating with the same scepticism I would apply to a suspiciously high accuracy score elsewhere. A quick sanity check is to compute the feature manually for a handful of rows by hand or with a slow, obviously correct loop, and compare it against the vectorised production version, since vectorised rolling and shift operations are exactly where off by one errors creep in.

The practical takeaway is straightforward even if the debugging is not. Treat every lag and rolling feature as a small pipeline in its own right, one that must be provably backward looking, tested against a synthetic series where the correct answer is obvious, and recomputed inside each walk-forward fold rather than fitted once on the whole dataset. Time series leakage rarely announces itself. It shows up quietly as an unusually good validation score, and the only reliable defence is to build the habit of asking, for every single feature, exactly what information was available at the moment that row's target was still unknown.