Time Series Cross-Validation Interview Questions: Forecast Horizons, Gaps, and Leakage

Time series cross-validation interview questions often turn on one detail: what information was available when the prediction would have been made? Sorting timestamps is necessary for many forecasting evaluations, but it does not establish that training labels had arrived or that historical features were available.

A gap parameter does not settle those questions either. In scikit-learn, TimeSeriesSplit.gap excludes a number of samples, not a calendar duration. TimeSeriesSplit documentation

This guide runs an original irregular-date fixture with scikit-learn 1.7.2. It shows exact train and test membership, removes unavailable labels, and masks two future-informed features. These are local split and preprocessing checks, not a forecasting accuracy benchmark.

Define the Forecasting Contract First

State the prediction origin, target horizon, retraining schedule, and scoring schedule before selecting a splitter. Forecasting evaluation should reproduce the intended use of the model. Rolling-origin evaluation can assess different horizons, rather than assuming that a one-step result answers a multi-step question. Forecasting: Principles and Practice

Our original exercise predicts a count over the next three calendar days at each scoring origin. The completed label arrives two days after that window ends. Consequently, a row's label becomes available five days after its origin.

We fit once at the start of each fold and keep that fitted model and preprocessing fixed while scoring the fold's two origins. Features may update at each scoring origin. That is different from issuing all future predictions together at the first origin, when later observations would still be unavailable.

All timestamps in the fixture are midnight UTC. A label whose availability equals the fit timestamp is treated as already ingested before fitting. In production, ordering at that boundary needs an explicit rule; a date without a publication time may not be sufficient evidence.

Inspect the Actual Dates Behind the Rows

The twelve scoring origins fall on these January 2026 dates:

row:   0  1  2  3  4  5  6  7   8   9  10  11
day:   1  2  3  4  5  6  8  9  20  21  22  30

Configure the splitter as follows, with the input already sorted by origin:

from sklearn.model_selection import TimeSeriesSplit

splitter = TimeSeriesSplit(
    n_splits=3,
    test_size=2,
    gap=1,
)

The executed output is below. Index ranges in this article are inclusive.

Fold	Candidate train	Test
1	0–4	6, 7
2	0–6	8, 9
3	0–8	10, 11

Each fold removes one row immediately before its test block: rows 5, 7, and 9 respectively. But the last candidate training origin and first test origin are separated by three, twelve, and two days. The same row-count parameter produces different calendar separation.

The two test origins span one day in the first fold, one in the second, and eight in the third. Equal row counts therefore do not imply comparable calendar coverage. Scikit-learn documents equal spacing as a condition for comparable fold durations. TimeSeriesSplit behavior

This irregular fixture is useful precisely because it exposes that limitation. It is not a recommendation to average its three fold scores as though they represented identical periods.

Repair the Delayed-Label Leak

The first fold fits on January 8. Candidate training row 3 has a January 4 origin, but its label arrives January 9. Row 4's January 5 label window and reporting delay finish January 10. Both origins precede the test period; neither label was available at fit time.

A row split that includes their eventual labels leaks future information. A one-row gap has already been applied and has not prevented the problem.

Use the availability contract to filter candidate training rows:

for train_idx, test_idx in splitter.split(df):
    fit_at = df.loc[test_idx[0], "origin"]
    known = (
        df.loc[train_idx, "label_available_at"] <= fit_at
    ).to_numpy()
    safe_train = train_idx[known]

Here df has the listed origins and label_available_at = origin + five days. The code assumes a reset integer index matching the displayed row numbers; do not silently reuse it on an unrelated DataFrame index.

Fold	Unavailable labels	Eligible train
1	3, 4	0–2
2	None	0–6
3	8	0–7

The local test asserted every resulting training label was available by that fold's fit time and that train and test indices were disjoint. Fold 3 excludes row 8 because its January 20 origin produces a label available January 25, after the January 22 fit.

This filter retains the original splitter's gap. It establishes eligibility among its candidates; it is not an algorithm for finding every possible usable historical row.

Check the Feature Clock Separately

A label can be eligible while its stored feature value is inappropriate. Row 2 has a January 3 prediction origin, but one feature arrives January 4. By the January 8 fit, both the feature and label exist. Nevertheless, using that feature as though it had been available on January 3 creates a historical snapshot that the scoring process could not have seen.

The same issue appears in test row 7: its origin is January 9, while the feature arrives January 10. Waiting until evaluation day to query the latest warehouse value would conceal the delay.

In the original fixture, all other feature values are available at their row's origin. We mask the two unavailable values before fitting preprocessing:

safe_feature = df["feature"].where(
    df["feature_available_at"] <= df["origin"]
)

Masking preserves the prediction rows. It does not quietly remove difficult test cases and improve the reported score by changing who receives a prediction. A real application needs a defined fallback, such as a permissible earlier snapshot or a trained missing-value treatment.

Availability also applies to revised data. A measurement can describe an old event while its corrected value was published much later. Event time alone cannot establish that the value belonged in the historical feature set.

Fit Preprocessing Only on Eligible Training Data

Our raw feature values are 10 through 21 in row order. Masking rows 2 and 7 makes their values missing. In the first fold, eligible training rows 0, 1, and 2 contain 10, 11, and missing. A mean imputer fitted there learns 10.5.

When applied to test rows 6 and 7, it produces 16 and 10.5. Both test rows remain in the evaluation population. The executed test checked the fitted mean against eligible training values and verified that no test row disappeared.

Scikit-learn recommends learning preprocessing from training data and applying the learned transformation to test data. A pipeline helps preserve that separation during cross-validation. Common pitfalls and data leakage

However, a pipeline cannot infer that a warehouse column arrived late. If you feed it a future-informed feature, correctly fitting the scaler or imputer does not repair the underlying snapshot. Availability filtering and train-only fitting address different failure modes.

The imputer here is a demonstration of fold-local preprocessing, not a recommendation that means are always appropriate for time series. A forward fill also needs scrutiny: it must use an earlier available value from the correct entity, without crossing an impermissible boundary.

Match the Test Block to the Forecast Horizon

Two test rows do not mean a two-day forecast. Each row in our fixture requests a three-day count, and the origins are irregular. The January 8 and January 9 targets overlap; that does not automatically mean training leakage, but their errors may be dependent.

Keep distinct the horizon of each prediction, the spacing between prediction origins, and the calendar period covered by a fold. If the product forecasts seven steps at once, evaluate that behavior instead of repeatedly supplying newly observed actuals as if each step were an independent one-step forecast. Multi-step rolling-origin evaluation

In our stated sequential-scoring contract, later-origin features can be valid even though they were unknown at the fold's first fit. For a batch issued entirely on January 8, those same features would be invalid. The contract, not the DataFrame layout, decides.

Labels must also be mature when you compute evaluation metrics. A January 30 origin has its complete label on February 4 in this exercise. Calling its missing outcome zero on January 31 would introduce a separate measurement error, even if the train/test split were perfect.

Choose Expanding or Rolling History Deliberately

The candidate sets above expand as time progresses. That lets later fits use more history, subject to the availability filter. A rolling window instead limits how much old history is retained.

Choose between them based on the forecasting problem and deployment schedule. More history may improve estimation of stable patterns, while old regimes may be less relevant after a change. Compare choices using appropriate past validation periods rather than choosing whichever produces the best final holdout result.

For irregular records, “the last 100 rows” and “the last 90 days” are different training policies. If calendar coverage matters, implement explicit timestamp boundaries and inspect the resulting sample counts. Do not describe a row-count limit as a fixed-duration window.

The same reasoning applies to gaps. A timestamp rule tied to label maturity is more informative than copying a gap from another example. You still need to consider other sources of leakage, including shared entities and transformations that aggregate across the wrong boundary.

Explain Entity Overlap and Honest Metrics

Forecasting future observations for known entities and generalizing to entirely new entities are different tasks. A time split can be appropriate for the former while answering little about the latter. If deployment includes new stores or customers, design an additional evaluation that reflects that population.

Conversely, holding out entities without respecting time can still allow future information into a past prediction. Specify which forms of overlap are allowed. A generic “use GroupKFold” or “use TimeSeriesSplit” answer is incomplete without the intended prediction setting.

When reporting results, retain fold dates, horizons, sample counts, and missing-feature rates alongside the metric. A single average can hide a weak seasonal period or a shrinking evaluated population. Report horizon-specific error when different horizons support different decisions.

The official scikit-learn lagged-feature example demonstrates an overly optimistic shuffled-split result compared with time-aware evaluation. That is evidence from that example, not a numerical improvement we reproduced here. Lagged-feature forecasting example

Our fixture deliberately stops before training a forecast model. Its value is narrower: it gives concrete assertions that catch unavailable labels, invalid feature snapshots, and accidental preprocessing leakage before any accuracy claim is made.

Practice With Related Questions

Question	Practice focus
Build a time-series forecasting model	Define origins, horizons, and retraining
Detect leakage and evaluate a prediction model	Audit the information available at scoring
Compute leakage-safe rolling features in pandas	Check entity and timestamp boundaries
Impute missing values without leakage	Keep preprocessing within the training fold
Forecast response-rate trends with backtesting	Explain evaluation periods and uncertainty

Start with the leakage question and write the two inequalities before discussing models: training labels must be available by fit time, and features must be available at the prediction origin they represent.

Sources and Further Reading

• scikit-learn: TimeSeriesSplit
• scikit-learn: Lagged features for forecasting
• scikit-learn: Common pitfalls and data leakage
• Hyndman and Athanasopoulos: Time series cross-validation

Checked September 10, 2026. Local split and imputation checks used scikit-learn 1.7.2 on original illustrative data. Current stable documentation may describe a newer release. No forecasting accuracy experiment is claimed.