"Time series breaks the one assumption every standard ML algorithm depends on: that rows are independent and identically distributed. Shuffle a time series and you're literally training tomorrow's model on the future and testing it on the past. That's why every junior ML engineer's first 'amazing' forecasting result is wrong — the leakage is in the split, not the model."
Learning Objectives
After this lesson, you will be able to:
Explain why time series data breaks the i.i.d. assumption that most ML methods rely on, and why ordinary random shuffling silently leaks the future into the past
Resample irregular timestamps onto a regular grid using downsampling (aggregation) and upsampling (interpolation), and pick a sensible strategy for gaps and missing intervals
Engineer the three workhorse temporal features — lags (y_{t-1}, y_{t-7}, y_{t-365}), rolling and expanding windows, and cyclical sin/cos encodings — without leaking future information
Diagnose stationarity with the ADF test and seasonality with STL decomposition, then apply differencing or seasonal adjustment to give your model a stable target
Set up a walk-forward / time-series split that evaluates a forecaster the way it will actually be used in production, and recognize when a feature store with point-in-time correctness is required
Build this --> Take 6 months of your own step-count, screen-time, or sleep data; resample it to a regular daily grid, build lag-7 and rolling-7-day-mean features, sin/cos encode day-of-week, and predict tomorrow's value with a simple gradient boost; compare a random-shuffled CV score against a walk-forward CV score and watch the random one lie to your face
Don't let the math notation scare you — once you see lags as "what happened yesterday" and rolling windows as "the average of the last week," everything clicks.
Time series shows up everywhere ML touches reality: demand forecasting, fraud detection, recommendation freshness, sensor data, server-load prediction, click-through-rate over time, weather, finance, healthcare vital signs. A surprisingly large fraction of "tabular ML" projects in production are time series in disguise -- and they fail in the exact same way every time, because someone treated them as i.i.d. data.
Real-world timestamped data is almost never on a clean regular grid. Sensors miss readings. APIs are rate-limited. Users arrive in bursts. Database failovers cause minute-long gaps. Before any model touches the data, you have to commit to a grid: hourly? daily? 5-minute buckets?
Downsampling -- you have data at a finer granularity than you want. Aggregate up: sum the orders within each hour, average the temperatures, take the last status flag of each minute.
Upsampling -- you have data at a coarser granularity than you want. Either interpolate (linear, spline, time-weighted) or forward-fill (carry the last known value). For irregular gaps inside a downsample, you usually forward-fill state variables and treat numeric variables as missing.
ythour=agg({ys:s∈[t,t+1h)})
What Do You Think?
Your hourly retail-sales data has gaps from system outages -- about 3% of hours are missing entirely. The next ML stage is gradient boosting on lag features. What should you do?
The right answer is the fourth one, and it is the answer almost no beginner picks. Dropping rows breaks every lag feature you compute (lag_1 no longer means "1 hour ago"). Forward-filling pretends a sale value carried through an outage. Linear interpolation invents data. Inserting NaN plus an indicator column is honest: the lag features still align to real clock time, gradient boosting handles NaN natively, and the model can learn that "was-missing" itself predicts the recovery pattern.
A lag feature is the value of a variable from k time steps ago.
lagk(y)t=yt−k
Which lags should you create? A pragmatic checklist:
lag_1 -- almost always useful; today is the best predictor of tomorrow
lag_7 for daily data, lag_24 for hourly -- captures weekly / daily seasonality
lag_365 for daily data with multi-year history -- captures yearly seasonality
lags at the forecast horizon -- if you predict 7 days ahead, lag_7 is needed because at prediction time lag_1 through lag_6 are not yet observable
lag_1 of the residual after detrending -- captures the auto-correlation that remains after seasonality is removed
Two common multipliers are: lags of the target (the thing you're forecasting), and lags of covariates (other variables that move with the target -- temperature lagging into ice-cream sales, marketing spend lagging into conversions).
A rolling window computes a statistic over the last w observations.
μt(w)=w1i=1∑wyt−i
The min_periods parameter is the unsung hero: without it, the rolling window is NaN for the first w-1 rows of your dataset (you do not have w past values to average). With min_periods=3, you get a usable feature as soon as 3 observations have accumulated -- much better than discarding the first month of data.
An expanding window is the same idea, but the window grows over time: at row t the expanding mean includes everything from the start up to t-1. This captures the long-run baseline.
Hour-of-day, day-of-week, month-of-year all loop. Hour 23 is right next to hour 0, but if you encode hour as an integer and feed it to a linear model, the model thinks hour 0 and hour 23 are 23 units apart. The fix is to put the cyclical variable on a circle.
hsin=sin(242πh),hcos=cos(242πh)
Always pair sin and cos -- one alone is ambiguous (sin(2pi h / 24) takes the same value at hour 6 and hour 18). Tree-based models technically handle the integer encoding fine because they make threshold splits, but the sin/cos pair is one of those features that costs nothing and helps every model class, so most production pipelines just always use it.
Holiday and event features ride alongside cyclical encodings. Black Friday, Eid, Diwali, Lunar New Year, the Super Bowl -- these are all huge effects that no calendar encoding will capture by itself. Standard practice: maintain a holiday calendar per country, build a one-hot column per important holiday, and a days_to_next_holiday / days_since_last_holiday numeric pair.
A series is stationary when its statistical properties (mean, variance, autocorrelation structure) do not change over time. Many classical methods (and a surprising number of modern ones) assume stationarity.
yt′=yt−yt−1
The Augmented Dickey-Fuller (ADF) test is the standard quick check: if p < 0.05, you can treat the series as stationary; otherwise, difference and re-test.
Seasonality decomposition with STL pulls a series apart into trend + seasonal + residual:
pythonrunnable cell
1
2
3
4
5
from statsmodels.tsa.seasonal import STL
result = STL(df['sales'], period=7, robust=True).fit()
df['trend'] = result.trend
df['seasonal'] = result.seasonal
df['resid'] = result.resid
Treating the trend, seasonal, and residual components as separate features (or forecasting them separately and recombining) is one of the most reliable accuracy wins in classical forecasting. STL also makes anomaly detection trivial: a residual point more than 3 standard deviations from zero is the time series equivalent of an outlier.
Pull a raw series apart into its trend, seasonal, and residual layers and watch how each piece adds back to the original.
Loading visualization...
Try it: Notice how mean and variance shift when a series is non-stationaryInteractive
Loading visualization...
A non-stationary series often looks like a normal distribution if you blur your eyes, but its mean drifts upward, downward, or seasonally. Take a stationary series and slide a window across it: every window has roughly the same mean and spread. Take a non-stationary series and do the same thing: each window tells a different story. That is what you are looking for visually before you run an ADF test.
Each fold mimics the production setup: you only ever know the past. sklearn's TimeSeriesSplit implements exactly this; you'll see it again in the Cross-Validation Strategies lesson, where we cover the gap parameter, expanding vs sliding windows, and nested CV for time series. For now: just remember that random K-fold is wrong for time series, and that the "correct" CV score is almost always lower (and more honest) than the random-shuffle score.
When you need to predict not just the next step but the next h steps:
Recursive (iterated) forecasting: train a one-step model, then feed its predictions back in as lag_1 to predict step 2, etc. Simple, but errors compound.
Direct forecasting: train one model per horizon (model_h1, model_h2, ..., model_h7). More compute, less compounding error.
Multi-output forecasting: a single model emits h predictions at once. Modern deep models (DeepAR, N-BEATS, TFT) work this way.
Pick recursive when h is small and your one-step model is strong. Pick direct when h is large or when you care about long-horizon accuracy more than short-horizon. Pick multi-output when the deep-learning library makes it easy and you have enough data.
Time series break i.i.d. -- order is information, and any data prep step that ignores order (random shuffling, centered rolling windows, training on the future) silently leaks the answer into the input
Get onto a regular grid first -- pick a frequency (hourly, daily) and resample with the right aggregation per column type: sum for counts, mean for measurements, last for state, max for risk metrics
Three feature families do most of the work -- lags (lag_1, lag_7, lag_365), rolling and expanding windows (mean, std, max), and cyclical sin/cos encodings of hour/day/week/month -- always computed with an explicit .shift(1) so the current row is excluded
Stationarity and seasonality are diagnostic, not optional -- ADF test for stationarity, STL for trend/seasonal/residual decomposition; differencing and seasonal adjustment are cheap and almost always help classical and tree-based models
Evaluate with walk-forward, not K-fold -- random shuffling in CV inflates accuracy for the same reason it does in train/test split; production will only ever see chronological data, so your CV must too
You're predicting daily sales. You write `df['sales_roll7_mean'] = df['sales'].rolling(7).mean()`. Why is this dangerous if `sales` is also your target?
The playground below builds the canonical time-series feature set -- lag features and rolling means -- from scratch on a small noisy sine wave, then trains a tiny model with a strict walk-forward split. Note the test MAE compared against a "predict yesterday" naive baseline.
Loading visualization...
Time-aware features are the single biggest predictor of whether a "tabular" project succeeds or silently overfits to the future. Next up: Imbalanced Datasets — what to do when one class dominates 99% of your rows and accuracy stops being a useful metric.