A power grid operator in Texas needs to forecast tomorrow's electricity demand within 2% — guess too low and brownouts hit, guess too high and they burn natural gas for nothing. A Walmart category manager needs to know how many turkeys to order for the week before Thanksgiving. A trading desk needs the variance of returns over the next 24 hours to size positions. None of these problems are i.i.d. — yesterday's value tells you almost everything about today's, the day-of-week effect is enormous, and a single shuffled train_test_split will silently leak the future into the past and make your model look ten times better than it is. This lesson teaches you how to handle data that arrives one timestamp at a time: how to spot stationarity, how to read ACF and PACF plots like radiographs, how ARIMA, Holt-Winters, Prophet, and lag-feature-fed XGBoost actually differ, and how to validate without cheating.
Learning Objectives
After this lesson, you will be able to:
Explain why time-series data violates the i.i.d. assumption and why standard k-fold cross-validation produces fantastically optimistic estimates
Decompose a series into trend, seasonality, and residual using additive and multiplicative models
Test for stationarity with the Augmented Dickey-Fuller (ADF) test and apply differencing to achieve it
Read ACF and PACF plots to choose the order (p, d, q) of an ARIMA model
Compare ARIMA, SARIMA, Holt-Winters exponential smoothing, Prophet, and lag-feature regression — and pick the right tool for the job
Validate forecasts honestly with TimeSeriesSplit, walk-forward validation, and forecasting-appropriate metrics (MAE, MAPE, sMAPE, MASE)
Financial risk. Almost every value-at-risk model in a bank starts with a GARCH or ARIMA fit on returns; getting stationarity wrong here means underestimating tail risk and blowing up a desk
Build this → Pull the classic airline-passengers dataset (or your own Spotify daily-streams export), decompose it, fit ARIMA + Holt-Winters + an XGBoost on lag features, and walk-forward validate all three on the last 24 months
Don't worry if "stationarity" sounds like physics — by the end of this lesson it will mean nothing more than "the series doesn't drift, doesn't change variance over time, and its autocorrelation depends only on the gap between two points, not where you are on the timeline." That's it. Three checkboxes.
Almost every other ML algorithm you have met so far assumes your training examples are independent and identically distributed (i.i.d.) — drawn from the same distribution, with no row depending on any other row. Time-series data violates this assumption violently.
Concretely, time-series data brings four inconveniences that ordinary tabular ML simply does not have to deal with:
Order matters. Row 5 must come before row 6. You cannot shuffle the rows.
Autocorrelation.y_t is strongly correlated with y_{t-1}, y_{t-7}, y_{t-365}, etc. Standard error formulas that assume independence will under-report the true uncertainty.
Non-stationarity. Means and variances drift. A model trained on 2018-2019 data may be calibrated for a world that no longer exists in 2024 (regime shifts, COVID, policy changes, viral TikTok trends).
Train/test split must respect time. The test set must come strictly after the training set in real time. Any leak of the future into the past — through preprocessing, through cross-validation folds, or through engineered features — produces a model that looks spectacular in development and humiliates you in production.
Here is the workhorse exercise — we load the classic 1949-1960 monthly airline-passengers series, decompose it both ways, and look at its ACF and PACF. This is the diagnostic ritual every time-series analyst performs before fitting a single model.
Loading visualization...
Look at the ADF p-values: the raw series is wildly non-stationary (p ≈ 0.99 — we cannot reject the unit-root hypothesis), one differencing helps but not enough, and after combining a first-order difference with a seasonal-12 difference we finally cross into stationary territory. That's the cookbook: difference until you reject H0, then model the residue.
You will hear two flavors of stationarity. The strict definition is rarely what people mean in practice:
Strict (strong) stationarity. The full joint distribution of (y_t, y_{t+1}, …, y_{t+k}) is invariant to shifts in t, for every k. This is too strong for any real dataset to satisfy.
Weak (covariance) stationarity.E[y_t] is constant, Var[y_t] is constant and finite, and Cov(y_t, y_{t+h}) depends only on the lag h, not on t. This is what ARIMA needs, and what every test you will run targets.
The standard test is the Augmented Dickey-Fuller (ADF) test. Its null hypothesis is "the series has a unit root" (i.e. is non-stationary). A p-value below 0.05 lets you reject that and call the series stationary. If you can't reject, you difference: replace y_t with Δy_t = y_t − y_{t-1} and test again. Most series become stationary after one or two regular differences plus (for seasonal data) one seasonal difference.
What Do You Think?
An ACF plot decays very slowly (still significant at lag 30+). The PACF cuts off sharply after lag 1. The raw series looks like a slow upward drift with no clear seasonality. What does this pattern suggest about the appropriate AR order?
The autocorrelation function (ACF) at lag h is the correlation between y_t and y_{t-h}. The partial autocorrelation function (PACF) is the correlation between y_t and y_{t-h}after removing the linear effects of y_{t-1}, …, y_{t-h+1}. Together they are how you read the "fingerprint" of a series:
ARMA combines both — y_t is a function of past values and past errors. ARIMA adds an "I" for Integrated, which is just a fancy word for "we differenced the series d times before fitting":
p. Number of autoregressive lags (read from PACF)
d. Number of times the series was differenced to make it stationary (read from ADF test)
q. Number of moving-average lags (read from ACF)
For monthly data with a yearly cycle, you almost always want SARIMA(p,d,q)(P,D,Q)[s], which adds a second (P,D,Q) block operating at seasonal lag s (12 for monthly-yearly, 7 for daily-weekly, 24 for hourly-daily, etc.).
Simple exponential smoothing (SES). Forecast = weighted average of past observations with exponentially decaying weights. Good for series with no trend, no seasonality.
Holt's linear method. SES + a separate exponentially-smoothed trend term. Good when there's a linear trend.
Holt-Winters. Holt + a seasonal term. Comes in additive and multiplicative flavors. This is the workhorse for monthly retail data.
Released by Facebook (now Meta) in 2017, Prophet decomposes a series as y(t) = g(t) + s(t) + h(t) + ε:
g(t). A piecewise-linear or logistic trend with automatically detected changepoints
s(t). Fourier-series seasonality at multiple periods (yearly, weekly, daily)
h(t). A regression on user-supplied holiday indicators
ε. Gaussian noise
Prophet's two killer features compared to ARIMA: it handles missing data gracefully and it lets you supply a list of business holidays as a first-class input. Whether you should reach for it instead of SARIMA depends on what your series looks like.
Quick check
You're forecasting daily sales for a retail chain. The series has strong yearly seasonality, a clear weekly pattern (Saturdays peak, Tuesdays trough), several years of history with occasional missing days, and the marketing team can hand you a CSV of every major US holiday plus the chain's promotional calendar. Which model is the most natural fit?
What Do You Think?
Why is standard k-fold cross-validation wrong for time series?
Classical models like ARIMA assume a parametric form. The pragmatic modern alternative is to engineer lag features and throw a gradient-boosting model at the problem:
lag_1, lag_7, lag_14, lag_28, lag_365 — the value at those past timestamps
rolling_mean_7, rolling_mean_28 — moving averages
rolling_std_7, rolling_std_28 — rolling volatility
day_of_week, month, is_holiday — calendar features
exogenous covariates — weather, promotions, price changes, anything else you have
You then call this a tabular regression problem and reach for XGBoost, LightGBM, or CatBoost. This has become the default winning strategy in essentially every Kaggle time-series competition since about 2018 (M5 retail forecasting, Walmart, Rossmann, etc.), beating both ARIMA and Prophet by wide margins when there is enough data.
The exercise below fits ARIMA, Holt-Winters, and an XGBoost-on-lags model to the airline-passengers series, walk-forward validated on the last two years.
Loading visualization...
A few things to notice about the cross-validation block above:
The fold uses an expanding window — each fold uses more training data than the previous one, just like in production. The alternative is a sliding window (fixed-length training history that rolls forward), which is better when you suspect the dynamics change over time and old data hurts more than it helps.
The ML model uses iterated 1-step forecasting: it predicts one step, feeds the prediction back in, and predicts the next. This is honest but the errors compound. An alternative is to train a separate model per horizon (a direct-strategy multi-output model).
Ridge is a stand-in for XGBoost — XGBoost runs in Pyodide but only after a hefty wasm download; the workflow and the diagnostic story are identical and that is the load-bearing point of the exercise.
The pragmatic guidance: report MAE in the original units to your business stakeholders (they understand "off by 4,000 widgets"), report MASE to other modelers (it normalizes against a sensible baseline, so values are comparable across series of different scales), and reach for sMAPE only when you actively need a unitless metric and MAPE is misbehaving because of values near zero. RMSE is fine but punishes large errors quadratically — useful when one big miss is catastrophic, distorting when you have a long tail of large but acceptable misses.
Time series breaks the i.i.d. assumption. Order matters. Autocorrelation is signal. Train/test splits must respect time, and standard k-fold cross-validation produces dishonestly optimistic estimates.
Stationarity is the prerequisite for ARIMA. Test with ADF, achieve it with differencing. A series is weakly stationary if the mean and variance are stable and the autocovariance depends only on the lag, not on the timestamp.
ACF and PACF are diagnostic plots. ACF cuts off at lag q → MA(q). PACF cuts off at lag p → AR(p). Slow decay in either means you need to difference more.
Pick the model by data shape, not by hype. ARIMA and Holt-Winters dominate short univariate series; Prophet dominates daily business data with holidays; gradient-boosted lag features dominate when you have rich covariates; deep models dominate when you have many series or long-context dependencies.
Validate honestly. Walk-forward (TimeSeriesSplit) with an expanding or sliding window, MAE in original units, MASE relative to a seasonal-naive baseline. A model that doesn't beat MASE = 1 is worse than telling a stakeholder "use last year's number."
An ACF plot shows the autocorrelation at lag 1 is 0.95, at lag 2 is 0.90, at lag 12 is 0.65, at lag 24 is 0.45 — values stay significant for dozens of lags. What is the most likely problem?
You can now forecast a series and defend the forecast. Next up: the Deep Learning track, where recurrent and attention-based networks learn the lag structure for you — but only if you've already internalized why ignoring time is the most expensive mistake in applied ML.