Before any model fits anything, someone calls df.describe(). Mean, median, std — these four numbers diagnose 90% of data problems: skew, outliers, scale mismatches, broken pipelines. The fanciest deep network can't save you from skipping this step. Statistics isn't homework — it's the first line of defense against silently broken ML.
Learning Objectives
After this lesson, you will be able to:
Calculate the average, middle value, most common value, and spread of a dataset -- and know when to use each one
Tell apart the four most common data patterns (bell curve, yes/no, counting successes, counting rare events) and spot real-world examples of each
Understand why the bell curve shows up everywhere in nature and AI, thanks to a powerful math rule called the Central Limit Theorem
Apply the 68-95-99.7 rule to identify outliers and perform z-score standardization, the preprocessing step that makes gradient descent converge faster
Statistics is not about math -- it is about storytelling with numbers. Every time you check your average screen time, compare your test scores to the class, or wonder if a game's matchmaking is fair, you are already doing statistics. This lesson just makes you precise about it.
This lesson covers the foundational statistics you need before diving into probability, inference, and machine learning. If you can add, subtract, and find a square root, you are ready.
Try it! Open the Python REPL (bottom-right of the screen: click Quick Actions, then Python) and type these lines yourself.
The mean (average) is the sum of all values divided by the count. It is the "balance point" of the data -- the value where, if you placed the data on a seesaw, it would balance perfectly.
The median is the middle value when you sort the data. Half the values are below it, half are above. Unlike the mean, the median is not affected by extreme values.
What Do You Think?
If the mean salary at a company is $500K, does that mean most employees earn around $500K?
Not necessarily. If the CEO earns $10M and 19 employees each earn $50K, the mean salary is ($10M + 19 x $50K) / 20 = $547,500. But the median is $50K -- and that represents the typical employee far better. This is exactly why economists report median household income rather than mean income. A handful of billionaires can distort the mean dramatically.
The mode is the value that appears most frequently. It is especially useful for categorical data (like "what genre of music do students prefer?") where computing a mean makes no sense.
Quick check
Which summary statistic should you report for the typical home price in a US city?
#Measures of Spread: How Spread Out Are the Scores?
Knowing the center is only half the story. Two datasets can have the same mean but look completely different. Imagine two classes both averaging 75: in one class, everyone scored between 70-80; in the other, scores ranged from 20 to 100. You need a measure of spread.
Variance measures the average squared distance from the mean. Standard deviation is the square root of variance -- it brings the units back to the original scale.
σ2=n1i=1∑n(xi−μ)2s2=n−11i=1∑n(xi−xˉ)2
nn.LayerNorm(dim) does exactly this — for each token, subtract the mean across dim and divide by the standard deviation. nn.BatchNorm1d does the same z-score, but across the batch dimension instead. Same arithmetic, different axis.
For data that follows a bell-shaped (normal) distribution, there is a powerful shortcut:
68% of values fall within 1 standard deviation of the mean
95% of values fall within 2 standard deviations
99.7% of values fall within 3 standard deviations
This means if the average test score is 75 with a standard deviation of 10, about 95% of students scored between 55 and 95. For normal data, anything outside 3 standard deviations is rare (~0.3%) — but the rule applies only to normal distributions.
#Probability Distributions: The Shape of Randomness
A probability distribution describes how likely each possible outcome is. It is the "shape" of your data. Different real-world processes produce different shapes, and recognizing the shape tells you a lot about the underlying process.
The normal distribution is the most famous distribution in all of statistics. You have seen it in class: the symmetric bell curve where most values cluster near the mean and extreme values are rare on both sides.
f(x)=σ2π1e−2σ2(x−μ)2
Why does the normal distribution show up everywhere? Heights, test scores, measurement errors, stock returns (roughly) -- the answer is the Central Limit Theorem, which we will get to shortly.
The simplest possible distribution. There are only two outcomes: success (1) or failure (0). Flipping a coin once. Clicking or not clicking an ad. A patient having or not having a disease.
What happens when you repeat a Bernoulli trial n times? The binomial distribution counts the number of successes in n independent trials. Flip a coin 10 times -- how many heads? Send 100 emails -- how many get opened?
The Poisson distribution models the number of times a rare event occurs in a fixed interval. How many customers arrive at a store per hour? How many server crashes per month? How many typos per page?
The viz below lets you scrub the parameters of the four named distributions side-by-side. Pay attention to shape (symmetric vs skewed), not just the center.
Loading visualization...
Try this: Set the Binomial to n=20, p=0.5 — symmetric and bell-like. Now slide p to 0.05 — extreme right skew, because almost every trial is a failure. Switch to Poisson(λ=1) — also right-skewed, with a fat right tail. The shape of a distribution is what mean and variance cannot tell you on their own; that is exactly why skewness and kurtosis exist.
df.describe() is the first thing a working ML engineer types on a new dataset. The cell below builds the same output by hand from a class of test scores, then plots the histogram and overlays the mean / median / one-σ band so you can see whether mean and median diverge.
Try it: Adjust the parameters and watch the distribution changeInteractive
Loading visualization...
Try this: Start with a normal distribution. Set the mean to 0 and standard deviation to 1 -- this is the "standard normal." Now increase the standard deviation and watch the bell flatten. Then switch to a binomial distribution: set n=10 and p=0.5 -- notice how it looks like a bell curve. Now set p=0.1 -- it becomes skewed right. The Poisson with lambda=1 is also skewed right, but as lambda increases it starts to look normal too. This is the Central Limit Theorem in action.
Take any distribution -- uniform, exponential, binomial, even a completely weird custom distribution. It does not matter what shape it is. The Central Limit Theorem works on all of them.
Draw a random sample of n values from that distribution. Compute the mean of your sample. This is one "sample mean." It is a single number -- the average of your random draw.
Do this again and again. Each time, draw n fresh values and compute the sample mean. After hundreds or thousands of repetitions, you have a collection of sample means.
Plot the distribution of those sample means. No matter what the original distribution looked like, the distribution of sample means will be approximately normal (bell-shaped). The larger n is, the more perfectly normal it becomes. This is the Central Limit Theorem -- the single most important result in statistics.
Before training any model, you compute descriptive statistics on every feature: mean, median, standard deviation, min, max, skewness. These numbers tell you whether features are on the same scale (if not, you need normalization), whether there are outliers (which might need clipping), and whether features are roughly normal (which some algorithms assume).
Many ML algorithms (gradient descent, KNN, SVMs) are sensitive to the scale of features. Standardization subtracts the mean and divides by the standard deviation: z = (x - mean) / std. This transforms every feature to have mean 0 and standard deviation 1. You cannot do this without knowing the mean and standard deviation first.
Values more than 3 standard deviations from the mean are rare (the 68-95-99.7 rule). This is the simplest anomaly detector: compute the mean and standard deviation, flag anything beyond 3 sigmas. Credit card fraud detection, server monitoring, and quality control all start here.
Linear regression assumes normally distributed errors. Naive Bayes often assumes Gaussian features. Understanding what distribution your data follows -- and what happens when it does not -- is the difference between a model that works and one that silently fails.
Tests · Verify mean is approximately 81, median is approximately 81.5, and std dev is approximately 8.8. Add an outlier and confirm median is more stable than mean.
Mean, median, and mode measure the center differently. The mean is sensitive to outliers, the median is robust, and the mode identifies the most common value; always check whether mean and median diverge, which signals skewed data
Standard deviation measures consistency. It quantifies how far values typically fall from the mean; small standard deviation means tightly clustered data, large means widely spread
The 68-95-99.7 rule gives instant intuition — For bell-shaped data, nearly all values fall within 3 standard deviations of the mean; anything beyond that is a potential outlier or anomaly
Different processes produce different distributions. Normal (sums of many factors), Bernoulli (yes/no), binomial (counting successes), and Poisson (rare events) each model different real-world phenomena
The Central Limit Theorem explains why normal distributions are everywhere. Average enough independent random values and the result is always approximately normal, regardless of the original distribution shape
A company reports a mean salary of $200K. The CEO earns $5M and the 49 other employees each earn $100K. What is the median salary?
Next up: Random Variables, Expectation & Variance. You will learn how to assign numbers to random outcomes and compute the long-run averages that underpin every ML loss function.