"Outliers are either the most valuable signal you have (fraud, anomaly detection, network intrusions) or the noise that breaks your model (mis-recorded inputs, sensor faults, transcription bugs). The hardest job in data science is knowing which. Drop them blindly and you delete the fraud signal; keep them blindly and they pull your regression line off the road. The 'right' answer always depends on what you're trying to predict."
Learning Objectives
After this lesson, you will be able to:
Detect outliers using the right statistical tool — Z-score for clean bell-shaped data, modified Z-score (with MAD) for skewed or heavy-tailed data, and the IQR rule when you cannot trust normality at all
Use multivariate methods — Mahalanobis distance, Local Outlier Factor (LOF), and Isolation Forest — to catch points that look normal on each axis individually but together form an impossible combination
Decide what to do with each outlier — drop it, cap it (winsorize), transform the variable (log), or keep it — based on whether the value is a data error, a rare-but-real event, or the actual signal you are trying to model
Avoid the four traps that ruin outlier work — removing points before splitting train/test, using mean-and-std on heavy-tailed data, deleting all extreme values without investigating, and treating fraud detection like data cleaning
Build this → Pull a year of your bank statements as CSV. Run an IQR detector on the "amount" column to find your weirdest transactions, then run a multivariate detector across (amount, merchant, day-of-week) — see which points look normal alone but flag in combination; that is exactly how a fraud-detection pipeline thinks
Don't worry if "outlier" feels like a fuzzy word — by the end of this lesson you will have four sharp definitions and a flowchart for when to use each.
There are three reasons a point might be flagged as an outlier, and your response depends entirely on which one you are looking at:
Data-entry error: Someone typed "999" for missing age. Drop it (or impute it).
Measurement error: A broken thermometer reported 200°C in your weather data. Drop it.
Rare-but-real event: A genuine $9,000 transaction (a wedding ring, an emergency vet bill, fraud). Do not drop it — this is often the signal you are trying to predict.
The hardest cases are when you cannot tell which of the three you have. That is when you reach for the multivariate methods later in this lesson.
#Statistical Methods: When You Trust the Distribution
The classic test: how many standard deviations is this point from the mean?
zi=σxi−xˉflag if ∣zi∣>3
Use Z-score only when: your data is roughly bell-shaped and you have already removed the most egregious typos. Otherwise the contamination problem kills you.
Replace the mean with the median and the standard deviation with the Median Absolute Deviation (MAD). Now a single huge outlier cannot poison the detector.
Mi=MAD0.6745(xi−x~),MAD=median(∣xi−x~∣)
Use modified Z-score when: your data might be skewed, contains real-but-rare extreme values, or you do not have time to verify normality.
Tukey's classic. No assumptions about shape — just look at the middle 50% and flag anything far from it.
\text{IQR} = Q_3 - Q_1, \qquad \text{flag if } x < Q_1 - 1.5 \cdot \text{IQR} \;\text{ or }\; x > Q_3 + 1.5 \cdot \text{IQR}
Use IQR when: you have no idea what shape the data is, or when you need a method that survives heavy skew without requiring you to compute a robust scale.
Try it: drag the distribution shape and watch which points each method flagsInteractive
Loading visualization...
What Do You Think?
A startup's salary dataset has values ranging from $40K to $200K, with one founder paid $1.2M. You apply the standard Z-score rule (|Z| > 3). What happens?
This is the snake-eating-its-tail problem. The single huge value pulls up both the mean and the standard deviation, so when you compute its own Z-score, the denominator has been inflated by the very outlier you are trying to catch. The modified Z-score (using median + MAD) does not have this problem — that is exactly why it exists.
#Multivariate Methods: When Single-Axis Detection Fails
Many outliers are not extreme on any single axis. They are extreme in combination.
A 6-foot-tall person is normal. A 100-pound person is normal. A 6-foot-tall, 100-pound person is an outlier. None of the univariate detectors above can find them.
Generalize the Z-score to multiple dimensions, accounting for correlations between features.
DM(x)=(x−μ)⊤Σ−1(x−μ)
The catch: Mahalanobis assumes a single Gaussian blob. If your data has multiple clusters, it fails — points between clusters look "central" by the global covariance.
Compare a point's local density to the density of its neighbors. If you live in a sparse area while your neighbors live in dense areas, you are flagged.
Use LOF when: your data has clusters of varying density and you want to catch local anomalies — "weird for its neighborhood" rather than "weird globally."
A different idea entirely: build random binary trees that split the data on random features at random thresholds. Outliers, by definition, get isolated in fewer splits, because they are sparse — there is "more room around them" for a random cut to separate them.
pythonrunnable cell
1
2
3
4
from sklearn.ensemble import IsolationForest
iso = IsolationForest(contamination=0.01, random_state=42)
labels = iso.fit_predict(X) # -1 = outlier, +1 = inlier
scores = iso.score_samples(X) # lower = more anomalous
Use Isolation Forest when: you have high-dimensional data, you do not know what shape it is, you do not know how many outliers to expect, and you need it to run fast on millions of rows. This is the modern default.
Learns a tight boundary around the bulk of the data; anything outside is an outlier. Powerful but slow on large datasets and very sensitive to the kernel choice. Rarely the right first tool today — Isolation Forest and LOF have largely replaced it for tabular data, though it is still used in network-traffic anomaly detection.
#The Decision Rubric: Drop, Cap, Transform, or Keep?
Detection is the easy part. The hard question is what to do with the flagged points. Here is the decision tree professional teams actually use:
1. Is the value impossible? (age = 999, temperature = 5000°C)
→ DROP (or impute) — it's a data error.
2. Is the value rare-but-real, AND is it the signal you want to predict?
→ KEEP — this is anomaly detection, not cleaning.
(e.g. fraud, equipment failure, rare disease)
3. Is the variable heavy-tailed (income, file sizes, response times)
AND your model assumes normality?
→ TRANSFORM (log, Box-Cox, Yeo-Johnson) — keeps all data, fixes the shape.
4. Is the value real but unrepresentative of the population
you'll deploy on?
→ CAP (winsorize at 1st/99th percentile) — keeps the row, neutralizes
the extreme.
5. Is the value real and representative? Don't touch it.
Tests · Verify each method returns a boolean mask of the same length as the input. Verify Z-score and modified-Z disagree on the synthetic data. Verify IQR(3.0) flags strictly fewer points than IQR(1.5).
Watch a dirty dataset get cleaned step-by-step — flag, decide, transformInteractive
Loading visualization...
For high-dimensional outliers, single-feature detectors are useless — you need to look at how points relate across features. The patterns of which combinations of features are missing or extreme tells you which detector to reach for.
Inspect missingness and extreme-value patterns across multiple columnsInteractive
The detector you reach for depends on the distribution shape. Z-score for clean bell curves, modified Z-score (with MAD) for skewed or heavy-tailed data, IQR when you cannot trust the shape at all, Mahalanobis or Isolation Forest when single-axis detectors miss the multivariate combinations
Detection ≠ removal. Once you have flagged a point, ask whether it is a data error (drop), a heavy-tail expression of the same variable (transform), an unrepresentative real value (cap), or the signal you are trying to predict (keep)
Z-score lies on the data it was meant to clean. The mean and standard deviation are themselves dragged toward the outliers, so the Z-score of the worst outlier comes back below 3 and your detector reports "all clear"; switch to median + MAD or IQR the moment you see a skewed histogram
Multivariate outliers are invisible to univariate methods. A 6-foot, 100-pound person is normal on each axis; only Mahalanobis distance, LOF, or Isolation Forest can catch the impossible combination, and combinations are where most production fraud and anomaly cases actually live
Fit detectors on the training set only, then apply to test. Fitting an outlier detector on the full dataset before splitting is the same kind of leakage as fitting a scaler on it; in cross-validation, the detector goes inside the sklearn Pipeline so it refits per fold
Your dataset of website session durations has a histogram with a long right tail (most sessions 1-5 minutes, but some go for hours). You apply the Z-score rule with |Z| > 3 and find zero outliers. What is most likely happening?
You can now spot the points that do not belong. Next up: Categorical Encoding — turning text labels like "Engineering" and "Marketing" into numbers your model can actually do math on, and avoiding the four encoding traps that quietly tank tree models and linear models in opposite ways.