10M transactions, 50 known frauds — and you need to ship a detector by Friday. The trick isn't a better classifier; it's flipping the problem entirely. By the end, you'll understand why anomaly detection succeeds where supervised learning silently fails.
Learning Objectives
After this lesson, you will be able to:
Distinguish anomaly detection (where anomalies are the SIGNAL you want to find) from outlier cleaning (where outliers are NOISE you want to remove) — and pick the right framing for your problem
Choose between supervised, semi-supervised novelty detection, and unsupervised regimes based on whether you have labels for normal data, anomalous data, both, or neither
Apply Isolation Forest, One-Class SVM, GMM-based density scoring, and LOF — and know which one to reach for based on data shape, dimensionality, and contamination rate
Tune the decision threshold and contamination rate using precision-at-k, downstream cost, and human-in-the-loop labeling instead of pretending you have ground-truth labels you don't
Don't worry if the math feels heavy at first — the algorithms here are surprisingly intuitive once you see them in action.
The credit-card-fraud framing: You have 10 million transactions and only 50 known frauds. A supervised classifier is hopeless — 50 labels can't train anything reliable. So you flip the problem: train a model that learns the shape of the 9,999,950 normal transactions. Anything that looks unusual under that model gets flagged. The known frauds become validation labels for tuning the threshold, not training labels for fitting the model. This is the heart of anomaly detection — and why Isolation Forest, One-Class SVM, and GMM-density scoring exist.
The simplest form of "looks like a stranger" is the 3-sigma rule for univariate data: under a Gaussian, ~99.7% of normal points sit within ±3 standard deviations of the mean, so anything beyond that is rare enough to flag. Modern detectors generalize this idea to high-dimensional, non-Gaussian data, but it's worth seeing the 1D version first.
Normal distribution & the 3-sigma rule — the simplest possible anomaly thresholdInteractive
Loading visualization...
Try this: Slide the mean and variance around. Points beyond μ ± 3σ have density < 0.3% under a Gaussian — that's the classical "3-sigma outlier" rule. Now imagine doing this in 50 dimensions with non-Gaussian marginals and overlapping subpopulations. That's why we need Isolation Forest, One-Class SVM, and LOF instead of just z-scores.
The supervised case is rare for real anomaly problems because you almost never have enough labeled anomalies. Most production systems use semi-supervised novelty detection (you have a clean baseline) or unsupervised methods with a contamination prior.
The genius of Isolation Forest is its inversion of the usual approach. Most algorithms try to model the dense normal region — Isolation Forest models how quickly a point can be separated from the rest.
s(x,n)=2−c(n)E[h(x)]wherec(n)=2H(n−1)−n2(n−1)
Why it works in high dimensions: Random feature splits naturally pick up multivariate structure. Adding noise dimensions hurts less than it does for distance-based methods (KNN/LOF), because the trees just stop using those features.
Practical knobs
n_estimators=100 is usually fine; more trees = more stable scores but slower
contamination is your prior: what fraction of the dataset do you think is anomalous? Wrong by 10x → predictions still ranked correctly, but the threshold cutoff is wrong
max_samples=256 (default auto) bounds tree depth; the original paper showed 256 is enough for huge datasets
#Worked Example: Finding 1 Outlier in 10 Normal Points
Suppose you have 10 spending records (daily $ amount): [12, 15, 18, 11, 14, 17, 13, 16, 12, 850]. The last one ($850) is the obvious outlier.
A single isolation tree picks a random feature (here only one) and a random threshold within the min-max range. The min is 11, max is 850 — so the first random threshold is uniformly drawn from [11, 850]. With probability (850−18)/(850−11) ≈ 99%, the random threshold lands above 18 — splitting [850] off into its own leaf on iteration 1. So path length h(850) = 1 most of the time.
For a normal point like 13, the random threshold has to keep landing between the data points around it (12, 14, 15, …) to isolate it. That requires roughly log₂(10) ≈ 3.3 random splits on average. So h(13) ≈ 3.
Anomaly score s(x, n=10) = 2^(−E[h(x)]/c(10)) where c(10) ≈ 3.75 (the normalizer).
s(850) ≈ 2^(−1/3.75) ≈ 0.83 (anomaly!).
s(13) ≈ 2^(−3/3.75) ≈ 0.57 (borderline, but with 100 trees this stabilizes near 0.5).
This is why Isolation Forest works: anomalies have short path lengths because random splits happen to peel them off fast. No density estimation, no kernel — pure path-length statistics.
When to reach for One-Class SVM over Isolation Forest
You genuinely have only normal data to train on (semi-supervised regime)
The normal region has a complex non-convex shape that benefits from the RBF kernel
You have under ~10K training points (One-Class SVM is O(n²) at training time and does not scale)
When to skip it
More than ~50K training points → Isolation Forest is faster and usually as accurate
High dimensionality (>50) → kernel methods suffer; trees handle dim-curse better
What Do You Think?
You have 5 million unlabeled credit card transactions per day, no labels for fraud. You need a first-pass detector that runs nightly. Which algorithm do you reach for first?
The answer is Isolation Forest. One-Class SVM cannot scale past ~10K points without approximations. GMMs assume a parametric form (Gaussian-shaped clusters) that rarely matches transaction data. An autoencoder works but is overkill for a nightly batch job — start simple. In real fraud teams, Isolation Forest is the universal first pass; deeper models come after you have labeled some flags.
#GMM-Based Anomaly Detection: The Probability Path
If you have already fit a Gaussian Mixture Model to your data, you almost get anomaly detection for free.
Strengths: Probabilistic — you get an actual likelihood, not a kernelized "distance to boundary." Easy to combine with prior beliefs (Bayes). Handles cluster structure naturally — if normal data has 5 modes, fit n_components=5 and each mode gets its own anomaly threshold implicitly.
Weaknesses: Assumes Gaussian-shaped clusters. Fails on heavy-tailed data, manifold-shaped data, or anything where the "normal" region is genuinely non-Gaussian. Sensitive to dimensionality (covariance estimation gets unstable above ~30 dims unless you use covariance_type='diag' or 'tied').
LOF (Breunig et al. 2000) flags points whose local density is much lower than that of their neighbors. This catches anomalies that are far from THEIR neighborhood, even if other parts of the dataset are even sparser.
LOFk(x)=∣Nk(x)∣∑y∈Nk(x)ρk(x)ρk(y)
When to use LOF over Isolation Forest: when your normal data has regions of genuinely different density (e.g., a customer segmentation with sparse luxury buyers AND dense everyday buyers). Global methods like Isolation Forest will mistakenly flag the sparse-but-normal luxury cluster.
When to skip it: large datasets (LOF is O(n²) for the kNN step) and high-dim spaces (kNN distance metric breaks down past ~30 dims).
A neural autoencoder learns to compress and reconstruct normal data. Anomalies don't follow the patterns the encoder learned, so they reconstruct poorly — high reconstruction error = anomaly. Full coverage is in track-04-deep-learning. Use it when:
You have huge volumes of normal data (>100K samples)
Inputs are high-dim (images, time-series windows, embeddings)
You can afford a GPU for training
For tabular data under 1M rows, classical methods (Isolation Forest first) are still usually faster, more interpretable, and competitive in accuracy.
Now run both Isolation Forest and One-Class SVM on the same synthetic dataset (normal blobs + injected anomalies) and compare recall and false-positive rate at a fixed contamination prior.
Loading visualization...
Quick check
On the playground above, you change the contamination parameter from the true 4.8% to 20%. What happens to recall and the false-positive rate?
Tests · Verify all three methods produce a precision @ k > 0.5 on this simple two-blob dataset. Try varying contamination and observing how it shifts.
Visualize how anomaly scores cluster — see the normal data densely scored and anomalies on the high-score tailInteractive
Loading visualization...
Quick check
A new fraud team has 50 confirmed fraud labels from the last year and 10 million unlabeled transactions. They ask whether to train a supervised classifier (XGBoost) or an unsupervised anomaly detector (Isolation Forest). What's the right framing?
Quick check
When does Isolation Forest typically outperform GMM-based density scoring for anomaly detection?
Quick check
What does the 'contamination' parameter in scikit-learn's IsolationForest actually do?
#Picking a Detector: Isolation Forest vs One-Class SVM vs GMM vs LOF
Dataset shape
First choice
Reason
> 100K rows, mixed numeric features, no labels
Isolation Forest
Scales linearly, handles high dim, contamination prior built-in
< 10K rows, semi-supervised (have only normal data), complex shape
One-Class SVM (RBF)
Tight kernel boundary; O(n²) but n is small
Already have a fitted GMM, or care about p(x) probabilities
GMM (negative log-likelihood)
Reuses the model; probabilistic scores compose well with priors
Density varies across regions (sparse cluster + dense cluster, both normal)
LOF
Local density comparison catches relative outliers
> 100K rows, deep features (images, embeddings, time-series windows)
Autoencoder reconstruction error
Trees suffer on high-dim continuous; covered in track-04
Rule of thumb: if you don't know which to pick, start with Isolation Forest and contamination tuned to reviewer capacity — it scales, has no kernel mysteries, and handles 90% of tabular use cases.
Anomaly detection is the inverse of outlier cleaning. Same algorithms, opposite intent: in this lesson the weird points are the signal you build a model to find, not noise you remove
Pick the regime by what labels you have. Supervised is rarely available; semi-supervised novelty detection (train on normals only) and unsupervised (assume contamination) are the realistic options
Isolation Forest is the universal first pass for tabular data. It scales to millions of rows, handles high dimensionality better than distance-based methods, and has a clean contamination prior baked in
One-Class SVM, LOF, and GMM each have a sweet spot. One-Class SVM for small clean training sets with complex shapes, LOF for varying-density datasets, GMM when you need probabilistic scores or already fit a mixture for clustering
Tuning the threshold is the real production work. Accuracy is meaningless on imbalanced data; use precision @ k tied to your reviewer capacity, PR-AUC, downstream cost, and human-in-the-loop labeling
You are deciding between an outlier-detection lesson approach (track-02) and an anomaly-detection lesson approach (this lesson) for a credit-card fraud system. What's the right framing?
You now know how to find the rare points that matter. Next, you will learn how to make any model better with the single highest-leverage practical skill in classical ML: hyperparameter tuning — Grid, Random, Bayesian, and Hyperband, with Optuna in real code.
You have ~hundreds of labeled anomalies
Supervised classifier (XGBoost + class_weight)
Labels are the strongest signal — use them when you have them