"Fraud detection: 0.1% positive. Cancer screening: 1% positive. Click-through prediction: 2% positive. The most valuable ML problems are all imbalanced — and treating them like balanced classification is the #1 ML production failure mode. A 99% accurate model that never predicts the positive class is a 0% useful model. Imbalance isn't an edge case; it's the default in any problem worth solving."
Learning Objectives
After this lesson, you will be able to:
Spot when your dataset has way more of one class than another (like 99% not-fraud and 1% fraud) and measure how bad the imbalance is with a clear imbalance ratio
Fix imbalanced data using tricks like giving rare examples more weight (class_weight), creating synthetic examples (SMOTE/ADASYN), or removing excess majority examples (undersampling)
Pick the right success metric for imbalanced data — F1, AUC-ROC, and Precision-Recall AUC — because plain accuracy is a liar when one class dominates
Combine SMOTE with cleaning techniques (SMOTETomek, SMOTEENN) for better decision boundaries, and know when to reframe extreme imbalance as anomaly detection instead of classification
Don't worry if this feels tricky — class imbalance trips up even experienced engineers. The key insight is simple: when one class vastly outnumbers the other, your model takes the lazy shortcut of always guessing the common class. This lesson teaches you how to fix that!
When you train a standard classifier on imbalanced data, the following sequence happens:
Training sees mostly majority-class examples — if your dataset is 99% "not fraud," most batches contain almost no fraud examples
Gradient is dominated by majority class — the model updates primarily to classify "not fraud" correctly, since that is where most of the loss comes from
Minority class is ignored — the model learns that predicting "not fraud" for everything minimizes the loss function
Accuracy is misleadingly high — because 99% of predictions are correct by simply predicting the majority class
The result: a high-accuracy model that fails at its actual job.
import pandas as pd
import numpy as np
from collections import Counter
# Check class distribution
print(df['label'].value_counts())
print(df['label'].value_counts(normalize=True)) # as percentages
# Compute imbalance ratio
counts = df['label'].value_counts()
imbalance_ratio = counts.iloc[0] / counts.iloc[1]
print(f"Imbalance ratio: {imbalance_ratio:.1f}:1")
# Counter works for any sequence
print(Counter(y_train))
Try it! Open the Python REPL and type these lines yourself. Simulate an imbalanced dataset: from collections import Counter; import numpy as np; labels = np.array([0]*990 + [1]*10); print(Counter(labels)); print(f"Imbalance ratio: {990/10:.0f}:1") — see how extreme 99:1 looks!
Severity guidelines
Mild imbalance (< 4:1): Class weighting usually sufficient
Moderate imbalance (4:1 to 20:1): Class weighting or SMOTE
Severe imbalance (20:1 to 100:1): SMOTE + undersampling combination
Extreme imbalance (100:1+): Anomaly detection framing may be more appropriate than classification
Precision: Of all examples predicted positive, what fraction are truly positive?
Precision=TP+FPTP
Recall (Sensitivity): Of all truly positive examples, what fraction did you catch?
Recall=TP+FNTP
F1 Score: The harmonic mean of precision and recall:
F1=2⋅Precision+RecallPrecision⋅Recall
AUC-ROC: Area under the receiver operating characteristic curve. Measures discrimination ability across all thresholds (0 = random classifier, 1 = perfect classifier). Robust to class imbalance.
Precision-Recall curve: Better than ROC for extreme imbalance (< 1%). The PR curve focuses exclusively on the minority class performance and does not reward a model for correctly predicting the majority class.
What Do You Think?
Your fraud dataset is 1:500 imbalanced (0.2% fraud). You try class_weight='balanced' and SMOTE separately. SMOTE gives substantially better F1. Why might class weighting fail here?
At extreme imbalance ratios (1:500+), class weighting adjusts the loss scale but does not change the number of fraud examples the model sees. The model still processes 500 non-fraud samples for every 1 fraud sample per epoch. The weights compensate mathematically but the model may still fail to learn the diverse patterns of the minority class because it has not seen enough distinct fraud examples. SMOTE creates new synthetic minority samples, increasing the effective diversity of fraud patterns during training.
Class weighting is the simplest fix: tell the algorithm to pay more attention to minority class errors.
pythonrunnable cell
1
2
3
4
5
6
7
8
9
10
11
12
13
14
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
# sklearn: class_weight='balanced' automatically computes weights
# inversely proportional to class frequency
model = LogisticRegression(class_weight='balanced')
model.fit(X_train, y_train)
# Manual weights: give 10x penalty to minority class errors
model = RandomForestClassifier(class_weight={0: 1, 1: 10})
# In keras/tensorflow:
# class_weight = {0: 1.0, 1: 500.0}
# model.fit(X, y, class_weight=class_weight)
How it works: sklearn computes weight_class = n_samples / (n_classes * n_samples_in_class). A class with 100 samples in a dataset of 10,000 with 2 classes gets weight 10000 / (2 * 100) = 50. Its misclassification errors count 50x more in the loss function.
When class weighting works well:
Mild to moderate imbalance (up to ~50:1)
Limited time / simple pipeline (one parameter to add)
When the minority class has sufficient examples to learn from (even if underrepresented)
ADASYN (Adaptive Synthetic Sampling) is an improvement over SMOTE that generates more samples in regions where the model is confused:
pythonrunnable cell
1
2
3
4
5
6
7
8
# ADASYN: generates more samples near the decision boundary
# where classification is hardest
adasyn = ADASYN(
sampling_strategy='auto',
n_neighbors=5,
random_state=42
)
X_res, y_res = adasyn.fit_resample(X_train, y_train)
ADASYN vs SMOTE: SMOTE distributes synthetic samples uniformly across the minority class neighborhood. ADASYN concentrates synthetic samples near the decision boundary — the hard-to-classify region — which often gives better results when the boundary is noisy or complex.
Tomek links removes majority-class samples that are borderline near minority samples — cleaning the decision boundary without discarding distant majority samples:
pythonrunnable cell
1
2
3
4
5
from imblearn.under_sampling import TomekLinks
tl = TomekLinks()
X_res, y_res = tl.fit_resample(X_train, y_train)
# Only removes majority samples that are nearest neighbors of minority samples
The most powerful approaches combine SMOTE with cleaning of borderline majority samples:
pythonrunnable cell
1
2
3
4
5
6
7
8
9
from imblearn.combine import SMOTETomek, SMOTEENN
# SMOTE + Tomek: oversample minority, then clean borderline majority
smote_tomek = SMOTETomek(random_state=42)
X_res, y_res = smote_tomek.fit_resample(X_train, y_train)
# SMOTE + ENN: more aggressive cleaning than Tomek
smote_enn = SMOTEENN(random_state=42)
X_res, y_res = smote_enn.fit_resample(X_train, y_train)
Why combine?: SMOTE can create synthetic samples near the boundary that overlap with majority-class samples (noisy synthetic samples). Tomek links and ENN remove these borderline overlapping samples, giving a cleaner decision boundary.
Accuracy is a liar with imbalanced data. A model predicting the majority class always can have 99.9% accuracy while catching zero minority-class examples; use F1, AUC-ROC, or precision-recall AUC as your primary metrics
Detect imbalance before modeling. Compute class distribution and imbalance ratio; choose your strategy based on severity (mild: class weights; severe: SMOTE; extreme: anomaly detection framing)
Class weighting is free and should be your first try. class_weight='balanced' in sklearn adds a single parameter and significantly improves minority-class recall with no extra compute cost
SMOTE creates synthetic minority samples by interpolating between neighbors. It increases the diversity of minority-class training examples; use it after splitting (never before) and prefer imblearn's Pipeline for cross-validation safety
Always apply resampling inside the training fold only. SMOTE on the full dataset before splitting leaks test information into training and inflates evaluation metrics
A credit card fraud model has 99.9% accuracy on a 0.1% fraud dataset, but the business team reports it never catches any fraud. What is the most likely explanation?