Data Quality & Cleaning

"'Garbage in, garbage out' is the oldest ML cliché, but in production it's a law of physics. Anthropic's training run on Claude 3 spent more compute on data quality filtering than on the model itself. Andrew Ng's 2024 'Data-Centric AI' talk made it official: 80% of ML engineering effort, in any serious team, is data quality work. The model architecture you choose matters; the data you feed it decides whether you ship."

Learning Objectives

After this lesson, you will be able to:

Spot the four biggest data problems: missing values, outliers (weird extreme values), duplicates, and inconsistencies
Classify missingness as MCAR, MAR, or MNAR — because the type of missingness determines which imputation strategy is safe to use
Pick the right fix for each problem — fill in blanks, remove bad rows, cap extreme values, or deduplicate — based on why the problem happened
Build a step-by-step data cleaning process that keeps a record of every change so you can always explain or undo what you did, and measure quality with completeness, uniqueness, and validity scores

Prerequisites

Before You Start

Review

Why

Why This Matters

Why YouTube sometimes recommends random garbage (bad training data) — when mislabeled videos, spam accounts, and bot-generated watch-time leak into YouTube's training set, the recommendation AI learns the wrong patterns; that bizarre mukbang in your gaming feed is not a bad algorithm — it is bad data that poisoned a good algorithm
Why Snapchat and Instagram filters occasionally glitch on your face — face-filter models are only as good as the face data they trained on; if the training set has mislabeled landmarks, duplicate images, or inconsistent lighting labels, the filter will warp your nose instead of your ears
Cleaning data beats upgrading the AI — Google found that fixing data quality issues improved model accuracy 2-5x more than switching to fancier architectures; the "boring" work of removing duplicates and fixing missing values is secretly the most powerful move in all of AI
Build this → Take a messy dataset of movie or game ratings (with missing scores, duplicate entries, and obvious outliers like a rating of 999) and build a data quality checker that flags every issue — then compare how a simple model performs on dirty vs. clean data

Don't worry if this feels tedious — cleaning data is not glamorous, but it is genuinely the highest-impact skill in all of ML. Once you get the hang of it, it becomes second nature!

Strategy	When to Use	Trade-offs
Drop rows	Few missing rows (<5%), MCAR	Lose data, may introduce bias if not MCAR
Drop columns	>50% missing, feature is not important	Lose information
Mean/median imputation	Numeric, MCAR, quick baseline	Reduces variance, distorts correlations
Mode imputation	Categorical variables	Overrepresents the most common category
KNN imputation	MAR, sufficient nearby data	Computationally expensive, assumes local similarity
Model-based imputation	MAR, complex relationships	Best accuracy, most complex, risk of data leakage
Indicator variable	MNAR, missingness is informative	Adds a column; lets the model learn that "missing" is meaningful

Data Quality & Cleaning

After this lesson, you will be able to:

Before You Start

Your Reflection

Continue Learning

Discussion

#Garbage In, Garbage Out — But HOW?

#The Four Horsemen of Bad Data

#1. Missing Values

Types of missingness

Strategies for missing values

#2. Outliers

Detection methods

Strategies for outliers

#3. Duplicates

Types of duplicates

#4. Inconsistencies

#A Systematic Cleaning Pipeline

#Try It Yourself

#Deep Dive: Imputation Strategies

#Measuring Data Quality

#The Cost of Not Cleaning

#Key Takeaways

#Quick Check

Deep Dive

Related Lessons