Modern AI papers obsess over architecture; the real bottleneck is data. Cleaning, splitting, drift detection, monitoring — the field manual for the work that decides whether your model actually ships.

Track 02 · Data Foundations · 12 min

Garbage in, garbage out. The 80% of ML nobody talks about.

Modern AI papers obsess over architecture. The real bottleneck is data — collecting it, cleaning it, labeling it, splitting it, watching for drift. The ten ideas in this article cover the work that decides whether your model actually ships.

DataMLOpsFoundations

“The model gets all the credit, but the data does all the work.”
— Andrew Ng · Data-Centric AI movement, 2021

#The hook

Andrew Ng said it most clearly: "The model gets all the credit, but the data does all the work."

You can read every paper from NeurIPS, master every architecture, and your model will still embarrass you in production — because the data was wrong. 80% of professional ML work is data work. Not pretty, not glamorous, but the difference between a research demo and a system that survives contact with reality.

This is the field manual for that 80%.

#Why this matters in 2026 — the receipts

Where the time really goes

Data work is the actual job

80%

ML time spent on data

CrowdFlower 2016 + 2024 follow-up

$175B+

Data infrastructure market 2026

IDC

60%

Models that drift in prod

MLflow community survey 2025

50%

ML projects killed by data bugs

Gartner 2025

#The 30-second answer

Data Foundations is the discipline of getting raw, messy, real-world information into a shape your model can learn from — and keeping it that way.

It splits into four phases. Each phase has its own tools, traps, and best practices.

The pipeline

Four phases of data work

1. Ingest & explore

Get the data, look at it

Pull from APIs, databases, logs, spreadsheets. Then EDA — exploratory data analysis. Plot everything.

Pandas / Polars / DuckDB for tabular. Pyarrow for big files.
Always plot distributions, correlations, missing-value patterns BEFORE you train anything.
Most production bugs come from being surprised by the data.

2. Clean & engineer

Make it model-ready

Handle missing values, outliers, encoding, feature engineering. Where most of the time goes.

Median imputation for numerics, 'unknown' category for strings, but check that it doesn't break things.
One-hot vs ordinal vs target encoding for categoricals — the choice changes performance.
Domain features still beat learned features for tabular data. Always.

3. Split & validate

Honest evaluation

Train, validation, test. Cross-validation. Stratification. Group-aware splitting.

Random split is wrong for time series — you'd be predicting the past from the future.
Always stratify on the target if it's imbalanced. Most real-world datasets are.
Group-aware splitting prevents data leakage when one user has many rows.

4. Monitor & adapt

Survive in production

Detect drift. Track input distributions. Re-label. Retrain. Forever.

Data drift: input distributions shift. Concept drift: the relationship between inputs and outputs shifts.
PSI (Population Stability Index) and KS-test are the field-standard drift metrics.
Production ML is 90% monitoring, 10% modeling. Get used to it.

80%

of ML time goes to data work

Repeated study after study (CrowdFlower 2016, Anaconda 2024) puts data preparation at 60-80% of ML practitioner time. The fancy modeling part is the small remainder. If you want to ship AI that works in production, you spend most of your career here.

Anaconda 'State of Data Science' 2024

Vocabulary

Six data terms that show up daily

Concept

EDA

Exploratory Data Analysis — plot everything before modeling.

Like: A doctor's checkup before surgery.

e.g. Histograms, correlation plots, missing-value heatmaps

Concept

Train/val/test split

Hold out data the model never trains on, to measure generalization.

Like: Separate practice tests from the real exam.

e.g. 60/20/20 random split

Concept

Feature engineering

Transform raw inputs into model-friendly features.

Like: Prepping ingredients before cooking.

e.g. log_income, age_bucket, days_since_signup

Concept

Data drift

Input distributions change between training and production.

Like: Yesterday's map of a changing city.

e.g. Users got younger over six months

Concept

Concept drift

The relationship between inputs and outputs shifts.

Like: Spam tactics evolve — yesterday's filter misses today's.

e.g. COVID destroyed every demand-forecasting model in 2020

Concept

Feature store

Single store of features for both training and serving.

Like: One pantry for the kitchen and the truck.

e.g. Tecton, Feast, Hopsworks

#A real cleaning + EDA pipeline — runnable

Here's the workflow that handles 80% of real datasets, end-to-end, in one runnable cell:

pythonrunnable cell

import pandas as pd
import numpy as np

# Make a "messy" dataset that looks like real-world data
np.random.seed(42)
n = 500
data = pd.DataFrame({
    "age":         np.random.randint(18, 80, n),
    "income":      np.random.lognormal(10, 1, n).round(),
    "city":        np.random.choice(["NYC", "LA", "SF", None], n, p=[.35, .25, .25, .15]),  # 15% missing
    "subscribed":  np.random.choice([0, 1], n, p=[0.7, 0.3]),
})

# Inject 5% bad data — typical of real-world ETL
data.loc[np.random.choice(n, 25), "age"] = -1   # bad sentinel for "missing"
data.loc[np.random.choice(n, 15), "income"] = np.nan

# 1. EXPLORE
print("Shape:", data.shape)
print("\nMissing per column:")
print(data.isna().sum())
print("\nFirst 3 rows:")
print(data.head(3))

# 2. CLEAN
data["age"] = data["age"].replace(-1, np.nan)            # bad sentinel -> proper NaN
data["age"] = data["age"].fillna(data["age"].median())   # median imputation
data["income"] = data["income"].fillna(data["income"].median())
data["city"] = data["city"].fillna("unknown")            # explicit unknown category

# 3. ENGINEER FEATURES
data["log_income"] = np.log1p(data["income"])             # log-transform a skewed feature
data["age_bucket"] = pd.cut(data["age"], bins=[0, 30, 50, 100], labels=["young", "mid", "older"])

# 4. ONE-HOT ENCODE CATEGORICALS
data_encoded = pd.get_dummies(data, columns=["city", "age_bucket"], drop_first=True)

print("\nFinal shape:", data_encoded.shape)
print(f"Subscription rate: {data_encoded['subscribed'].mean():.1%}")
print(f"Columns ready for ML: {list(data_encoded.columns)}")

That cell does what 90% of real ML pipelines do, in 30 lines. Try modifying the missing-value strategy or the bucket boundaries and watch the output shift.

#The drift problem — and how to detect it

Models go stale because the world changes. The way to detect this without ground-truth labels is distribution comparison: pick a metric that compares "training data" to "today's production data" and alert when it drifts.

pythonrunnable cell

import numpy as np

# Train-time distribution (e.g., user ages from 6 months ago)
train_ages = np.random.normal(loc=42, scale=12, size=2000)

# Today's production distribution — the world has shifted slightly younger
prod_ages  = np.random.normal(loc=37, scale=14, size=2000)

# Population Stability Index — the field-standard drift metric
def psi(train, prod, bins=10):
    cuts = np.percentile(train, np.linspace(0, 100, bins + 1))
    cuts[0], cuts[-1] = -np.inf, np.inf
    train_pct = np.histogram(train, cuts)[0] / len(train) + 1e-6
    prod_pct  = np.histogram(prod,  cuts)[0] / len(prod)  + 1e-6
    return np.sum((prod_pct - train_pct) * np.log(prod_pct / train_pct))

score = psi(train_ages, prod_ages)
print(f"PSI score: {score:.3f}")
print("Interpretation:")
print("  < 0.10  no significant drift   ← all good")
print("  0.10–0.25  moderate drift       ← investigate")
print("  > 0.25  severe drift            ← retrain")

Run that. PSI quantifies how much the input distribution has shifted between train-time and production. Production ML teams compute this on a schedule for every important feature. It's 90% of the early-warning system.

#What's been built on solid data foundations

Where data discipline pays

Systems that lived or died by their data

Data + creativity

Netflix Prize

$1M

Prize for 10% improvement

Three years of competition, 40,000 teams. The winning team's edge was data engineering, not algorithm choice.

Feature engineering

Continuous data ops

Tesla Autopilot

4M+

Vehicles labeling for free

Every Tesla on the road continuously collects training data. The data flywheel is the product.

Data flywheel

Labeled dataset

ImageNet

14M+

Hand-labeled images

Fei-Fei Li's 2009 dataset created modern deep learning. The architecture (AlexNet) was 5%; the data was 95%.

Labeled scale

Dataset distribution

Hugging Face hub

100K+

Datasets hosted

GitHub for ML datasets. Search, version, fine-tune — all from one URL. Now reshaping how teams ship models.

Distribution

Curation

OpenAI GPT-4 prep

13T

Training tokens (curated, deduped)

Most of the GPT-4 effort was data curation: source filtering, deduplication, quality classifiers, content safety.

Curation at scale

Synthetic data

Anthropic Constitutional AI

100K+

AI-generated preference pairs

Used AI to generate training data for the same AI. Started a wave of synthetic-data techniques across the industry.

Synthetic data

#The 2026 frontier

1. Synthetic data. Increasingly mainstream. GPT-4 and Claude generate high-quality training data that real-world labels can't compete with on speed. 2. Vector + structured + lakehouse. Postgres with pgvector, DuckDB lake formats, Iceberg/Delta — the stack is consolidating around mixed engines. 3. Real-time feature stores. Tecton, Hopsworks, Feast — same feature for offline training and online serving, no skew. 4. Data contracts. Schema-as-code between producers and consumers. The 2025 movement to make pipelines stop breaking silently.

#Where to go next

Data Foundations track — 15 lessons covering pipelines, EDA, cleaning, feature engineering, drift detection.
SQL Mastery — most data work happens in SQL. Pair this with the SQL track.
Python Foundations — pandas, polars, NumPy live in Python.
Classical ML — apply your clean data to real models.

#Key takeaways

Recap

Key Takeaways

80% of ML work is data work. Cleaning, engineering, splitting, monitoring.
Four phases: ingest+explore → clean+engineer → split+validate → monitor+adapt.
Always EDA before you model. Plot everything. Find the surprises before the model does.
Random splits break time-series and grouped data. Use TimeSeriesSplit and GroupKFold.
Production ML is 90% monitoring. PSI and KS-test are the bread-and-butter drift metrics.
The 2026 frontier: synthetic data, real-time feature stores, data contracts, lakehouse architectures.

#References & further reading

Designing Machine Learning Systems by Chip Huyen — the bible for production ML data work.
Andrew Ng's Data-Centric AI talks (deeplearning.ai). Started the field-shift.
Feature Engineering for Machine Learning by Alice Zheng & Amanda Casari.
Hugging Face Datasets documentation — practical reference for modern dataset workflows.
Tecton / Hopsworks / Feast docs — modern feature store architectures.

Track 02 · Data Foundations · 12 min

Garbage in, garbage out. The 80% of ML nobody talks about.

DataMLOpsFoundations

“The model gets all the credit, but the data does all the work.”
— Andrew Ng · Data-Centric AI movement, 2021

#The hook

Andrew Ng said it most clearly: "The model gets all the credit, but the data does all the work."

This is the field manual for that 80%.

#Why this matters in 2026 — the receipts

Where the time really goes

Data work is the actual job

80%

ML time spent on data

CrowdFlower 2016 + 2024 follow-up

$175B+

Data infrastructure market 2026

IDC

60%

Models that drift in prod

MLflow community survey 2025

50%

ML projects killed by data bugs

Gartner 2025

#The 30-second answer

Data Foundations is the discipline of getting raw, messy, real-world information into a shape your model can learn from — and keeping it that way.

It splits into four phases. Each phase has its own tools, traps, and best practices.

The pipeline

Four phases of data work

1. Ingest & explore

Get the data, look at it

Pull from APIs, databases, logs, spreadsheets. Then EDA — exploratory data analysis. Plot everything.

Pandas / Polars / DuckDB for tabular. Pyarrow for big files.
Always plot distributions, correlations, missing-value patterns BEFORE you train anything.
Most production bugs come from being surprised by the data.

2. Clean & engineer

Make it model-ready

Handle missing values, outliers, encoding, feature engineering. Where most of the time goes.

Median imputation for numerics, 'unknown' category for strings, but check that it doesn't break things.
One-hot vs ordinal vs target encoding for categoricals — the choice changes performance.
Domain features still beat learned features for tabular data. Always.

3. Split & validate

Honest evaluation

Train, validation, test. Cross-validation. Stratification. Group-aware splitting.

Random split is wrong for time series — you'd be predicting the past from the future.
Always stratify on the target if it's imbalanced. Most real-world datasets are.
Group-aware splitting prevents data leakage when one user has many rows.

4. Monitor & adapt

Survive in production

Detect drift. Track input distributions. Re-label. Retrain. Forever.

Data drift: input distributions shift. Concept drift: the relationship between inputs and outputs shifts.
PSI (Population Stability Index) and KS-test are the field-standard drift metrics.
Production ML is 90% monitoring, 10% modeling. Get used to it.

80%

of ML time goes to data work

Anaconda 'State of Data Science' 2024

Vocabulary

Six data terms that show up daily

Concept

EDA

Exploratory Data Analysis — plot everything before modeling.

Like: A doctor's checkup before surgery.

e.g. Histograms, correlation plots, missing-value heatmaps

Concept

Train/val/test split

Hold out data the model never trains on, to measure generalization.

Like: Separate practice tests from the real exam.

e.g. 60/20/20 random split

Concept

Feature engineering

Transform raw inputs into model-friendly features.

Like: Prepping ingredients before cooking.

e.g. log_income, age_bucket, days_since_signup

Concept

Data drift

Input distributions change between training and production.

Like: Yesterday's map of a changing city.

e.g. Users got younger over six months

Concept

Concept drift

The relationship between inputs and outputs shifts.

Like: Spam tactics evolve — yesterday's filter misses today's.

e.g. COVID destroyed every demand-forecasting model in 2020

Concept

Feature store

Single store of features for both training and serving.

Like: One pantry for the kitchen and the truck.

e.g. Tecton, Feast, Hopsworks

#A real cleaning + EDA pipeline — runnable

Here's the workflow that handles 80% of real datasets, end-to-end, in one runnable cell:

pythonrunnable cell

import pandas as pd
import numpy as np

# Make a "messy" dataset that looks like real-world data
np.random.seed(42)
n = 500
data = pd.DataFrame({
    "age":         np.random.randint(18, 80, n),
    "income":      np.random.lognormal(10, 1, n).round(),
    "city":        np.random.choice(["NYC", "LA", "SF", None], n, p=[.35, .25, .25, .15]),  # 15% missing
    "subscribed":  np.random.choice([0, 1], n, p=[0.7, 0.3]),
})

# Inject 5% bad data — typical of real-world ETL
data.loc[np.random.choice(n, 25), "age"] = -1   # bad sentinel for "missing"
data.loc[np.random.choice(n, 15), "income"] = np.nan

# 1. EXPLORE
print("Shape:", data.shape)
print("\nMissing per column:")
print(data.isna().sum())
print("\nFirst 3 rows:")
print(data.head(3))

# 2. CLEAN
data["age"] = data["age"].replace(-1, np.nan)            # bad sentinel -> proper NaN
data["age"] = data["age"].fillna(data["age"].median())   # median imputation
data["income"] = data["income"].fillna(data["income"].median())
data["city"] = data["city"].fillna("unknown")            # explicit unknown category

# 3. ENGINEER FEATURES
data["log_income"] = np.log1p(data["income"])             # log-transform a skewed feature
data["age_bucket"] = pd.cut(data["age"], bins=[0, 30, 50, 100], labels=["young", "mid", "older"])

# 4. ONE-HOT ENCODE CATEGORICALS
data_encoded = pd.get_dummies(data, columns=["city", "age_bucket"], drop_first=True)

print("\nFinal shape:", data_encoded.shape)
print(f"Subscription rate: {data_encoded['subscribed'].mean():.1%}")
print(f"Columns ready for ML: {list(data_encoded.columns)}")

That cell does what 90% of real ML pipelines do, in 30 lines. Try modifying the missing-value strategy or the bucket boundaries and watch the output shift.

#The drift problem — and how to detect it

pythonrunnable cell

import numpy as np

# Train-time distribution (e.g., user ages from 6 months ago)
train_ages = np.random.normal(loc=42, scale=12, size=2000)

# Today's production distribution — the world has shifted slightly younger
prod_ages  = np.random.normal(loc=37, scale=14, size=2000)

# Population Stability Index — the field-standard drift metric
def psi(train, prod, bins=10):
    cuts = np.percentile(train, np.linspace(0, 100, bins + 1))
    cuts[0], cuts[-1] = -np.inf, np.inf
    train_pct = np.histogram(train, cuts)[0] / len(train) + 1e-6
    prod_pct  = np.histogram(prod,  cuts)[0] / len(prod)  + 1e-6
    return np.sum((prod_pct - train_pct) * np.log(prod_pct / train_pct))

score = psi(train_ages, prod_ages)
print(f"PSI score: {score:.3f}")
print("Interpretation:")
print("  < 0.10  no significant drift   ← all good")
print("  0.10–0.25  moderate drift       ← investigate")
print("  > 0.25  severe drift            ← retrain")

#What's been built on solid data foundations

Where data discipline pays

Systems that lived or died by their data

Data + creativity

Netflix Prize

$1M

Prize for 10% improvement

Three years of competition, 40,000 teams. The winning team's edge was data engineering, not algorithm choice.

Feature engineering

Continuous data ops

Tesla Autopilot

4M+

Vehicles labeling for free

Every Tesla on the road continuously collects training data. The data flywheel is the product.

Data flywheel

Labeled dataset

ImageNet

14M+

Hand-labeled images

Fei-Fei Li's 2009 dataset created modern deep learning. The architecture (AlexNet) was 5%; the data was 95%.

Labeled scale

Dataset distribution

Hugging Face hub

100K+

Datasets hosted

GitHub for ML datasets. Search, version, fine-tune — all from one URL. Now reshaping how teams ship models.

Distribution

Curation

OpenAI GPT-4 prep

13T

Training tokens (curated, deduped)

Most of the GPT-4 effort was data curation: source filtering, deduplication, quality classifiers, content safety.

Curation at scale

Synthetic data

Anthropic Constitutional AI

100K+

AI-generated preference pairs

Used AI to generate training data for the same AI. Started a wave of synthetic-data techniques across the industry.

Synthetic data

#The 2026 frontier

#Where to go next

Data Foundations track — 15 lessons covering pipelines, EDA, cleaning, feature engineering, drift detection.
SQL Mastery — most data work happens in SQL. Pair this with the SQL track.
Python Foundations — pandas, polars, NumPy live in Python.
Classical ML — apply your clean data to real models.

#Key takeaways

Recap

Key Takeaways

80% of ML work is data work. Cleaning, engineering, splitting, monitoring.
Four phases: ingest+explore → clean+engineer → split+validate → monitor+adapt.
Always EDA before you model. Plot everything. Find the surprises before the model does.
Random splits break time-series and grouped data. Use TimeSeriesSplit and GroupKFold.
Production ML is 90% monitoring. PSI and KS-test are the bread-and-butter drift metrics.
The 2026 frontier: synthetic data, real-time feature stores, data contracts, lakehouse architectures.

#References & further reading

Designing Machine Learning Systems by Chip Huyen — the bible for production ML data work.
Andrew Ng's Data-Centric AI talks (deeplearning.ai). Started the field-shift.
Feature Engineering for Machine Learning by Alice Zheng & Amanda Casari.
Hugging Face Datasets documentation — practical reference for modern dataset workflows.
Tecton / Hopsworks / Feast docs — modern feature store architectures.

Garbage In, Garbage Out: The 80% of ML Nobody Talks About

Garbage in, garbage out. The 80% of ML nobody talks about.

#The hook

#Why this matters in 2026 — the receipts

Data work is the actual job

#The 30-second answer

Four phases of data work

1. Ingest & explore

2. Clean & engineer

3. Split & validate

4. Monitor & adapt

Six data terms that show up daily

EDA

Train/val/test split

Feature engineering

Data drift

Concept drift

Feature store

#A real cleaning + EDA pipeline — runnable

#The drift problem — and how to detect it

#What's been built on solid data foundations

Systems that lived or died by their data

Netflix Prize

Tesla Autopilot

ImageNet

Hugging Face hub

OpenAI GPT-4 prep

Anthropic Constitutional AI

#The 2026 frontier

#Where to go next

#Key takeaways

#References & further reading

Garbage In, Garbage Out: The 80% of ML Nobody Talks About

Garbage in, garbage out. The 80% of ML nobody talks about.

#The hook

#Why this matters in 2026 — the receipts

Data work is the actual job

#The 30-second answer

Four phases of data work

1. Ingest & explore

2. Clean & engineer

3. Split & validate

4. Monitor & adapt

Six data terms that show up daily

EDA

Train/val/test split

Feature engineering

Data drift

Concept drift

Feature store

#A real cleaning + EDA pipeline — runnable

#The drift problem — and how to detect it

#What's been built on solid data foundations

Systems that lived or died by their data

Netflix Prize

Tesla Autopilot

ImageNet

Hugging Face hub

OpenAI GPT-4 prep

Anthropic Constitutional AI

#The 2026 frontier

#Where to go next

#Key takeaways

#References & further reading