The ML Framework

Why does your phone unlock when it sees your face, your email auto-sort spam, and Netflix know you'd love that obscure documentary? They're all the same trick: a machine learned a pattern from examples instead of being told explicit rules. By the end of this lesson, you'll be able to look at any real-world AI feature — Tesla autopilot, ChatGPT, Shazam, Google Translate — and instantly identify which of three flavors of machine learning is running under the hood, plus exactly why each one was the right choice.

Don't worry if "bias-variance tradeoff" sounds scary — it is really just the balance between a model that is too simple (underfitting) and one that memorized the answers (overfitting). It clicks once you see the training curves!

#How Machines Learn: Three Paradigms

#Supervised Learning

Supervised learning is the most common paradigm in industry. You have a dataset of input-output pairs, and the goal is to learn a function that maps inputs to outputs.

Try it! Open the Python REPL and type these lines yourself. Train a quick classifier: from sklearn.datasets import load_iris; from sklearn.tree import DecisionTreeClassifier; X, y = load_iris(return_X_y=True); m = DecisionTreeClassifier().fit(X, y); print(f"Accuracy: {m.score(X, y):.0%}") — you just trained your first ML model!

There are two main flavors:

#Classification

#Regression

The answer is regression -- the output is a continuous percentage. However, if you threshold it (above 50% = high risk, below = low risk), you turn it into a classification problem. Many real-world systems combine both: predict a continuous score, then threshold for decisions.

#Unsupervised Learning

In unsupervised learning, there are no labels. The algorithm must discover structure in the data on its own.

Key tasks

• Clustering -- group similar data points (K-Means, DBSCAN, hierarchical clustering)
• Dimensionality reduction -- compress high-dimensional data while preserving structure (PCA, t-SNE, UMAP)
• Anomaly detection -- find data points that do not fit the normal pattern
• Density estimation -- model the underlying probability distribution of data

#Reinforcement Learning

RL is behind AlphaGo, robotics, self-driving cars, and the RLHF (Reinforcement Learning from Human Feedback) that makes ChatGPT helpful rather than harmful.

#The Training Loop

Regardless of the paradigm, every ML model follows the same fundamental loop:

#Step 1: Prepare Data

Split into training, validation, and test sets. Feature engineering, normalization, handling missing values.

#Step 2: Choose a Model

Select an algorithm family (linear model, tree, neural network, etc.) based on data size, feature types, and interpretability needs.

#Step 3: Define a Loss Function

A mathematical measure of "how wrong is the model right now?" The loss function translates the abstract goal ("classify emails correctly") into a concrete number the optimizer can minimize.

#Step 4: Train (Optimize)

Iteratively adjust model parameters to minimize the loss. For gradient-based methods, this is gradient descent. For trees, it is greedy splitting.

#Step 5: Evaluate

#Step 6: Iterate

Adjust hyperparameters, try different models, engineer better features. Repeat until performance is satisfactory.

#The Bias-Variance Tradeoff

#Bias

Bias is the error from incorrect assumptions in the model. A linear model applied to non-linear data will always have high bias -- no matter how much data you give it, it will never learn the true curved relationship. Bias measures how far off the model's average prediction is from the truth.

#Variance

Variance is the error from sensitivity to fluctuations in the training data. If you retrain a model on a slightly different dataset and the predictions change wildly, the model has high variance. Complex models (deep trees, high-degree polynomials) are prone to high variance -- they latch onto noise.

#The Tradeoff

As you increase model complexity:

• Bias decreases (the model can capture more patterns)
• Variance increases (the model becomes more sensitive to training data)
• Total error first decreases, then increases

#Choosing the Right Paradigm

#Worked Example: A 5-Patient Diabetes Predictor

Let us walk through the training loop with concrete numbers. You have 5 patients with their glucose level (mg/dL) and a binary label (1 = diabetic, 0 = not):

That is the full training loop in 6 numerical steps. Every ML model — from logistic regression to GPT — repeats this same loop, just with millions of weights instead of two.

#When to Use Each Paradigm vs. Alternatives

#Try It Yourself

#Key Takeaways

• Three paradigms cover all ML. Supervised learning maps inputs to outputs with labeled data, unsupervised learning discovers structure without labels, and reinforcement learning optimizes strategies through reward signals
• The bias-variance tradeoff is the central tension. Too simple a model underfits (high bias, misses patterns), too complex a model overfits (high variance, memorizes noise), and the art is finding the sweet spot
• Always evaluate on held-out data. A model that memorizes training data gets 100% training accuracy but may be useless on new data; generalization performance is the only metric that matters
• Start simple, increase complexity only when needed. A well-tuned logistic regression often beats a poorly tuned neural network; Occam's Razor applies to model selection

#Quick Check