Track 01 · Math Foundations · 14 min
The math behind the machines.
You don't need a math degree to do AI. You need exactly four ideas: vectors, matrices, derivatives, and probability. Played out across six interactive visualizations, this is the math you actually need — animated, explained, and runnable in your browser.
“The thing I love about deep learning is that it's just math. The fact that the math has been hiding in plain sight in linear algebra textbooks for sixty years should make you suspicious of any math you don't understand.”
#The hook
#Why this matters in 2026 — the receipts
The math undercurrent
What modern AI rests on
4
Concepts you actually need
vectors / matrices / derivatives / probability
175B
Parameters in GPT-3 — all gradient-descended
OpenAI 2020
1T+
Floating-point ops per ChatGPT message
estimate
350
Years since Newton + Leibniz invented derivatives
1670s
#The four ideas
The map
Vectors → Matrices → Derivatives → Probability
1. Vectors
Linear algebraA vector is a list of numbers. That's it. Everything in modern AI is vectors.
- An image is a vector. A sentence is a vector (after embedding). A user's preferences is a vector.
- Distance, dot product, cosine similarity — three operations you'll use a thousand times.
- Vector search powers RAG, recommender systems, semantic search.
2. Matrices
Linear algebraA matrix is a stack of vectors. Multiplying a matrix by a vector is a transformation in space.
- Every layer of a neural network is a matrix multiply followed by a nonlinearity.
- Eigenvalues and SVD are how PCA, recommendation systems, and stable training all work underneath.
- Matrix-multiply is the operation GPUs were invented to make fast.
3. Derivatives & gradients
CalculusHow much does the output change if the input wiggles? That number is the gradient.
- Training a neural network = compute the gradient of the loss, take a step opposite, repeat 10^12 times.
- Backpropagation is just the chain rule applied carefully through a computation graph.
- Stochastic gradient descent (SGD), Adam — variations on the same idea: follow the slope downhill.
4. Probability
StatsReason about uncertainty. The whole game when data is noisy.
- Bayes' theorem — how to update beliefs given new evidence. The foundation of all probabilistic ML.
- Distributions — Gaussian, Bernoulli, Categorical. The vocabulary of model assumptions.
- Cross-entropy loss is just the log-likelihood with a minus sign. Information theory in disguise.
#Idea 1 — Vectors and matrices, animated
#Idea 2 — Derivatives and the loss landscape
When you have a complicated loss landscape — the kind every real neural network creates — the choice of optimizer matters. Below, watch SGD, Momentum, and Adam race across the same surface. Notice how Adam handles the ravines.
#Idea 3 — Probability, from coins to Bayes
#Idea 4 — Information theory, briefly
Vocabulary
The math terms you'll see in every paper
Concept
Vector
A list of numbers. The atom of all modern AI.
Like: An arrow from origin to a point in space.
e.g. Word embedding: a 384-D vector
Concept
Matrix multiply
Combine two stacks of vectors via the dot-product rule.
Like: Composing two transformations of space.
e.g. Every neural net layer's core operation
Concept
Gradient
How much the output changes when each input wiggles.
Like: The slope under your feet on a hiking trail.
e.g. ∇L points in the steepest-uphill direction
Concept
Backprop
The chain rule, run backward through a computation graph.
Like: Tracing blame for a final mistake back to causes.
e.g. PyTorch's autograd does this automatically
Concept
Distribution
Function that says how likely each value is.
Like: A histogram, but mathematical.
e.g. Gaussian, Bernoulli, Categorical
Concept
Cross-entropy
How surprised the model is by the right answer.
Like: Hot/cold game. Lower = closer to right.
e.g. The loss every classifier minimizes
#What's been built on this math
Where math becomes product
Real systems built on the four ideas
Vectors + matrices
AlphaFold
200M+
Protein structures
Pure linear algebra at planetary scale. Each protein is a point in a learned vector space; folds are matrix transformations.
Linear algebra
Matrix factorization
Recommender systems
80%
Netflix watches recommended
SVD on user-movie matrices powered Netflix Prize ($1M, 2009). Same math now drives Spotify, YouTube, TikTok.
SVD / matrices
Gradient descent
Stable Diffusion
1T+
Gradient steps to train
Trained by computing gradients of pixel-similarity loss, billions of times. Pure calculus + GPUs.
Calculus / autograd
Probability + KL
ChatGPT (RLHF)
175B
Parameters tuned via KL
RLHF uses KL-divergence to keep the fine-tuned model close to the base. Pure information theory.
Probability
Bayes' rule
Bayesian A/B testing
30%
Faster than frequentist tests
Modern experimentation platforms (Eppo, Statsig) use Bayesian inference to ship experiments 30% faster than classical methods.
Probability
Eigenvectors
PageRank → Search
1T+
Web pages ranked daily
Google's original algorithm: the most important page is the eigenvector of a hyperlink matrix. Same math, 25 years.
Linear algebra
#Where to go next
- Math Foundations track — 17 lessons through vectors, matrices, calculus, probability, and optimization, each with interactive viz.
- Python Foundations — math is more fun when you can run experiments. NumPy is your math toolbox.
- Classical ML — apply the math: linear regression, logistic regression, decision trees, ensembles.
- Deep Learning — where the four ideas combine into modern AI.
#Key takeaways
Key Takeaways
- Four ideas cover 95% of the math you need: vectors, matrices, derivatives/gradients, probability.
- A matrix is a transformation in space. Every neural network layer is one of these (plus a nonlinearity).
- Training a model = compute gradient of loss, step downhill, repeat. Backprop is just the chain rule.
- Cross-entropy loss is information theory in disguise. Lower entropy = better predictions.
- Bayes' rule is the formula for updating beliefs. The most important single equation in applied probability.
- You don't need a math degree. You need intuition — and that comes from playing with viz, not from textbooks.
#References & further reading
- Gilbert Strang — Linear Algebra and Its Applications. The standard. His MIT OCW lectures are gold.
- 3Blue1Brown — Essence of Linear Algebra and Essence of Calculus (YouTube). The most beautiful math explainers ever filmed.
- Jaynes — Probability Theory: The Logic of Science. Once you're hooked.
- Marc Peter Deisenroth, A. Aldo Faisal, Cheng Soon Ong — Mathematics for Machine Learning (free PDF, mml-book.com). Best ML-targeted math text.
- Goodfellow, Bengio, Courville — Deep Learning (Part I, free at deeplearningbook.org). Every prerequisite, derived clean.
- Khan Academy linear algebra and calculus playlists.
- Distill.pub on attention, momentum, t-SNE — animated explanations of specific topics.