From tensors and autograd to ResNet, transformers, GANs/VAEs, ViTs, object detection, diffusion models, graph neural networks, self-supervised pretraining, mixture-of-experts, state-space-models/Mamba, flow matching, and FSDP-distributed training. 27 lessons covering the foundations, the architectures, the generative stack, the post-transformer family, and the production-scale tricks that turn a notebook prototype into a 70B-parameter model.
Tensors as multi-dim arrays + autograd as the engine that records every op and replays the chain rule backwards. The substrate every modern DL framework runs on.
From the McCulloch-Pitts neuron to the multilayer perceptron. Why a single perceptron fails on XOR, and how stacked layers + nonlinearity solved it.
Why neural nets need nonlinearity, and the activation evolution from sigmoid to GELU.
MSE, MAE, Huber, cross-entropy, focal loss — and why picking the wrong one silently breaks training.
The chain rule applied to neural networks. How gradients flow backwards through a computational graph.
SGD, Momentum, RMSprop, Adam, AdamW, plus learning rate scheduling — the optimizer evolution that powers modern deep learning.
Why deep networks failed before 2010 and what fixed them. Xavier and He initialization, dying ReLUs, and the gradient flow that depth depends on.
Dropout, weight decay, label smoothing, mixup, CutMix, stochastic depth — keeping deep nets from memorizing.
BatchNorm, LayerNorm, GroupNorm, RMSNorm — the family of techniques that stabilized training and let networks reach 100+ layers.
Warmup, OneCycle, gradient clipping, mixed precision (fp16/bf16), gradient checkpointing — the tricks that decide whether a training run succeeds or wastes a week of GPU time.
Filters, feature maps, and the hierarchy of visual features that made vision a deep-learning problem.
LeNet → AlexNet → VGG → Inception → ResNet → DenseNet → EfficientNet → ConvNeXt — the evolution of vision architectures and what each one contributed.
Stand on the shoulders of giants. Pretrained backbones, freezing layers, discriminative learning rates — fast.ai's signature method.
Processing sequences with memory. Unrolling, BPTT, and why vanilla RNNs vanish on long sequences.
Gating mechanisms (LSTM forget/input/output, GRU update/reset), encoder-decoder, teacher forcing, and Bahdanau attention as the bridge to transformers.
Self-supervised learning by reconstruction. Vanilla, denoising, sparse, and contractive autoencoders — foundational representation learning that bridges to VAEs and diffusion.
Multi-GPU, FSDP/ZeRO, pipeline + tensor parallelism, profiling, and the production capstone for training models bigger than your GPU.
LSTMs read a sentence one word at a time, accumulating memory in a hidden state.
A discriminative model answers "what is this?" A generative model answers "what could this be?" The first asks for a label given a picture; the second asks for a picture…
Classification answers "what is this picture?" Detection answers "what is in this picture, where, and how many?" Segmentation answers "for every single pixel, what does it…
In September 2016, Google replaced ten years of statistical phrase-table machinery with a single neural network — eight stacked LSTMs wired into an encoder-decoder shape,…
The cleanest way to see diffusion is not as a "score-based generative model" or as "non-equilibrium thermodynamics" — it is as a Variational Autoencoder taken to its…
A CNN assumes your data lives on a regular grid of pixels.
*The setup:*
*Reach for Mamba / a Mamba-heavy hybrid when:*
A dense neural network has one knob — make it bigger and you pay more compute on every input.
Diffusion training is a tower of Jenga blocks: pick a noise schedule, derive the ELBO across noise levels, switch to score parameterization, re-derive the loss as a…
10 interactive labs — hands-on exercises for this track
A neuron deciding whether to fire — explore how different activation functions shape the decision
Build a brain layer by layer — add neurons, connect them, and watch data flow through
When the network makes a mistake, blame flows backwards — watch error propagate through each layer
You're training a model live — watch the loss drop, accuracy rise, and spot when overfitting starts
See what a neural network is actually looking for — a map of every feature detector inside a CNN
Peer through the eyes of a convolutional filter — see exactly which patterns it's trained to detect
Trace the mathematical operations as data flows forward — then watch gradients flow back
A free-form neural network playground — wire up any architecture and watch it learn
5-layer network with activation distributions evolving across 60 epochs. BN off: wild drift. BN on: stable + trains at 10× learning rate.
Plain vs ResNet vs DenseNet vs Highway. Depth slider 4-40 layers. Animated gradient wavefront — gradients die in Plain, survive in the others.
675 questions across 27 modules — check how well you understood this track.