Track 06 · Generative AI · 13 min
How AI learned to dream.
From Goodfellow's 2014 GAN paper to FLUX and Sora — the journey from 'looks like noise' to 'looks like reality.' Five interactive demos cover the architectures (VAE, GAN, diffusion) that now generate the images, music, and video the world consumes.
“What I cannot create, I do not understand.”
#The hook
In 2014, a PhD student named Ian Goodfellow scribbled an idea on a napkin at a Montreal bar: train two neural networks to fight each other. One generates fake data, the other tries to spot the fakes. Train them long enough and the generator gets very, very good.
This is the field manual for the engine that learned to dream — and now generates the images, music, and video the internet runs on.
#Why this matters in 2026 — the receipts
Generative AI by the numbers
What 'AI dreaming' is worth
100M+
Images generated/day worldwide
Estimate from FLUX, MJ, SDXL
60s
Coherent video, single prompt
Sora, Veo, 2024
$50B+
Generative AI market by 2026
Gartner
1T+
Tokens generated daily by chat AIs
OpenAI, Anthropic, Google
#The four families of generative models
The architecture map
Four ways to make an AI generate
1. Autoencoders / VAEs
2013Compress + reconstruct. The encoder squashes, the decoder rebuilds.
- Learn a low-dimensional 'latent space'; sample from it to generate.
- Outputs are smoother but blurrier than GANs.
- Foundation for latent diffusion (Stable Diffusion uses a VAE-encoded latent space).
2. GANs
2014Two networks fight: a generator tries to fool a discriminator.
- Sharper outputs than VAEs, harder to train (mode collapse, instability).
- Owned image generation 2017–2021 (StyleGAN, BigGAN, etc.).
- Mostly displaced by diffusion in 2022, but still used for face/style work.
3. Diffusion
2020+Learn to remove noise step by step. Start from pure noise, denoise to image.
- Generates the highest-quality images in 2026. Stable Diffusion, FLUX, DALL-E 3.
- Rooted in 2015 physics (Sohl-Dickstein) but ignored until DDPM (2020) made it viable.
- Conditional diffusion (text-to-image) is the standard interface.
4. Flow matching / Consistency models
2023+Make diffusion 100× faster — single-step or few-step generation.
- Replaces the 50-step diffusion loop with 1-4 steps.
- FLUX, SDXL Turbo, Latent Consistency Models all use this.
- Brought generation latency from seconds to milliseconds.
100M+
AI images generated daily on Earth
As of mid-2026, somewhere between 100M and 200M images are generated each day across FLUX, Midjourney, Stable Diffusion, DALL-E 3, and dozens of consumer apps. More AI-generated imagery is now produced per week than the entirety of stock-photography output before 2020.
Industry estimate, multiple sources 2025–2026
Vocabulary
Six generative-AI terms in every paper
Concept
Latent space
Low-dimensional learned space where similar things sit nearby.
Like: Concept GPS — coordinates for ideas.
e.g. Stable Diffusion operates in a learned latent
Concept
Forward / reverse
Diffusion's two phases — add noise, then learn to remove it.
Like: A teacher showing a clock unwinding.
e.g. DDPM is the canonical version
Concept
U-Net
The denoising backbone in most diffusion models.
Like: A funnel that compresses then expands data.
e.g. Stable Diffusion's denoiser
Concept
ControlNet
Add a conditioning signal (pose, depth, sketch) to a diffusion model.
Like: Guardrails on a highway.
e.g. Generate a portrait matching a pose photo
Concept
CFG (guidance)
How strongly to follow the text prompt vs. roam.
Like: How precisely to follow the recipe.
e.g. CFG=7.5 is a typical Stable-Diffusion setting
Concept
Distillation
Train a small fast 'student' to mimic a big slow 'teacher'.
Like: An apprentice learning the master's moves.
e.g. SDXL Turbo distilled from SDXL
#Family 1 — Autoencoders, the foundation
The latent space — the compressed middle — is where the magic lives. Distance in latent space corresponds to similarity in image space.
#Family 2 — GANs, the original showstopper
GAN training is famously unstable. The generator can collapse to producing only a few outputs (mode collapse). The discriminator can win too fast. Half the GAN literature is recipes to make training stable.
#Family 3 — Diffusion, the 2026 default
Look at the timestep more closely — you can see exactly what the model is doing at each scale:
#What's been built with generative AI
Generative AI in production
What this family of models has actually shipped
Discord-native AI art
Midjourney
20M+
Subscribers
Profitable from day one (no VC). Diffusion-based, custom UX. Largest paid AI consumer product after ChatGPT.
Diffusion
Open-weights image
Stable Diffusion
1B+
Lifetime generations
Open-weights diffusion model that triggered the consumer creative-AI explosion in 2022.
Latent diffusion
Black Forest Labs
FLUX
12B
Parameters
2024's open-weights leader. Built by ex-Stable-Diffusion team. Photorealistic, fast.
Flow matching
Text-to-video
Sora / Veo / Kling
60s
Coherent video output
2024 broke the 'looks real' barrier for AI-generated video. Hollywood is reckoning with the implications.
Video diffusion
Voice cloning
ElevenLabs
30+
Languages, near-human quality
Diffusion-based voice synthesis. Indistinguishable from human in many cases.
Audio gen
Music generation
Suno / Udio
1M+
Tracks/day generated
Text-to-song. Diffusion + transformer hybrids. Reshaping music production economics.
Audio diffusion
#The 2026 frontier
#Where to go next
- Generative AI track — 18 lessons: VAEs, GANs, diffusion, flow matching, video, 3D, audio.
- Deep Learning — prerequisites for the underlying architectures.
- NLP & Transformers — text encoders feed every modern generative model.
- Math Foundations — the math of diffusion is a beautiful application of stochastic calculus.
#Key takeaways
Key Takeaways
- Four generative families: VAEs (compress+reconstruct), GANs (adversarial), diffusion (denoise), flow matching (single-step diffusion).
- Diffusion is the 2026 default for image quality. FLUX, Stable Diffusion, DALL-E 3 all diffusion.
- Latent space is the compressed middle representation. Distance in latent space = similarity in real space.
- Flow matching / consistency models brought generation latency from seconds to milliseconds.
- Video generation crossed photorealistic in 2024 with Sora. Hollywood is reckoning with the consequences.
- ControlNet, IP-Adapter, and similar steering tools are how production creative AI actually ships.
#References & further reading
- Goodfellow et al. — Generative Adversarial Nets (NeurIPS 2014). The GAN paper.
- Ho et al. — Denoising Diffusion Probabilistic Models (NeurIPS 2020). Modern diffusion's birth.
- Rombach et al. — High-Resolution Image Synthesis with Latent Diffusion Models (CVPR 2022). Stable Diffusion.
- Lipman et al. — Flow Matching for Generative Modeling (ICLR 2023). Flow matching foundation.
- Lilian Weng — What Are Diffusion Models? (lilianweng.github.io). Best technical explainer.
- Distill.pub — A Visual Guide to Diffusion Models (interactive).