A discriminative model answers "what is this?" A generative model answers "what could this be?" The first asks for a label given a picture; the second asks for a picture given a label, or just for a picture at all. That single flip — from p(Y|X) to p(X) — is the difference between a classifier that recognizes cats and a model that paints them. This lesson walks you through the two foundational ways neural networks learned to paint: the adversarial game of GANs and the variational compression of VAEs. Both are stepping stones on the road that eventually led to ThisPersonDoesNotExist, StyleGAN, DALL-E, and Stable Diffusion.
Learning Objectives
After this lesson, you will be able to:
Tell the difference between modeling p(Y|X) (classification, what discriminative nets do) and modeling p(X) (generation, what we want here) — and list four real things you can do with p(X) beyond making pretty pictures
Read a GAN training loop and predict what will go wrong: when D wins too fast, when G collapses to a few modes, when neither converges — and pick the loss function (minimax, non-saturating, Wasserstein) that fixes each failure mode
Derive the VAE ELBO from the intractable log-evidence in three lines, then explain why each term has the interpretation it does: reconstruction quality plus KL regularizer
Apply the reparameterization trick to a Gaussian latent: rewrite z = μ + σ·ε so gradients flow through μ and σ instead of through a random number
Compare GANs and VAEs across four axes — sample quality, training stability, density estimation, and latent interpretability — and predict which one a colleague should reach for given their constraints
If you only remember one phrase from this lesson: GANs play a game; VAEs solve an optimization. Both end up at "generate stuff that looks like the training data," but they take wildly different routes — and the routes determine everything about what you can and can't do with the model.
Up to this lesson, every neural network you have built has been discriminative: given an input x, predict an output y. The CNN classifier maps a picture to a digit. The sentiment model maps a review to a label. Under the hood, you are estimating p(Y|X) — the conditional distribution of the label given the input.
Generative models flip the script. Instead of p(Y|X), they model p(X) directly — the distribution over inputs themselves. Once you have p(X), you can:
Sample new inputs that look like the training data (the obvious one — fake faces, fake songs, fake molecules)
Estimate density at a query point: how likely is this specific x under my model? Useful for anomaly detection — low p(x) means "out of distribution"
Do semi-supervised learning: use unlabeled data to learn what the input distribution looks like, then fine-tune on a tiny labeled set
Sample conditionally: model p(X|Y) — give me an image of a "7", give me a song in the style of Coltrane, give me a molecule that binds to this protein
Interpolate in latent space: pick two real examples, find their latent codes, walk a smooth path between them and decode — this is how you do "morph this cat into that dog" demos
There are roughly four families of neural generative models:
Autoregressive models — model p(x) as a product of conditionals: p(x_1) · p(x_2 | x_1) · ... — this is how language models work (next-token prediction). Slow to sample, exact density.
Normalizing flows — model p(x) as an invertible transformation of a simple base distribution. Exact density, exact sampling, but architectural constraints.
Implicit models (GANs) — never write down p(x) directly; instead train a network that samples from it via adversarial training. Sharp samples, no density, can be unstable.
Variational models (VAEs) — write down p(x) as a marginalization over a latent variable, optimize a lower bound (the ELBO). Stable, slightly blurry samples, approximate density via the bound.
This lesson covers (3) and (4) — the two paradigms that dominated 2014-2020 and still anchor most of the diffusion-model literature you'll meet in track-06.
The headline idea of a Generative Adversarial Network is breathtakingly simple and was sketched on a bar napkin in 2014. Train two networks against each other:
Generator G. Takes a random noise vector z (typically z ~ N(0, I) with maybe 100-512 dimensions) and produces a fake sample G(z). Its job is to make G(z) look indistinguishable from a real training sample.
Discriminator D. Takes any sample (real x or fake G(z)) and outputs the probability that the sample is real. Its job is to spot the fakes.
They play a minimax game. D is rewarded for catching fakes; G is rewarded for fooling D. As D gets better at spotting fakes, G is forced to get better at making them. As G gets better at making fakes, D is forced to get more discriminating. In the limit, G produces samples indistinguishable from real data and D is reduced to chance (50/50 — a useless coin flip on every input).
Watch a GAN LearnInteractive
A small GAN trained on 2D data. Toggle real samples on/off, watch the generator and discriminator update each step. The dashed boundary is the discriminator's decision surface; the colored points are real vs generated.
This is mathematically elegant but practically broken. The reason: gradient saturation. Early in training, G is terrible and D easily classifies G(z) as fake — so D(G(z)) is near 0. Look at the gradient of log(1 - D(G(z))) at D(G(z)) ≈ 0: it's essentially flat, near zero. G gets almost no learning signal exactly when it most needs one.
What Do You Think?
Early in GAN training, the discriminator easily spots the generator's terrible fakes — so D(G(z)) is near 0. What happens to the generator's gradient under the original minimax loss G = log(1 - D(G(z)))?
The fix in Goodfellow's original paper was to swap G's objective:
Instead of: G minimizes log(1 - D(G(z)))
Use: G maximizes log D(G(z)) [equivalent to minimizing -log D(G(z))]
Same fixed point (G wants D to say "real"), but the gradient at D(G(z)) ≈ 0 is now much stronger. This is the non-saturating loss and it's what every practical GAN implementation uses — the original minimax form mostly survives in textbooks for clean mathematical exposition.
By 2017, GAN training was infamous for being a black art. Tiny hyperparameter changes blew up training. Some seeds worked, others diverged. Researchers spent more time managing optimizer schedules than designing architectures.
Arjovsky, Chintala, and Bottou published WGAN in 2017 and gave the field a theoretical framing of what was going wrong. The original GAN, they showed, implicitly minimizes the Jensen-Shannon divergence between the data distribution and the generator distribution. When those two distributions don't overlap (which they almost never do early in training, since G(z) is in a thin manifold inside a high-dimensional space), JS divergence is constant and saturates at log 2 — no gradient anywhere.
WGAN replaces JS with the Wasserstein-1 distance (a.k.a. Earth-Mover distance), which measures how much "mass" you'd have to move and how far to transform one distribution into another. Earth-Mover distance is well-defined and has meaningful gradients even when the two distributions are completely non-overlapping — so G always gets a useful learning signal.
W(pr,pg)=γ∈Π(pr,pg)infE(x,y)∼γ[∥x−y∥]
The follow-up paper, WGAN-GP (Gulrajani et al. 2017), replaced weight clipping with a gradient penalty — penalize ‖∇D(x̃)‖ for being far from 1 on samples x̃ along the line between real and fake. WGAN-GP was the first GAN variant that trained reliably across a wide range of architectures and is still the default starting point for new image-GAN experiments.
The pathology that haunts every GAN trainer is mode collapse: instead of capturing the full diversity of the data distribution, G learns to produce only a few output modes that reliably fool D, ignoring everything else.
Imagine training a GAN on MNIST. A collapsed G might produce only "1"s and "7"s — the easiest digits to draw — and never produce a "5" or an "8". D might even be fooled, because the "1"s look realistic. But the generator has covered maybe 20% of the data distribution.
Symptoms of mode collapse
Samples have low diversity even when noise z is varied — multiple z values map to nearly identical outputs
Some classes / regions of the data are systematically missing from generated samples
Training loss looks fine; both D and G losses are bounded; only inspection reveals the failure
Often appears suddenly mid-training — G's output diversity crashes over a few hundred steps
Remedies (in rough order of practical use)
WGAN / WGAN-GP. The Earth-Mover formulation is much less prone to mode collapse because there is no "fool D once and you win" shortcut
Minibatch discrimination (Salimans 2016) — let D look at a whole minibatch at once and detect if all the samples are too similar; G is then penalized for producing low-diversity batches
Unrolled GANs (Metz 2016) — when updating G, simulate several future D updates and let G plan around them, preventing G from chasing one easy attack on D
Spectral normalization (Miyato 2018) — normalize the weights of D so its Lipschitz constant stays bounded, stabilizing the dynamics
Pac-GAN (Lin 2018) — concatenate multiple samples before feeding to D so the discriminator sees diversity directly
Quick check
You train a GAN on CIFAR-10 (10 image classes). After 50 epochs, the generated samples all look like cars and trucks — almost no dogs, birds, or ships. Training losses look healthy. What is happening?
The 2015 DCGAN paper (Radford, Metz, Chintala) was the first to give a reliable recipe for training GANs on real images. The architectural rules are still the default starting point for image GANs today:
All convolutional, no fully-connected layers. Both G and D are CNNs (except for input projections); FC layers introduce too much capacity for D and tend to make training unstable
Strided convolutions instead of pooling. Let D learn its own spatial downsampling; let G use transposed convolutions (strided "deconv") for upsampling instead of fixed nearest-neighbor or bilinear interpolation
Batch normalization in both G and D (with two exceptions: not in the output layer of G, not in the input layer of D) — stabilizes gradients
ReLU in G, except Tanh at the output. Tanh outputs in [-1, 1], matching normalized image pixels
LeakyReLU in D. Small negative slope (0.2) avoids vanishing gradients on the negative side, especially important for D
Adam optimizer with β1 = 0.5 (not the default 0.9) — slower momentum prevents D from running away from G
DCGAN samples on CelebA at 64x64 were the first GAN images that made non-researchers say "wait, that's a real photo, right?" They weren't, but the bar had been crossed.
Plain GANs are unconditional — give them random z, get a random sample. Conditional GANs (cGANs) add a label y as an input to both G and D, letting you control what gets generated. Want a "5"? Feed y = 5 alongside z.
pix2pix (Isola et al. 2017) is the image-to-image version: instead of conditioning on a class label, condition on an entire input image. Train pairs like (edge map, real photo) and pix2pix learns to fill in the photo from the edges. Same recipe powers maps-to-satellite-photo, sketch-to-photo, day-to-night, and the famous "draw a cat, get a cat photo" demos.
CycleGAN (Zhu et al. 2017) drops the requirement that the training pairs be paired. Got 10,000 horse photos and 10,000 zebra photos (but no matched pairs)? CycleGAN learns a horse → zebra translation and a zebra → horse translation simultaneously, with a "cycle consistency" loss that requires translating a horse to a zebra and back to give you the original horse. This is the architecture that lets you turn Monet paintings into photos, summer into winter, and (controversially) male into female faces.
NVIDIA's StyleGAN line (2018, 2019, 2020) refined GAN architectures specifically for face generation, hitting near-perfect 1024x1024 quality. Two key ideas:
Mapping network. Instead of feeding z directly to G, first transform it through an 8-layer MLP into an intermediate latent w. The intermediate space w turns out to be much more disentangled than z, meaning you can edit one attribute (smile) without affecting another (pose).
Adaptive Instance Normalization (AdaIN). At each layer of G, scale and shift the activations using statistics derived from w. This injects "style" at every resolution, separately — coarse features (pose, identity) from early layers, fine features (hair detail, skin texture) from late layers.
The result: you can mix the "style" of one face at coarse levels with the "style" of another at fine levels, producing realistic interpolations. This is what powers all the "AI face editor" apps that let you change age, hair color, expression, and pose on a single photo.
By 2022, diffusion models (covered deeply in track-06) had largely overtaken GANs for general image generation — DALL-E 2, Imagen, and Stable Diffusion all use diffusion, not GANs. The reasons: diffusion training is far more stable (no adversarial dynamics), the model produces a useful density estimate, and conditional generation via classifier-free guidance is easier to control.
But GANs still win in two regimes:
Real-time inference. A trained GAN does generation in a single forward pass (a few milliseconds on a phone GPU). Diffusion models need 20-50 forward passes for a sample, putting them out of reach for mobile real-time use until distillation tricks (Latent Consistency Models, SDXL Turbo) closed the gap recently.
Specialized high-resolution domains. StyleGAN3 at 1024x1024 on faces still produces samples that are competitive with the best diffusion models, and far cheaper to sample from. Face-editing apps, gaming, and synthetic data generation pipelines still rely on GANs for production.
Now flip to the second paradigm. VAEs come from a completely different intellectual tradition — variational inference, a tool from statistics for approximating intractable posteriors. The result is a model that looks almost identical to the autoencoder you built in the previous lesson, but with one critical change: the bottleneck is stochastic.
You already know the plain autoencoder shape. An encoder f maps input x to a code z. A decoder g maps z back to a reconstruction x̂. Train end-to-end by minimizing ‖x - g(f(x))‖².
Autoencoder RecapInteractive
Quick refresher on the plain autoencoder shape — encoder squeezes input down to a low-dim code, decoder reconstructs from the code, train to minimize the gap.
Loading visualization...
The problem with a plain autoencoder as a generative model: the latent space z has no defined distribution. If you sample a random z and feed it to the decoder, you mostly get garbage, because the decoder was only ever trained on the specific z values that the encoder happened to produce for real training inputs. The codes might cluster in weird shapes, leaving most of latent space "outside" the manifold the decoder learned.
VAEs fix this by training the latent space to follow a specific, simple distribution — typically N(0, I) — so you can sample new z values from that distribution and the decoder will know what to do with them.
The VAE's encoder no longer outputs a single point z. Instead, for each input x, it outputs a distribution q(z|x) — typically a Gaussian with mean μ(x) and standard deviation σ(x). To get a code, you sample from that distribution. The decoder then reconstructs from the sampled z.
The VAE Encoder Outputs a GaussianInteractive
The encoder produces μ and σ — the mean and spread of a Gaussian in latent space. Each input x maps to an ellipse, not a point. Larger ellipses mean more uncertainty about where the code should land.
Loading visualization...
Why on earth would you do this? Because if you train so that q(z|x) ≈ N(0, I) for every x, then sampling z from N(0, I) at test time will land you inside a region the decoder has seen during training — and you get coherent samples.
But there's a snag: how do you backprop through a random sample? You can't differentiate through z = sample(N(μ, σ)) directly — the sampling step is a non-differentiable random function.
What Do You Think?
The VAE encoder outputs μ and σ, and the latent code is z ~ N(μ, σ). You want gradients to flow from the reconstruction loss back to μ and σ so the encoder can learn. Why is this hard with naive sampling?
This is the single most important idea in the VAE. Instead of writing z ~ N(μ, σ), rewrite it as:
ε ~ N(0, I) # fixed random draw, treated as constant for backprop
z = μ + σ ⊙ ε # deterministic function of μ, σ, and the constant ε
Now z is a deterministic function of μ, σ, and ε. Gradients flow cleanly through μ and σ (via the addition and multiplication), and ε just plays the role of a fixed Monte Carlo sample at this step. Stochasticity is "pushed outside" the deterministic computation, so backprop works.
Now for the math. The goal is to maximize the log-likelihood log p(x) of the training data under the model. The catch is that p(x) involves marginalizing over the latent variable:
p(x) = ∫ p(x | z) p(z) dz
That integral is intractable for any non-trivial decoder. The variational trick is to introduce an approximate posterior q(z|x) — the encoder — and derive a lower bound on log p(x) that is tractable. Here's the three-line derivation:
The bound has two terms with clear interpretations:
Reconstruction term E_q[log p(x|z)] — given a code z sampled from q(z|x), how well does the decoder reconstruct x? Maximizing this term pushes the encoder/decoder pair to produce accurate reconstructions, just like a plain autoencoder.
KL term KL(q(z|x) ‖ p(z)) — how far is the encoder's output distribution q(z|x) from the prior p(z) = N(0, I)? Minimizing this term pushes every q(z|x) to look like the standard Gaussian, which is what makes sampling at test time work.
These two terms pull in opposite directions. The reconstruction term wants q(z|x) to be sharply concentrated around a unique code per input (so the decoder can perfectly reconstruct). The KL term wants every q(z|x) to be diffuse and look like N(0, I) (so the latent space is smooth and samplable). The optimal VAE balances these.
For a Gaussian q(z|x) = N(μ, σ²) and prior p(z) = N(0, I), the KL term has a closed form:
So you never have to numerically integrate — both terms are differentiable closed-form expressions of the encoder outputs.
Walking the VAE Latent SpaceInteractive
Once trained, the VAE's latent space is smooth and continuous. Pick any two real samples; their codes are nearby in latent space; the line between them decodes into a smooth morph. This is the famous 'walking the latent space' property that makes VAEs useful for interpolation tasks.
The two best-known VAE failure modes are flip sides of the same problem.
Posterior collapse: the KL term wins so completely that the encoder outputs q(z|x) ≈ N(0, I) regardless of input. The latent code carries no information about x; the decoder learns to ignore z entirely and produce the marginal data mean. Reconstructions are blurry and unrelated to inputs.
Blurry VAE samples: even when the model trains "correctly," VAE samples have a characteristic softness that GAN samples don't. The reason is the maximum-likelihood objective combined with limited decoder capacity — when the decoder is uncertain about the high-frequency details of x given z, the optimal MLE prediction is the average of plausible outcomes, which looks blurry.
In 2017, Higgins et al. proposed a one-character tweak to the VAE objective that opened a new line of research:
L = E_q[log p(x|z)] - β · KL(q(z|x) ‖ p(z))
With β = 1, this is the standard VAE. With β > 1, the KL term is up-weighted — the model is pushed harder to make q(z|x) match N(0, I). Empirically, this encourages disentangled latent representations: individual dimensions of z come to control individual semantic factors (rotation, color, lighting, identity) instead of all being entangled.
The tradeoff is straightforward: β > 1 hurts reconstruction quality (because the KL term is pulling harder) but improves the interpretability of the latent space. β-VAE is the foundation of much of the disentangled-representation research that came after.
Quick check
You train a β-VAE with β = 4 instead of β = 1, hoping to get more disentangled latents. What changes about the model's behavior?
VQ-VAE (van den Oord et al. 2017) replaces the continuous Gaussian latent with a discrete codebook. Instead of z being a vector of real numbers, z is an index into a learned codebook of K vectors. The encoder outputs are snapped to the nearest codebook entry; the decoder reconstructs from the snapped vector.
Why discrete? Because once your latents are tokens, you can train an autoregressive model (transformer or PixelCNN) over latent token sequences — and that autoregressive model gives you sharp, high-quality samples that pure continuous VAEs can't match. VQ-VAE is the substrate underneath many modern image and audio generative systems: DALL-E v1 used a VQ-VAE for image tokenization, MuseNet and Jukebox used VQ-VAEs for audio, and modern multimodal models like Parti and Make-A-Scene use VQ-VAE-like tokenization.
VQ-VAE is your bridge from VAEs (this lesson) to autoregressive image models (forward-referenced in track-06 generative AI).
Time to see this work end-to-end. The cleanest demo of GAN training is on 1D data, where you can plot the generator's output distribution against the real one at every step.
Loading visualization...
This is a toy implementation — no autograd, no minibatch tricks, no fancy regularizers. Production GANs would use PyTorch, the Adam optimizer, batch normalization, and either WGAN-GP or spectral normalization. But the core dynamic is identical: G and D climb against each other, and you can watch G's output distribution morph from random noise into a match for the target.
Two failure modes to provoke and inspect:
Mode collapse: shrink G's hidden width to 4. G has less capacity to model the full Gaussian and tends to collapse its output to a single mode (the std of fake_final crashes near zero).
D too strong: set LR for D to 0.1 (10x higher than G's). D learns to perfectly separate real from fake almost immediately, and G's gradient signal vanishes (with this non-saturating loss it's less catastrophic than minimax, but G still struggles to make progress).
The VAE side is mathematically clean enough to demo end-to-end on a toy 2D dataset.
Loading visualization...
This is the disentanglement-vs-fidelity tradeoff visible in code. β = 0.1 produces the best reconstructions and the worst-aligned latent space (the encoder has freedom to use whatever code shape minimizes recon error). β = 4 produces the worst reconstructions but the tightest match to N(0, I), giving you a latent space that's smooth and samplable but loses some detail.
Production VAEs add tricks on top of this skeleton: KL annealing (start β near 0 and ramp up to 1 over training to avoid early posterior collapse), free bits (don't penalize KL below a small floor, encouraging the encoder to use some non-trivial capacity), and β scheduling tied to data difficulty. But the core tension between reconstruction and KL is the same.
Now that you've seen both, here's the comparison at a glance:
Axis
GAN
VAE
Density estimation
Implicit — no p(x) you can evaluate
Explicit lower bound on log p(x)
Sample sharpness
Very high — sharp, photorealistic
Often blurry — MLE smoothing
Training stability
Notoriously fragile pre-WGAN; better with WGAN-GP/spectral norm
Very stable — standard SGD just works
Latent space
No semantic structure unless you bolt it on (StyleGAN tricks)
Smooth, samplable, often somewhat disentangled
Mode coverage
Prone to mode collapse
Captures full distribution by construction
Conditional generation
cGAN, pix2pix, CycleGAN
Conditional VAE, very natural via posterior
The two paradigms historically had complementary strengths: GANs won on sample quality, VAEs won on training stability and latent interpretability. By 2020 the field had largely settled into "use a GAN if you only care about samples, use a VAE if you need a structured latent space, use a flow if you need exact density." Then diffusion models came along and beat both on sample quality while being even more stable than VAEs — which is why GANs are no longer the default for new image generation work in 2025.
But the ideas persist. Adversarial losses are still bolted on top of other models to sharpen outputs (the VAE-GAN literature, the GAN-style refiner in NVIDIA's facial reenactment systems). Variational inference is the substrate for diffusion models themselves — diffusion is a hierarchical VAE in disguise. And the reparameterization trick, posterior collapse, mode coverage — all the concepts you learned in this lesson — show up in every modern generative paper, just dressed in different notation.
You came in from the autoencoder lesson and you're leaving toward diffusion. Here's how this lesson sits between them:
From autoencoders to VAEs: replace the deterministic bottleneck with a stochastic Gaussian, add the KL regularizer, and apply the reparameterization trick so gradients still flow. The encoder-decoder shape is identical; the bottleneck is now probabilistic.
From GANs to diffusion (forward ref to track-06): diffusion models drop the adversarial game entirely. Instead, they train a single denoising network to reverse a fixed noise corruption process — turning a Gaussian into a sample via many small steps. The training objective is a weighted reconstruction loss (no GAN-style discriminator). This trades GAN's instability and mode collapse for a slower (multi-step) inference at much higher quality.
From VAEs to diffusion (forward ref to track-06): diffusion can be derived as a deep hierarchical VAE where the encoder is fixed (just adds Gaussian noise at each step) and only the decoder is learned. The ELBO derivation in this lesson carries over almost verbatim — just summed across noise levels instead of being a single-step bound.
From autoencoders / VAEs to masked autoencoders (MAE): the modern self-supervised vision recipe (He et al. 2021) is a denoising autoencoder applied to masked image patches. After pretraining, you throw away the decoder and use the encoder as a vision backbone. MAE is a direct descendant of the encoder-decoder shape in this lesson and the previous one.
From VAEs to VQ-VAE to autoregressive image models (forward ref to track-06): discrete VAE latents let you train a transformer over latent tokens. This is the architecture behind DALL-E v1, Parti, Make-A-Scene, and the latent space of most modern multimodal image models.
The encoder-decoder shape is the through-line. Every modern generative model is either an autoencoder with a clever twist or two such models stacked together.
Generative modeling is about p(X), not p(Y|X). You're modeling the full input distribution, which unlocks sampling, density estimation, anomaly detection, conditional generation, and latent-space interpolation in ways a classifier can't
GANs play a game; VAEs solve an optimization. GANs train two networks adversarially with no explicit density; VAEs derive a tractable lower bound on the log-likelihood and optimize it directly with the reparameterization trick
The original GAN minimax loss saturates. When D is winning, G's gradient vanishes; the non-saturating loss G = -log D(G(z)) and the Wasserstein loss (WGAN, WGAN-GP) both fix this in different ways
Mode collapse is the GAN failure mode you'll see most often. Symptoms are low-diversity samples and missing data modes; remedies are WGAN-GP, spectral norm, minibatch discrimination, or pac-GAN style multi-sample inputs to D
The VAE encoder outputs a distribution, not a point. Q(z|x) = N(μ, σ²), the reparameterization trick z = μ + σ·ε makes gradients flow, and the prior p(z) = N(0, I) lets you sample novel z at test time and decode coherent outputs
The ELBO has two terms with opposite pressures. Reconstruction wants q(z|x) sharp and per-sample; KL wants q(z|x) diffuse and prior-matched; the balance determines whether you get a useful generative model or a posterior collapse / a blurry baseline
β-VAE trades reconstruction for disentanglement. Β > 1 up-weights KL, pushes the latent space toward independent factors, and is the foundation of disentangled representation research
GANs and VAEs are stepping stones to diffusion. Diffusion models inherit variational inference from VAEs and architectural ideas from GANs, while sidestepping both adversarial instability and posterior collapse
Why is the reparameterization trick necessary for VAEs?
You've seen the two foundational neural generative models. Both shape every modern image generator running in production today, even when those generators are diffusion-based — because diffusion models inherit the variational framework from VAEs and the adversarial sharpening idea from GANs. Next up in the deep-learning track: the techniques that took models like StyleGAN, BERT, and GPT from research demos to systems you can actually train, deploy, and keep running at scale.