In 2020 GPT-3 was trained on roughly 500B tokens scraped from the open web — Common Crawl, books, Wikipedia, code. That felt huge. Four years later it looks small.
By 2024 Meta disclosed Llama 3 used 15 trillion tokens of pre-training data — a 30x jump in four years. Chinchilla-style scaling laws say a frontier model with ~400B parameters wants on the order of 8T tokens of high-quality text just to be compute-optimal, and frontier labs are pushing well past that. Meanwhile the high-quality web has not grown 30x. Most of the genuinely useful prose, code and math worth training on is already in the bag — Epoch AI estimated in 2024 that we will exhaust the stock of high-quality public English text somewhere between 2026 and 2032 at current consumption rates.
By 2025-2026 the answer the field landed on is blunt: frontier models are now more than 50% synthetic data. Microsoft's Phi-4 disclosed in December 2024 that its training mix was effectively 100% synthetic (curated + generated). Llama 4 disclosed substantial synthetic contributions to its post-training stack. Claude and GPT-class models are rumoured to be heavily synthetic, particularly for reasoning, code and math. The interesting question is no longer "do we use synthetic data?" — it is "how do you generate training data that is better than what is on the web?"
Will we run out of data? Limits of LLM scaling based on human-generated data
Villalobos et al. (2024)
Epoch AI's projection: stock of high-quality public text gets exhausted between 2026 and 2032 at current consumption rates.
Microsoft's Phi line (Phi-1 → Phi-1.5 → Phi-2 → Phi-3 → Phi-4) is the single most influential public demonstration that small models can punch above their weight if you feed them better data instead of more data. The recipe is straightforward.
Curate a seed list of educational concepts — topics from textbooks, programming tutorials, mathematics curricula, reasoning patterns.
Prompt a strong teacher model (originally GPT-3.5, later GPT-4) to generate "textbook-style" expositions of each concept: definitions, examples, worked problems, common pitfalls.
Filter aggressively for quality. Phi-1 used an educational-content classifier; later iterations added perplexity gating, deduplication, and topic balance.
Train a small model from scratch on the resulting synthetic corpus.
Textbooks Are All You Need
Gunasekar et al. (Microsoft) (2023)
The Phi-1 paper. A 1.3B model trained on 7B tokens of synthetic Python textbooks beat much larger models on HumanEval.
Textbooks Are All You Need II: phi-1.5 technical report
Li et al. (Microsoft) (2023)
Generalises the textbook approach beyond code into common-sense reasoning.
The headline result of Phi-2 (released late 2023) was that a 2.7B-parameter model trained mostly on synthetic textbook-quality data outperformed Llama-2-7B — a model with ~2.6x more parameters — on a broad slate of common-sense and reasoning benchmarks. Phi-4 (December 2024) pushed this further: 14B parameters, ~100% synthetic training mix, performance competitive with much larger open-weights baselines.
Phi-4 Technical Report
Abdin et al. (Microsoft) (2024)
14B parameters, training mix dominated by synthetic data generated and filtered with a careful curriculum.
What Do You Think?
Phi-4's training data is roughly what percentage synthetic?
#Self-Instruct and Alpaca: bootstrapping SFT from 175 seeds
Long before Phi-4, the cheapest way to make a fine-tuned chat model was to copy ChatGPT. The mechanism that made this concrete was Self-Instruct.
Self-Instruct: Aligning Language Models with Self-Generated Instructions
Wang et al. (2022)
The seminal paper that showed you can bootstrap a high-quality instruction-tuning dataset from a tiny seed pool plus a strong teacher LLM.
The recipe is almost embarrassingly simple:
Hand-write ~175 seed instructions covering a diverse set of tasks — summarise this, explain that, write code for X, classify Y.
Repeatedly sample 8 of these as in-context examples and prompt a strong LLM ("write a new instruction in the style of the examples").
Have the same LLM write a response to its own newly-generated instruction.
Filter: drop near-duplicates, drop instructions that produce empty / refused / malformed answers, drop low-diversity outputs.
Fine-tune a target model on the resulting (instruction, response) pairs.
When Stanford applied this in March 2023 to GPT-3.5-derived completions over a Llama-7B base, they produced Alpaca — the model that kicked off the open-weights instruction-tuning era. The training-data cost was a few hundred dollars of OpenAI API time; the human cost was the 175 seed prompts.
Stanford Alpaca: An Instruction-following LLaMA Model
Taori, Gulrajani, Zhang et al. (2023)
Applied Self-Instruct to text-davinci-003, fine-tuned Llama-7B on 52K synthetic instructions. Cost: ~$600 of API calls.
This is the moment "training your own ChatGPT clone" became something a graduate student could do over a weekend. It also forced every major lab to think carefully about whether teacher-output licensing terms applied to distilled students.
#Evol-Instruct: turn easy problems into hard problems
Self-Instruct gives you breadth — many tasks, many surface forms. It does not give you depth. The instructions a strong LLM hands you when asked for "a new instruction in the style of the examples" tend to cluster around the difficulty of the seed set. WizardLM (April 2023) introduced Evol-Instruct to fix that.
WizardLM: Empowering Large Language Models to Follow Complex Instructions
Xu, Sun, Xu et al. (2023)
Introduced Evol-Instruct — four prompt-engineered operators that iteratively make seed instructions harder. Spawned WizardCoder and WizardMath.
The trick is to treat instruction-difficulty as something you can climb, not something you can only sample. Evol-Instruct uses four prompt-engineered "evolution operators":
Operator
What it asks the teacher to do
Example
Add complexity
Add a new constraint or requirement
"Now require the answer to also handle negative inputs"
Deepen
Push the topic into more specialised territory
"Rephrase using domain-specific terminology"
Concretize
Replace abstract terms with concrete entities
"Use a specific real-world example instead of the abstract case"
Increase reasoning
Force multi-step rather than single-step thinking
"Require the answer to combine at least three independent facts"
You apply one operator per iteration. A seed like "list three benefits of exercise" might evolve, after five rounds, into something like "for a sedentary office worker in their 40s with mild hypertension, compare three exercise regimes by VO2-max impact, joint-loading risk, and time cost, and justify which to start with assuming a 30-minute weekday window." Same shape, much harder.
The resulting WizardLM models (and the derivative WizardCoder for code, WizardMath for math) consistently outperformed Alpaca on benchmarks that test multi-step reasoning, which is exactly the dimension Evol-Instruct was designed to amplify.
Quick check
Which of these is NOT one of the four canonical Evol-Instruct operators?
Self-Instruct + Evol-Instruct give you breadth and depth, but both still draw from the teacher's default voice. The model writes the kind of instructions a model tends to write — clean, well-formed, vaguely tutorial. Real users are far weirder. They ask half-formed questions in second languages, they mix domains, they have specific occupations and obsessions. Persona Hub (Tencent AI Lab, May 2024) showed you can recover the long tail of human queries by conditioning generation on personas.
Scaling Synthetic Data Creation with 1,000,000,000 Personas
Chan, Wang, Yu, et al. (Tencent AI Lab) (2024)
Built a billion-persona library and used it to condition synthetic data generation. Demonstrated dramatic diversity gains on math, code and tool-use benchmarks.
The pipeline:
Build a large persona library. Persona Hub describes ~1 billion personas as Cartesian products of occupation × interests × demographic features × situational context. A persona might be "a 52-year-old veterinary surgeon in rural Wales who is learning Python to automate clinic billing."
Condition every synthetic prompt on a sampled persona: "Write an instruction this person would realistically send to an AI assistant."
Generate the response as normal — possibly with Evol-Instruct on top.
Filter and dedupe globally.
The diversity gain is dramatic. Without personas, "ask the model to generate an instruction" produces a heavy concentration of generic tasks. With personas, you cover the actual surface of human queries — including the long tail that benchmarks struggle to measure but real deployments live in.
Quick check
What is the main benefit persona-conditioned synthesis adds over plain Self-Instruct?
A great deal of synthetic-data work is dressed-up distillation: have a strong model generate data, train a smaller model on it. This is how Alpaca was built; it is how a great deal of the open-weights chat ecosystem still works. The recipe is simple, but it has two failure modes you have to engineer around.
Failure mode 1 — coverage collapse. The student inherits whatever the teacher chooses to write, which is a strict subset of the original web. Rare facts, niche dialects, unusual reasoning paths disappear.
Failure mode 2 — hallucination amplification. The teacher is wrong some of the time. The student is trained as if the teacher is always right. The student learns to confabulate confidently exactly where the teacher does — and gets graded on that, because the LLM-as-judge in the eval loop is from the same model family.
The standard mitigations: mix synthetic with a substantial real-data tail, regenerate from the base teacher periodically rather than chaining student-of-student-of-student, filter aggressively with a quality classifier, and use diverse personas / decoding temperatures to avoid mode collapse.
For mathematics, synthetic data is uniquely powerful because math has a cheap, exact verifier — you can check whether the final number is correct. MetaMath (2023) and the MathInstruct family pioneered the pattern.
MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models
Yu, Jiang, Shi et al. (2023)
Augments MATH and GSM8K via rephrasing, backward reasoning, and self-verification. The verifier filter is the load-bearing piece.
The recipe:
Start with a real corpus (MATH, GSM8K, NumGLUE).
For each problem, ask the teacher LLM to generate multiple solution chains with explicit chain-of-thought.
Use the same teacher as a verifier — give it the problem and a candidate solution, ask "is this correct?"
Keep only the (problem, chain-of-thought, answer) triples where the verifier agrees and the final numeric answer matches a reference (when available).
Round-trip filter: paraphrase the question; run the solver again; demand the same final answer. Inconsistency under paraphrase is a strong signal of luck rather than reasoning.
For code, the same pattern is even cleaner — the verifier is the language interpreter. CodeAlpaca (March 2023) and Magicoder / OSS-Instruct (Wei et al., 2023) generate problem-solution pairs and execute the solution against generated test cases. A solution that doesn't compile, or that fails any test, is dropped. This catches a class of bugs that text-level inspection alone cannot.
Magicoder: Empowering Code Generation with OSS-Instruct
Wei, Wang, Lin et al. (2023)
Conditions instruction generation on real open-source code seeds, then verifies via execution. The execution loop is what makes the synthetic data trustworthy.
For SFT, a teacher LLM generates the labels. For RLHF / DPO, you traditionally need humans (or at least a human-trained reward model) to provide preference judgements. Self-Rewarding Language Models (Yuan et al., Meta, January 2024) asks whether the model can grade its own outputs well enough to skip humans entirely.
Self-Rewarding Language Models
Yuan, Pang, Cho et al. (Meta) (2024)
Iterative DPO using the model itself as the preference judge. No human labellers needed beyond the seed instructions.
The loop:
Start with seed instructions (Self-Instruct style).
Sample multiple response candidates per instruction.
Use the same model as an LLM-as-judge to rank the candidates against a rubric (helpfulness, correctness, conciseness).
Construct preference pairs (chosen, rejected) from the rankings.
Run DPO on these synthetic preferences.
The post-DPO model is both a better responder AND a better judge — iterate.
The Meta paper showed that this iterative loop produces consistent gains over multiple rounds, with the model improving as both responder and grader. The human input is minimal — only the seed instructions; the rest is the model judging itself.
What Do You Think?
Self-Rewarding training (Yuan et al., 2024) requires what kind of human input?
Synthetic data is not free. There is a specific, mathematically tractable failure mode if you do it wrong: model collapse.
AI models collapse when trained on recursively generated data
Shumailov, Shumaylov, Zhao et al. (2024)
Published in Nature, July 2024. Showed empirically and theoretically that recursive training on synthetic outputs causes loss of tail-of-distribution coverage and eventual mode collapse.
The setup is simple. Train model A on real data. Generate synthetic samples from A. Train model B only on those synthetic samples. Generate samples from B. Train model C on B's samples. Iterate.
What happens — and Shumailov's group showed this in language, image, and Gaussian-mixture settings — is that the distribution narrows. Tails disappear first. Rare events, rare vocabulary, rare reasoning paths drop below the model's representational threshold and never come back. After enough generations the model is fluent but pathologically repetitive, having forgotten the part of the data manifold that was never well-represented in the synthetic outputs.
The empirical demonstrations in the paper are striking. For a Wikipedia-trained OPT-125M model resampled across nine generations, the model's outputs on the prompt "some books that some recommend to others" devolve from coherent recommendations into surreal repetition about jackrabbits within five generations. The math case is even cleaner — the Gaussian-mixture variance progressively shrinks until the support is a single mode.
What Do You Think?
What is the canonical defence against model collapse?
Synthetic data is also how modern safety training works. The Constitutional AI line at Anthropic was the first widely-known instance.
Constitutional AI: Harmlessness from AI Feedback
Bai, Kadavath, Kundu et al. (Anthropic) (2022)
Replaced human harmfulness labels with model-generated critique-and-revise pairs guided by a written constitution. The synthetic preference data drove RLAIF training.
The pipeline turns safety alignment into a self-supervised synthetic-data exercise:
Generate a response to a potentially harmful prompt — let the assistant answer freely.
Critique the response against a written constitution (a list of principles — be helpful, avoid harm, respect autonomy, etc.). The critic is the same model, in a different prompt role.
Generate a revised response that addresses the critique.
Use (revised, original) as a synthetic preference pair: chosen = revised, rejected = original.
Train via DPO / RLAIF on the resulting preference dataset.
This is synthetic-data generation pointed at alignment instead of capability. The human input shrinks to the constitution itself — a few hundred words. Anthropic's Claude line was the first commercial product to train its safety behaviour predominantly on data of this shape; the technique has since been widely adopted under various names (RLAIF, self-critique, principle-guided revision).
Verify. Code → execution. Math → calculator + round-trip paraphrase. Prose → LLM-as-judge with a rubric. Without a verifier, you are gambling.
Mix with real data. Typical frontier mixes today are 50-90% synthetic, with the rest real for tail-of-distribution grounding. The Phi line is the high-end of synthetic share; most other labs sit lower.
Train and evaluate. Eval on held-out benchmarks AND on red-team probes targeting known synthetic-data failure modes.
Iterate on the gap. Identify skills where the student underperforms. Generate more data targeting those skills. Loop.
The economic argument is brutal. Suppose you need 1 trillion tokens of high-quality instruction-tuning data.
Source
Unit cost
Total
Notes
GPT-4o-mini at $0.60 / 1M output tokens
$0.60 / M
~$600,000
Wall-clock: days, parallelised across many API keys.
Human labellers at $25 / hour, ~500 tokens / hour quality output
$50 / 1K tokens
~$50,000,000
Wall-clock: years of human-time.
Human experts (lawyers, doctors, PhDs) at $150 / hour
$300 / 1K tokens
~$300,000,000
Wall-clock: longer; expert recruitment dominates.
That is ~100x in favour of synthetic vs general human labour and ~500x against domain experts, at comparable token counts. The catch — repeatedly — is verification. The $600K cost gets you raw tokens; getting them to ship-quality requires the verifier infrastructure (execution sandboxes, math checkers, judge-LLMs, classifiers) on top. Even with the verifier overhead, the cost gap to human labelling is generally one to two orders of magnitude.
This is why the frontier shifted. The economics of "ask GPT-4o-mini to write a million textbook problems and filter the ones that pass an automatic check" simply destroys the economics of "hire a thousand contractors to write the same problems." The only thing humans are still essential for is the verifier itself — designing the rubric, building the test cases, deciding what counts as ship-quality. The labels themselves are cheap.
1The public high-quality web is effectively exhausted; frontier models are now >50% synthetic data, and Phi-4 disclosed ~100% synthetic.
2The Phi recipe = curated seed concepts + GPT-4 textbook-style generation + aggressive quality filtering. It is the public proof that small models can punch up if fed better data.
3Self-Instruct (Wang 2022) bootstraps SFT data from ~175 seed instructions. Evol-Instruct (Xu 2023) iteratively raises difficulty via four operators: add-complexity, deepen, concretize, increase-reasoning.
4Persona-driven synthesis (Persona Hub, 2024) recovers the long tail of human queries by conditioning generation on diverse personas.
5Synthetic-math and synthetic-code rely on EXACT verifiers (execution, calculators, round-trip paraphrase). Without verification you amplify the teacher's biases.
6Self-Rewarding (Yuan 2024) and Constitutional AI (Anthropic 2022) close the RLHF loop with the model judging itself — humans only write seed instructions and principles.
7Model collapse (Shumailov 2024) is the canonical failure mode of recursive synthetic training. Defences: mix with real data, regenerate from base, use diverse personas, dedupe aggressively.
8Cost: ~$600K for 1T synthetic tokens via GPT-4o-mini vs ~$50M for the same volume of human labels. ~100x advantage — but only if your verifier infrastructure is sound.
Synthetic data closes the NLP & Transformers track — thirty-five lessons from tokenization to RLHF, FlashAttention, MoE, Mamba, ViT, mech-interp, tool-calling, knowledge-editing, distributed pretraining, and the 2026 evaluation frontier. The text-and-token half of generative AI is now yours. Track 6 (Generative AI) carries the same probabilistic objectives — likelihood, ELBO, score, KL — into pixels, video, and audio, and Track 7 (Reinforcement Learning) doubles back to the RLHFRLHFRLHF (Reinforcement Learning from Human Feedback) fine-tunes language models using a reward model trained on human preference comparisons.Learn more → recipe at full algorithmic depth so you can read the GRPO / DPO / PPO literature on its own terms.