The LLM gets the answer wrong. You ask "are you sure?" and it correctly finds its own mistake. That's not a bug — that's a 10–30% quality lift on hard tasks, hiding in plain sight, at the cost of 2x tokens and ~30% latency. Reflexion (Shinn 2023) is the canonical pattern; o1, DeepSeek-R1, and Claude's extended thinking are variants.
Learning Objectives
After this lesson, you will be able to:
Explain why adding a self-critique step lifts agent quality 10-30% on hard tasks — at the cost of 30% latency and 2x token usage
Shinn 2023 names three distinct roles inside a Reflexion agent. Conflating them is the most common implementation mistake.
Component
Job
Common implementation
Actor
Attempt the task. Produces an action / output.
The "main" LLM call, usually a capable model (Claude Opus, GPT-4o). Sees the task plus all prior reflections in its context.
Evaluator
Score the actor's output. Pass/fail, possibly with structured feedback.
Either (a) external code (unit tests, regex match, ground-truth comparison), or (b) an LLM critic prompt that returns JSON like {score, passing, notes}.
Self-reflection
Write a verbal lesson learned from the failed attempt. Compresses (action, evaluation) into one or two sentences of actionable advice.
A separate LLM call with a reflection-specific prompt. Output is appended to memory.
The fourth ingredient is the memory itself — a list of natural-language reflections retained across trials. Crucially, reflections do NOT replace the failing actions in memory; they are additional context. On trial t, the actor sees: original task + the running reflection list. It cannot see the raw failing outputs from earlier trials (Shinn argues this prevents the model from anchoring on its prior bad answer).
1. Actor: attempt the task → action_1
2. Evaluator: score the result
- Pass → return result
- Fail → continue
3. Self-reflection: write a lesson on why action_1 failed
4. Memory: append the reflection to a persistent reflection list
5. Actor (retry): attempt the task with reflections in context → action_2
6. Repeat from step 2 (with max retries)
A worked example on a math problem makes the roles concrete:
Trial 1
Task: "What is the smallest positive integer divisible by 1..10?"
Actor: "It's 2 * 3 * 5 * 7 = 210." (uses unique primes <= 10)
Eval: FAIL (expected 2520; got 210)
Reflect: "I only multiplied prime factors. I must use the LCM, which means
including the highest power of each prime <= 10: 2^3, 3^2, 5, 7
= 8 * 9 * 5 * 7 = 2520."
Trial 2 (memory now contains the trial-1 reflection)
Actor: "LCM(1..10) = 2^3 * 3^2 * 5 * 7 = 2520."
Eval: PASS
The reflection isn't just "be more careful" — it names the specific misconception (primes vs. prime powers) and the specific fix. That specificity is what makes trial 2 different from trial 1.
What Do You Think?
In the math example above, suppose Trial 1's reflection had been only 'I made an arithmetic error.' What's the most likely Trial 2 outcome?
Tests · Verify Reflexion catches cases where the first attempt forgets case-insensitivity or punctuation handling. Verify it converges within 3 attempts on most palindrome cases.
#Self-Refine: Reflexion Without an External Evaluator
The closely related Self-Refine technique (Madaan, Tandon, Gupta, Hallinan, et al., 2023) drops the external evaluator entirely. The same LLM generates an output, then critiques its own output, then revises — all from prompts alone, no ground truth required.
The pipeline is:
output_0 = actor(task)
for k in range(max_iters):
feedback_k = self_critic(task, output_k)
if feedback_k.contains("no changes needed"):
return output_k
output_{k+1} = actor(task, output_k, feedback_k)
Where Reflexion needs a pass/fail signal (tests, ground truth, downstream environment), Self-Refine works on any open-ended task — summarization, code style, story generation. Madaan et al. reported 5-40% gains across math, code, dialogue, and constrained generation tasks, without any external feedback channel.
Two practical differences to keep in mind:
Risk of degradation. Without ground truth, the critic can push the output in the wrong direction. A safety check: keep output_0 and only commit to output_k if the critic explicitly approves it; otherwise return the earliest "good enough" version.
Domain mismatch. Self-Refine shines on tasks with clear stylistic or structural criteria (e.g., "make this code more readable"). It struggles when correctness is what you want — there, you still need Reflexion with executable checks.
Quick check
Your team wants to improve the quality of LLM-generated meeting summaries. There's no ground-truth 'correct summary'. Which pattern fits?
In practice, the cross-trial-within-episode regime is what most production code does. Persistent cross-task reflections sound powerful but introduce a new failure mode: reflections from unrelated tasks contaminating context ("I should remember the array was 0-indexed last time" applied to a task that has nothing to do with arrays). If you do go persistent, gate retrieval by task similarity or summarize aggressively.
The 2024-2026 frontier: train models to do Reflexion internally rather than wrapping them with an external loop. OpenAI o1 generates a long internal "reasoning chain" that includes self-critique steps — Reflexion built into the weights.
For most production: external Reflexion loops still win for tasks where you have explicit ground truth (tests, fact-check sources). For exploratory reasoning: o1/o3-class models with built-in critique are simpler.
Why does Reflexion improve LLM agent quality without any model retraining?
What Do You Think?
A team adds Reflexion to a customer-support chatbot. Quality on hard tickets goes up 12%, but the median p95 latency climbs from 2s to 6s. What's the most defensible production decision?
Reflexion lets agents learn from mistakes within a session. Next: Anthropic's catalog of workflow patterns that prescribe WHEN to use ReAct vs Reflexion vs structured chains.