Building an Agent (Capstone)

You've learned the pieces. Now wire them into something that survives Monday morning in production: an agent with MCP tools, ReAct reasoning, agentic memory, an evaluator-optimizer loop, hard cost caps, distributed tracing, and a regression eval set in CI. This is the 12-point checklist that separates a Devin demo from a Devin you'd actually ship.

Learning Objectives

After this lesson, you will be able to:

Architect a production-grade autonomous agent end-to-end — tool use + memory + planning + reflection + safety + observability + evaluation
Wire together MCP servers + ReAct + agentic memory + evaluator-optimizer into a Devin/Manus-style coding or research agent
Apply the 12-point production hardening checklist before shipping any agent to users
Identify the most common production failure modes in agentic systems and the architectural patterns that prevent them

Prerequisites

Before You Start

Review

Why

Why This Matters

Why solo devs and tiny startups built billion-dollar agentic products in 2024-2026 -- Cursor (Anysphere $9B valuation), Devin (Cognition $4B), Manus (China's GPT-Operator competitor), Lindy, Cognition's other products — all built by teams under 50 people using off-the-shelf models + clever agent design
Why production agents look nothing like the LangChain quickstart -- caching, fallbacks, streaming, observability, eval pipelines, safety layers, cost monitoring, A/B testing, human-in-the-loop infrastructure are all required at scale
Why this capstone matters more than any individual technique -- you can know ReAct + tools + memory in isolation but still ship a fragile agent; productionizing requires synthesizing everything
Build this --> Build a Devin/Manus-style coding agent in ~300 lines: takes a GitHub issue, plans subtasks, edits files, runs tests, iterates with reflection, opens a PR; deploy to Modal/Railway with full observability

The Production Agent Architecture

┌─────────────────────────────┐ │ USER REQUEST (issue, query)│ └─────────────┬───────────────┘ │ ┌─────────────▼───────────────┐ │ 1. Intake & Sanitize │ │ - Rate limit │ │ - PII redaction │ │ - Prompt injection scan │ └─────────────┬───────────────┘ │ ┌─────────────▼───────────────┐ │ 2. Plan │ │ - Decompose into steps │ │ - Dispatch to right │ │ workflow pattern │ └─────────────┬───────────────┘ │ ┌─────────────▼───────────────┐ │ 3. Execute (ReAct loop) │ │ - Tool calls via MCP │ │ - Memory updates │ │ - Cost tracking │ │ - Max iterations cap │ └─────────────┬───────────────┘ │ ┌─────────────▼───────────────┐ │ 4. Evaluate (Reflexion) │ │ - Self-critique │ │ - Run tests/checks │ │ - Retry if failed │ │ - Confidence threshold │ └─────────────┬───────────────┘ │ ┌─────────────▼───────────────┐ │ 5. Safety Filter │ │ - Output classifier │ │ - Citation validation │ │ - Refusal detection │ └─────────────┬───────────────┘ │ ┌─────────────▼───────────────┐ │ 6. Human-in-Loop Gate │ │ - High-stakes actions │ │ require approval │ └─────────────┬───────────────┘ │ ┌─────────────▼───────────────┐ │ 7. Stream Response │ │ + Log trace │ │ + Update memory │ │ + Run eval pipeline │ └─────────────────────────────┘

#	Concern	Solution
1	Cost runaway	Per-user/session token budget; hard cap at 100K tokens
2	Infinite loops	Max iterations 10-20; no-progress detection; LLM stuck-detection
3	Prompt injection	Sanitize tool outputs before re-injecting; treat external content as untrusted; never let chunks override system prompt
4	Tool sandboxing	Code execution in containers; file system in scoped temp dirs; web requests via proxy with allow-list
5	Streaming responses	First-token latency < 1.5s; full response < 30s for typical tasks
6	Observability	Distributed tracing with correlation IDs; per-step latency, cost, errors logged
7	Eval pipeline	Golden set + RAGAS-style metrics in CI; fail builds on > 3% regression
8	Fallback behavior	If primary LLM fails, fall back to secondary; if tools fail, return graceful error
9	Safety filter	Output classifier blocks unsafe responses; citation validator catches hallucinations
10	Human-in-loop	Critical actions (delete data, send email, money transfer) require user approval
11	A/B testing	Run 5-10% of traffic on candidate config; compare RAGAS-style metrics
12	Cost monitoring	Per-query and per-user cost dashboards; alert on anomalies

System	Pattern
Devin (Cognition)	Long-running session, full IDE access, browser tool, comprehensive memory, human-in-loop on critical actions
Cursor (Anysphere)	Inline-edit agent + agent mode (multi-file changes) + composer; deep IDE integration
Claude Code (Anthropic)	Terminal-native, MCP server ecosystem, hooks for project-specific commands, sub-agent orchestration
Manus AI (Butterfly)	General-purpose autonomous agent; tool-use heavy; runs in cloud sandboxes
Perplexity Pro Search	Lightweight agentic RAG: retrieve → reason → answer with citations; not a full agent
GitHub Copilot Workspace	Repo-scoped multi-step planning + edits; tight integration with PR workflow

Building an Agent (Capstone)

After this lesson, you will be able to:

Before You Start

The Production Agent Architecture

Your Reflection

Continue Learning

Discussion

#The 12-Point Production Hardening Checklist

#Implementation: Devin-Style Coding Agent

#Production Patterns from Real Systems

#Common Production Failure Modes

#Key Takeaways

#Quick Check

Deep Dive