A stateless agent meets you fresh every conversation — same five questions, same blank slate. A stateful agent remembers your name, your codebase, your preferences. The gap between the two is a vector store, a summarization step, and a thoughtful schema for what's worth remembering. ChatGPT Memory, Claude Projects, and Cursor's codebase index all live here.
Learning Objectives
After this lesson, you will be able to:
Understand the difference between short-term memory (what the AI can see right now in the conversation) and long-term memory (facts saved to a database that survive across sessions)
Explain three ways to give agents memory: keeping a rolling window of recent messages, saving important facts to a vector store, and summarizing old conversations into compact notes
Classify agent memories into three types: episodic (what happened), semantic (facts and knowledge), and procedural (how to do things)
Build a practical memory system that lets an agent remember your name, preferences, and past conversations across sessions
Navigate the tradeoffs between how much the agent can remember, how fast it can recall, and how relevant the recalled memories are
Memory is what turns an AI from a stranger you meet at a party into a friend who actually knows you. This lesson is one of the most practically useful in the whole track -- you will understand exactly why ChatGPT sometimes "forgets" and how to fix it.
Two kinds of memory, with different physics. Short-term memory is the context window: recent messages, tool results, and reasoning traces — fast and complete, but limited in capacity and gone when the session ends. Long-term memory is external storage the agent reads from and writes to — slower and selective, but it survives across sessions and grows without bound. Agents need both, and most agent failures come from putting something in the wrong one.
Short-term (context window) vs long-term (vector store) memory
Try it! Start a new conversation with ChatGPT and tell it "My name is Alex and I love astronomy." Then start a completely new conversation and ask "What is my name?" If it remembers, it is using long-term memory. If it does not, you just experienced the short-term memory limit firsthand. This is the exact problem this lesson teaches you to solve.
An agent without memory is like a person with amnesia. Every conversation starts from scratch. It cannot remember what you discussed yesterday, what your preferences are, or what it learned from previous mistakes. Memory transforms a stateless text generator into something that feels genuinely intelligent -- an assistant that knows you.
Try it: Explore agent memory systemsInteractive
Loading visualization...
Explore the three views above: Architecture shows how short-term (context window) and long-term (vector store) memory relate. Memory Types lets you examine episodic, semantic, and procedural memory with examples. Retrieval Flow walks through a real memory recall scenario step by step.
Short-term memory is what the LLM can see right now -- the messages in the current conversation. Every user message, every assistant response, every tool call and result sits in the context window. The LLM reads all of it before generating its next response.
The context window has hard limits. GPT-4o supports 128K tokens. Claude supports 200K tokens. Gemini supports up to 2M tokens. These sound enormous, but they fill up fast in agentic workflows. A single tool call result might be 2,000 tokens. A research task with 10 tool calls generates 20,000+ tokens of context. Add reasoning traces and the original system prompt, and you can exhaust even a 200K window on a complex task.
When the context window fills up, something has to give. There are three strategies:
Truncation. Drop the oldest messages. Simple but brutal -- the agent forgets the beginning of the conversation, which often contains the original goal and critical context.
Sliding window. Keep the system prompt and the last N messages. Better than raw truncation because the system prompt is preserved, but still loses early context.
Summarization. Periodically compress older messages into a summary. "The user asked about flights to Tokyo. We found ANA at $780 as the cheapest option. The user then asked about hotels." This preserves the key information while freeing up space. It is the most effective strategy but requires an additional LLM call to generate the summary.
The agent starts a new task. The context window contains: system prompt (500 tokens), user message (100 tokens). Total: 600 tokens out of 128,000 available. Plenty of room.
After 5 tool calls, the context has grown: system prompt (500) + user message (100) + 5 reasoning traces (2,500) + 5 tool results (10,000) = 13,100 tokens. Still fine, but growing fast.
After 20 tool calls and extensive reasoning, the context hits 85,000 tokens. The agent starts to lose track of earlier findings. It might re-search for information it already found in step 3. Performance degrades.
A smart memory manager detects the context is 65% full. It summarizes messages 1-15 into a compact 800-token summary: key findings, decisions made, current plan status. The context drops from 85,000 to 32,000 tokens, and the agent regains clarity.
The agent continues with the summary replacing the verbose history. It retains what matters (the ANA flight at $780 is the best option, the user prefers nonstop flights) without the noise (raw API responses, failed searches, verbose reasoning traces). The task completes successfully.
Short-term memory dies when the conversation ends. Long-term memory survives. This is what allows an agent to remember that you prefer window seats, that your project uses Python 3.12, or that the last time it tried approach X it failed because of Y.
There are three architectures for long-term memory, each with different strengths:
The most common approach. Memories are embedded into vectors and stored in a vector database (ChromaDB, Pinecone, Weaviate, FAISS). When the agent needs to recall something, it embeds the current query and searches for the nearest vectors.
How it works: After each conversation, the agent extracts key facts, decisions, and preferences. Each fact is embedded into a vector and stored with metadata (timestamp, topic, confidence). At the start of the next conversation, the agent embeds the user's message, searches for relevant memories, and injects the top-K results into the system prompt.
Strengths: Scales to millions of memories. Retrieval is fast (milliseconds). Semantic search finds relevant memories even when the wording is different -- "book a flight" matches a memory about "airline reservations."
Weaknesses: Embedding models can miss nuanced connections. A memory about "the user dislikes layovers" might not surface when the query is "find flights to Tokyo" unless the embedding model understands the implicit connection. Recall quality depends heavily on the embedding model and chunking strategy.
Loading visualization...
Step through the memory retrieval pipeline above. Watch how the agent's current context gets embedded, compared against stored memory vectors, and the most relevant memories are injected back into the conversation. This is the same RAG pipeline from Track 8, repurposed as an agent's long-term memory system.
Inspired by operating system memory management, MemGPT treats the context window as "main memory" and an external store as "disk." The agent has explicit tools to manage its own memory: memory_save, memory_search, memory_delete. It decides what to page in and page out, just like an OS manages virtual memory.
How it works: The agent has a fixed-size "core memory" section in its context (say, 2,000 tokens) for the most important persistent facts. When it learns something new, it explicitly calls memory_save("user prefers window seats"). When the core memory is full, it pages older memories to the external store and retrieves them with memory_search when needed.
Strengths: The agent has full control over its memory. It decides what is important enough to remember, what to forget, and when to retrieve. This creates more intentional, higher-quality memories compared to automatic extraction.
Weaknesses: The agent must learn to use memory tools effectively. A poorly tuned agent might forget to save important information or save too much trivial detail. The memory management overhead adds tokens to every interaction.
The simplest approach. After each conversation, generate a structured summary: key topics discussed, decisions made, action items, user preferences discovered. Store the summaries as text, searchable by date and topic.
How it works: At the end of a session, the agent (or a separate summarization model) produces a summary like: "Session 2026-03-15: User is planning a trip to Tokyo in March. Found ANA flight at $780 (cheapest nonstop). User prefers nonstop flights and economy class. Next step: find hotels near Shinjuku station."
Strengths: Simple to implement. Summaries are human-readable and debuggable. Works with any storage backend (even a flat file). No embedding model needed.
Weaknesses: Summaries lose detail. The raw context might contain nuances that the summary drops. Search is keyword-based unless you also embed the summaries. Does not scale well to thousands of sessions -- finding relevant memories requires scanning many summaries.
What Do You Think?
An agent has had 100 previous conversations with a user. At the start of conversation 101, how should it decide which memories to load?
What Do You Think?
Your agent has 200K tokens of useful context and a model with a 200K context window. Should you ship it that way?
Loading all 100 summaries would overflow the context window. Loading only the most recent conversation misses relevant older context (maybe the user asked about this same topic in conversation 37). Random loading is wasteful. The right approach is semantic retrieval: embed the user's current message and find the memories most relevant to this conversation, regardless of when they were created. A question about "Tokyo flights" should surface the memory from conversation 15 where Tokyo travel was discussed, not the unrelated conversation from yesterday.
The user says: "What was that hotel you recommended in Tokyo?" This is conversation 47 -- the Tokyo trip planning happened back in conversation 12. The answer is not in the current context window. The agent needs to reach into long-term memory.
The agent scans the current conversation history -- the context window. It finds no mention of Tokyo hotels in this session. Short-term memory does not have the answer. The agent recognizes it needs to search deeper.
The agent decides to search its long-term memory store. It formulates a query from the user's message: "hotel recommendation Tokyo." This query will be used to find semantically relevant memories across all past conversations.
The query "hotel recommendation Tokyo" is embedded into a vector using the same embedding model that encoded all past memories. This vector is compared against every stored memory vector using cosine similarity. The search finds the nearest neighbors -- memories most semantically related to the query, regardless of exact wording.
The vector search returns the top-K matches: (1) "Recommended Hotel Gracery Shinjuku for its Godzilla-themed terrace and proximity to the station" (similarity: 0.92), (2) "User prefers hotels near train stations" (similarity: 0.84), (3) "Tokyo trip dates: March 15-22" (similarity: 0.78). The most relevant memories surface, no matter when they were stored.
The retrieved memories are injected into the agent's context window, typically in a "relevant memories" section of the system prompt. Now the agent has the information from conversation 12 available in conversation 47 -- without needing to load all 46 prior conversations.
The agent responds: "I recommended Hotel Gracery Shinjuku -- it has a famous Godzilla-themed terrace and is right next to Shinjuku station, which I know you prefer. This was for your March 15-22 trip." The response feels personalized and informed because the agent remembered across sessions. The user experiences an assistant that truly knows them.
Cognitive science classifies human memory into three types. The same taxonomy applies to agents, and understanding it helps you design better memory systems:
What happened. Specific events and experiences, anchored in time. "On March 15, the user asked about flights to Tokyo and I found ANA at $780." "In yesterday's session, I tried to use the weather API but it was rate-limited."
Episodic memory is critical for agents that need to recall past interactions, learn from mistakes, and avoid repeating errors. If the weather API was down yesterday, the agent should try a different source today rather than hitting the same wall.
What is true. General facts and knowledge, not tied to specific events. "The user prefers Python over JavaScript." "The company uses PostgreSQL for production databases." "ANA flights to Tokyo are typically cheaper than JAL."
Semantic memory builds up over many episodes. After three conversations where the user chooses Python, the agent extracts the general fact. This is the most valuable type of memory for personalization because it captures stable preferences and knowledge.
How to do things. Learned procedures, workflows, and strategies. "When the user asks for a code review, first run the linter, then check for security issues, then assess readability." "When searching for flights, always check ANA first because they are usually cheapest."
Procedural memory is the hardest to implement but the most powerful. An agent that remembers how it successfully completed a task can replicate that strategy. It is the agent equivalent of building muscle memory -- learned patterns of behavior that improve with experience.
pythonplayground.py · Pyodide
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
Tests · Add confirmFact to boost confidence, recordProcedureOutcome to track success rates, and expireOldEpisodes to clean up stale memories.
Before diving into advanced architectures, here is a practical memory system you can build today. This shows the core pattern: a rolling short-term buffer with overflow to long-term storage.
pythonplayground.py · Pyodide
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
Tests · Build a memory system with short-term buffer overflow to long-term storage, keyword-based recall, and context assembly that combines recent messages with relevant long-term memories.
Different tasks demand different memory systems. Here is when each type pays off:
Memory Type
What It Stores
Example
Best For
Weakness
Short-term (Context Window)
Current conversation messages
"The user just asked about Tokyo flights"
Immediate task context, multi-turn reasoning
Limited capacity, dies when session ends
Long-term (Vector Store)
Embedded facts from past sessions
"User prefers nonstop flights"
Cross-session personalization, preference recall
Embedding quality varies, can miss nuanced connections
Episodic
Timestamped events
"On March 15, I recommended Hotel Gracery"
Recalling specific past interactions, learning from mistakes
Grows unbounded, needs expiration policy
Semantic
General facts and knowledge
"User's project uses PostgreSQL"
Stable preferences, user profile, domain knowledge
Can become stale without update mechanisms
Procedural
Learned strategies and workflows
"When reviewing code, run linter first"
Improving agent performance over time, repeating successful approaches
Hardest to implement, strategies may not generalize
Summarized
Compressed conversation summaries
"Session 42: Discussed Tokyo trip, booked ANA flight"
Lightweight cross-session context, auditability
Summaries lose detail, quality depends on summarizer
Which should you implement first? Start with short-term memory management (summarization to prevent context overflow). Then add semantic memory (key facts about the user). Episodic and procedural memory are advanced optimizations -- add them only when you have the infrastructure to maintain them.
Context window = working memory: everything in the context window is instantly accessible with zero retrieval cost, but is lost at session end — optimize what you put in the context, not just how much
Long-term memory requires explicit write + retrieval decisions: the agent must decide what is worth saving and when to query; a memory system that writes everything and retrieves everything is worse than no memory system
Three memory types serve three purposes: episodic (what happened, event logs) for continuity; semantic (what is true, facts and knowledge) for grounding; procedural (how to do things, tools and skills) for capability
Retrieval precision matters more than recall: flooding the context with loosely relevant memories degrades performance — prefer a precision-focused retrieval strategy that surfaces 3–5 highly relevant memories over 20 marginal ones
Multi-tenant memory isolation is non-negotiable: separate vector store collections or namespaces per user; cross-tenant memory leakage is a data breach, not just a bug
A few projects worth knowing if you build agent memory in 2025:
Letta (formerly MemGPT). The canonical OS-inspired memory architecture: tiered storage (in-context core memory, archival memory, recall memory) with the agent itself issuing tool calls to read/write archives. Reference for the "virtual context window" idea.
Mem0 (2024). Opinionated long-term memory with automatic fact extraction and consolidation; designed to be dropped into any framework.
Zep (2024). Temporal knowledge graph memory; tracks facts about users with valid-from/valid-to timestamps. Strong when user state changes over time.
LangMem and LangGraph stores (2024-2025). First-party memory primitives in the LangChain ecosystem.
OpenAI Memory and Claude Projects (2024-2025). Product-level memory: the assistant remembers user-stated facts across sessions, with explicit user control over the memory store.
The common shape across all of them: a memory write path (extract facts and write them when the user signals importance or the agent notices a stable preference), a memory read path (semantic retrieval, often re-ranked, scoped to this user), and a forget/decay path (so the store doesn't fill up with stale or contradicted facts).
You now understand how agents remember -- from the ephemeral context window to persistent vector stores, from episodic events to semantic facts to procedural strategies. Memory is what transforms a stateless chatbot into a personalized assistant that improves with every interaction. Next up: Planning and Reasoning, where we explore how agents decompose complex tasks and think multiple steps ahead.