#194 — Introducing dreaming: How Anthropic's self-improving agent memory works
July 17, 2026·8 min read
Contents
Why it matters: Claude agents already carry memory across sessions. What Anthropic calls "dreaming" is a separate process that reviews that existing memory afterward, finds what's stale or contradictory, and cleans it up so agents get sharper over time without anyone retraining the model.
How memory actually builds up before dreaming happens
Anthropic's context engineering work went through several stages over roughly a year. A single CLAUDE.md file loaded at the start of a session turned out to work well for steering an agent toward codebase conventions and user preferences, until the file grew too long and started crowding out the context window. The next step gave agents room to write to their own memory during a session, deciding on their own what to keep.
Skills fixed the bloat problem by loading only a short description upfront, similar to scanning book titles on a shelf and pulling one down only when you need it, with the full detail loading only when it's relevant. Current practice treats memory as a plain file system of markdown files that agents search with normal bash and grep commands, so nothing has to load all at once.
Why letting agents manage their own memory only goes so far
Agents writing their own memory during a session run into two problems once you scale past one agent doing one task.
- They split attention between finishing the task and spending effort on memory that only helps a future run, which is a real tradeoff, not something you can just tune away
- They only see their own session, so an agent can't tell that nine other sessions hit the same wall, and a fleet running in parallel has no shared view of the pattern
On top of that, production systems run into multiple agents writing to the same file at once, one agent overwriting shared context that everyone else reads from, and memories that quietly go stale, contradict each other, or in rare cases get corrupted by a prompt injection.
The engineering that has to sit underneath all of this
Anthropic built several concrete safeguards before dreaming comes into play:
- Versioning: track which session triggered an update and who made it, with a way to roll back
- Hashing to control concurrency: an agent hashes memory before drafting an edit and hashes again before saving. If the two don't match, something else changed first, so it pulls fresh data and tries again
- Permissioning: org-wide memory tends to be read-only for individual agents, while an agent's own scratchpad stays writable
- Portability: memory sits behind a clean API so the same curated context works across different tools and product surfaces
Anthropic describes this as standard database practice, versioning, locking, access control, applied to a setting where autonomous agents need to touch it safely.
What dreaming actually does
Dreaming runs separately from any live task, on its own schedule and its own compute. It takes an existing memory store along with a batch of past session transcripts and produces a new memory store: duplicates merged, outdated or conflicting entries replaced, and patterns surfaced that no single session would have revealed. The original store stays untouched. Only the new output store gets written, so you can check it and throw it away if you don't like what it produced.
Anthropic compares this to a school setting: students hand in work, but a head teacher reviews everything across the whole class, catches patterns no single student's work would show, and updates the curriculum accordingly. In practice, an orchestrator agent sends sub-agents to comb through the transcripts, decides which patterns show up often enough to matter, and proposes specific changes along with the transcript examples and how often the issue occurred, so a person can approve or reject each change before it goes live.
Examples of what turns up
- A topic that's missing from an agent's knowledge entirely, so every related question comes back wrong
- A tool misconfigured the same way every time, like a calculator set to radians instead of degrees
- A phrase or habit every agent in an organization keeps overusing
- Problems buried in tool calls and metadata that never show up in the visible back-and-forth
Why the sleep comparison holds up
During slow-wave sleep, the brain replays the day and moves information from short-term to long-term storage, a process called memory consolidation. REM sleep does something different: it links new memories to older ones and draws connections across experiences that happened far apart. Dreaming's consolidation step, where session logs turn into structured memory, tracks with slow-wave sleep. Its pattern-finding step, where it draws connections across many sessions, tracks with REM. Anthropic treats this as a functional parallel rather than just a name.
How dreaming compares to RAG and reinforcement learning
| Concept | What it does | Limit |
|---|---|---|
| RAG | Pulls in outside documents to answer a question | Adds outside information, not learned behavior from the agent's own past |
| Dreaming | Reviews the agent's own past sessions and turns them into cleaner, more useful memory | Leaves the model itself untouched, only the memory layer changes |
| RL experience replay | Also replays past experience offline | Runs on reward signals and gradient updates; dreaming works through reasoning in plain language instead |
What you can steer
- You can give the pipeline written instructions (up to 4,096 characters through Anthropic's API) telling it what to focus on or skip, like "focus on coding-style preferences, skip one-off debugging notes"
- Those instructions shape the whole pass, not individual lines. If you want to fix one specific memory, you edit the output store directly through the memory API
- You choose how many sessions feed a given run and which model handles it, which changes both cost and how thorough the result is
- You control which session transcripts go into a run, so permissioning on dreaming can match whatever access rules already apply to the agents themselves
Running mechanics to plan around
| Aspect | Detail |
|---|---|
| Timing | Runs asynchronously, takes anywhere from minutes to tens of minutes depending on how much you feed it |
| States | Moves from pending to running to completed, failed, or canceled; a failed or canceled run still leaves a partial output you can inspect |
| Visibility | You can watch the transcript of a dreaming run while it's in progress |
| Safety | You can't delete the output store mid-run, and if an input store or session disappears mid-run, the job fails cleanly instead of leaving a half-written mess |
| Cost | Billed at standard token rates, roughly proportional to how many sessions and how long they are |
Why the extra token spend pays off
Running dreaming costs tokens, so it's fair to ask whether it's worth it. Anthropic's production numbers say yes: agents with well-maintained memory answer repeat tasks more accurately, which lowers both latency and cost because the agent stops needing retries to get things right. The tokens spent on dreaming come back through fewer wasted tokens elsewhere.
Where this pays off most
Workflows that repeat often benefit the most: support tickets, research tasks, sales outreach, content pipelines, ops automation. Anything that happens once or varies wildly each time gives dreaming little to work with, since there's no repeating pattern to find in the first place. This isn't limited to coding either; the same loop works for something as simple as an agent learning your preferred slide format after enough repetition.
What to watch out for
- Bad sessions produce bad memory: short, vague, or messy sessions leave the pipeline with nothing solid to work from
- What comes out is an inference, not a fact: the pipeline can generalize too far from a handful of examples, so someone needs to check the output before trusting it
- Context window limits still exist after cleanup: a big memory store needs good retrieval, not a bigger dump loaded every time
- Session transcripts can hold sensitive information, so retention rules, access controls, and anonymization need deciding upfront rather than fixed after the fact
Bottom line: for statups, the choice is between building an approximation of this with scheduled jobs and a hand-rolled memory store, or using Anthropic's Dreams API directly, which already includes the non-destructive output design, steering instructions, and hashing-based concurrency controls that Anthropic's own systems run on.
Frequently asked questions
What's the difference between in-band and out-of-band agent memory?
In-band memory means an agent reads and writes to memory during a live session, splitting its attention between finishing the task and curating context for its future self. Out-of-band memory, which is what dreaming uses, runs as a separate asynchronous process with dedicated compute, so it can review patterns across many sessions without competing with the agent's actual work.
What's the difference between a CLAUDE.md file, Skills, and a memory store?
A CLAUDE.md file is a single static markdown file loaded at session start, effective for small amounts of context but prone to bloat as it grows. Skills solve that bloat through progressive disclosure, loading only a short description upfront and the full file when relevant. A memory store is a broader, searchable file system of markdown memories that agents query with tools like grep, and it's the input dreaming reads and rewrites.
Does Claude Dreaming work with GPT or Gemini agents, or is it Claude-only?
Dreaming is currently an Anthropic-specific API tied to Claude's Managed Agents infrastructure and requires Claude models (Opus 4.8, Opus 4.7, or Sonnet 4.6) to run the pipeline itself. Teams running mixed-model stacks with GPT or Gemini agents would need to replicate the same batch-review-and-consolidate pattern manually, since there's no equivalent native feature on those platforms yet.
How often should I schedule a dreaming or memory consolidation job?
There's no fixed answer, since it depends on session volume and how quickly your memory store accumulates duplicates or contradictions. High-frequency workflows like customer support might benefit from daily or even hourly runs, while lower-volume agents may only need weekly consolidation, and running it too often adds token cost without proportional benefit.
Can dreaming introduce hallucinated or false patterns into an agent's memory?
Yes. Consolidation is inference, not verified fact, and Claude can over-generalize from a small or unrepresentative batch of sessions. This is why the output store is kept separate from the input and why Anthropic recommends human review of proposed changes before promoting an output store into production use.
How do I measure whether dreaming actually improved my agent's performance?
Track task accuracy, retry rate, and token cost per task before and after swapping in a dreamed memory store, since Anthropic's own production data ties memory quality to fewer retries and lower latency. A/B testing the old and new memory stores against the same task set is the most reliable way to confirm a consolidation run helped rather than introduced regressions.
What's the maximum memory store size Claude can process in a dreaming job?
Anthropic enforces an input memory store size limit, and jobs that exceed it fail with an input_memory_store_too_large error. In practice, this means very large, long-running memory stores may need periodic trimming or splitting by domain before they can be fed into a single dreaming run.
Can multiple specialized agents share one memory store, or does each need its own?
Both patterns are supported. Specialized agents (research, content, scheduling) typically maintain their own memory stores tied to their specific function, while an orchestrator agent can read across multiple consolidated stores to catch system-wide patterns, like a data source that consistently causes one sub-agent to fail.
Is Claude Dreaming the same thing as reinforcement learning from human feedback (RLHF)?
No. RLHF updates model weights using reward signals and gradient updates during training, which changes the base model permanently. Dreaming runs after deployment, uses Claude's own language reasoning rather than reward signals, and only updates an external memory layer, leaving the underlying model completely unchanged.
What happens to an agent's memory if I switch it from Claude to a different model?
Since Anthropic's memory stores are structured as portable markdown behind a clean API, the curated content itself, such as preference profiles and decision rules, can in principle be exported and reused elsewhere. However, the dreaming process that generates and refines that memory is Claude-specific, so a model switch means losing the ability to run further consolidation on that store natively.
Do I need a separate memory store for each customer or user, or can I use one shared store?
This depends on your permissioning needs. Anthropic's guidance treats org-wide knowledge as broadly readable but restricts write access, while user- or task-specific context typically lives in narrower, more isolated stores, similar to giving each agent its own scratchpad while sharing a read-only organizational layer above it.
Keep reading
#195 — Migrating production AI agents from one frontier model to another
Founders assume swapping LLM providers is a config change. It's not. Changing frontier models is a bigger switch than it sounds.
#196 — Fable 5: Model card for founders
Claude Fable 5 is maybe the most proactive model ever released. It doesn't just write code — it goes rogue-proactive to solve problems, even when you didn't ask it to.
#197 — The doorman fallacy for founders
Startups are falling for what is known as the doorman fallacy: reducing rich and complex human roles to a single task and replacing people with AI.