Summary Memory

What is summary memory?

An AI's context window is small. Conversations get long and early turns get pushed out. Summary memory takes a direct approach: every so often, you ask the model to compress "what we've talked about" into a short summary and keep only that going forward, instead of hanging onto every original word.

How is it different from keeping the full log?

Full log: nothing lost, tokens burned
Storing every turn verbatim is the most faithful, but the window fills up fast and the bill climbs.
Summary memory: keep the point, save space
You hold on to conclusions, decisions and key facts — a few sentences replace hundreds of words, freeing space for more important context.

How are summaries usually made?

Let the model write them
The common move is to hand the LLM a chunk of dialogue and have it output a structured summary — something like "user goal / completed steps / to-dos".
Roll up in layers
Summaries can be summarized again. When you have too many, merge them into a higher-level summary, compressing layer by layer like a pyramid.

The trade-offs

Upside: cheap tokens, longer memory
You keep the thread of a conversation for very little cost, so multi-turn chats don't turn amnesiac.
Downside: details get lost
Compression is lossy. A small preference the user mentioned in passing can get swallowed. Keep the crucial details stored separately.

Bottom line: summary memory trades detail for duration — a short summary that lets the AI remember a long conversation.