Three Kinds of Memory Problem (and Why Games Keep Them Straight)
March 12, 2026
The agent-memory industry has a naming problem. Everything from a vector store to a skill library to a session log gets filed under one word: memory. That word then carries the weight of at least three structurally different problems, and systems built for one quietly fail the others.
Game research figured this out first, for a simple reason: games cannot fudge the distinction. A world that changes under the agent forces the question "what is true now?" every single tick. An unknown rulebook forces "what did interaction teach me?" There is no human in the loop to smooth over a stale fact or an unrevised prior. The environment just fails you.
Surveying the current landscape of AI game environments and benchmarks, three clusters emerge. They map cleanly onto three different memory problems.
1. What happened?
The episodic problem: store observations, retrieve them later when relevant.
FlashAdventure — thirty-four adventure games bundled as a benchmark — exists precisely because agents fail here in an unexpected way. Its authors call it the observation-behavior gap: agents observe the clue, then fail to retain or use it when a later scene needs it. The information entered the context window. It just never became behavior. MineNPC pushes the same axis further, instrumenting not just what agents remember but their actual memory reads and writes mid-task.
This is the problem most commercial memory products actually solve. Session logs, embeddings, retrieval-augmented anything. It is necessary and well-served. It is also the easiest of the three.
2. What is true about this world right now?
The state problem: maintain a continuously revised model of resources, dependencies, unfinished plans, threats, and what remains possible.
The Factorio Learning Environment makes this vivid through consequences. The agent's own actions reshape the world: build a bad mining architecture early and power fails hours later, silently invalidating a plan formed four hundred actions ago. Nothing was forgotten — every event is retrievable. But the current implications of those events must be tracked forward, or the agent operates on a world that no longer exists. Civilization-scale benchmarks push the same pressure across hundreds of turns with delayed consequences.
This is the problem I've argued elsewhere deserves to be called state, not memory — the question is never "what happened" but "what does the evidence currently permit?" It is also where most enterprise agent deployments quietly bleed: the archive grows while nobody maintains which parts of it still hold.
3. What did experience teach me about how worlds work?
The learning problem: convert accumulated episodes into hypotheses, skills, causal knowledge, and meta-knowledge that alter future behavior.
Here the game research gets genuinely strange — and genuinely ahead of product land.
MirrorCraft looks like Minecraft but secretly isn't: server-side recipes, drops, and mechanics have been changed. Pretrained knowledge now actively lies. The benchmark measures whether lived experience can override the model's priors — whether an agent that watches its crafting output differ from expectation will update, or keep confidently following its training data off a cliff.
SciCrafter decomposes miniature science: identify a knowledge gap about redstone mechanics, design experiments, consolidate findings, then engineer circuits from what was discovered. Frontier agents complete roughly a quarter of it. The bottleneck is not retrieval or state tracking — it's converting raw episodes into reusable causal knowledge.
ARC-AGI-3 strips away even the familiar skin: no instructions, no named mechanics. Observe, act, watch the consequence, infer a rule, test it, revise. The entire task is experience becoming knowledge. And older work like Voyager pointed the way years ago with its persistent executable skill library — memory whose write format is code that runs, not text that sits.
Why the split matters
Because each kind fails differently, and you diagnose by matching symptom to kind.
An agent that saw the deadline but didn't act on it: kind-one failure — observation never became retrievable intention. An agent executing a plan against yesterday's infrastructure: kind-two failure — the archive is fine, the state model is stale. An agent repeating an approach that already failed twice last sprint: kind-three failure — episodes never consolidated into policy.
Build retrieval harder and you fix exactly none of failures two and three. This is why long-context windows keep underperforming their promise: more episode storage does not produce state maintenance or consolidation. It produces a bigger pile of kind one.
It also suggests what to measure. One-shot scores barely capture kind two and miss kind three entirely. The interesting metrics track trajectories — improvement dynamics over repeated exposure, whether game N+1 goes faster than game N, whether a lesson learned in one world transfers to another. If your agents are supposed to be learning your company's work, a flat improvement curve is the loudest alarm there is.
The uncomfortable gap
Kind one has an industry. Kind two has a handful of local-first tools and my ongoing argument. Kind three — the one game research treats as the frontier — has almost no product presence at all. Most deployed agent systems cannot override their own priors with evidence, consolidate failed experiments into rules, or transfer lessons across projects. They re-derive or repeat.
Games make this visible because worlds are small enough to be honest. Work is messier, but the three problems don't change. Your team's agents have all three. Naming which one is failing is half of fixing it.