What a memory should keep apart

Far Akin Research Labs

Why this question

A language-model agent that works with someone over weeks or months gathers statements of very different standing. A deadline moves. A colleague's opinion is recorded beside a measured result. A preference stated in spring is reversed in autumn without anyone saying the earlier one no longer holds. A store that keeps all of this as retrievable text will often return the old and the new versions together, and the model has to sort them out at the moment of use.

The hypothesis is that agent memory should be built as a record of epistemic states rather than a store of text. Each item carries its source, time, scope and standing, and retrieval uses those fields to decide what may guide action now. The aim is an agent that does not act on an outdated fact, does not present one person's belief as a fact, and does not quietly merge two sources that disagree.

The opposite position is strong. Long context windows and careful retrieval over verbatim records do well on several memory benchmarks, and every added layer brings write cost, latency and new ways to be wrong. The question is which structure, if any, pays for itself, and on which kinds of failure.

What is already known

The rungs will be compared on published benchmarks. LoCoMo builds conversations of about 300 turns over up to 35 sessions, and finds that long-context models and retrieval help but still fall well short of human performance on long-range temporal and causal questions^[1]. LongMemEval tests five abilities, including knowledge updates and abstention, and reports a 30% drop in accuracy for commercial assistants and long-context models over sustained interactions^[2]. Harder benchmarks show where structure might matter. BeliefShift finds that models which personalise aggressively resist drift poorly, while more grounded models miss legitimate updates^[3]. STALE tests memories made invalid by a later observation without explicit negation; the best evaluated model reaches 55.2% overall, and models often accept a question that presupposes an outdated state^[4]. Agents near saturation on LoCoMo perform poorly in MemoryArena, where memory must guide later actions^[5], and Mem2ActBench finds current systems weak at using memory to choose tool parameters^[6]. StructMemEval tests memory organised into ledgers, lists and trees^[7], LongMemEval-V2 tests dynamic state tracking for web agents^[8], and BEAM tests dialogues beyond a million tokens^[9].

Typed records, temporal validity and conflict handling are not new as components. Zep's Graphiti engine is a temporally aware knowledge graph that keeps historical relationships as information changes^[10], APEX-MEM resolves conflicting or evolving information at query time^[11], and Memanto combines thirteen typed memory categories with conflict resolution and temporal versioning, and reports a five-stage ablation of its own components^[12]. Thin baselines are strong. A retrieval-centred system over verbatim events, with no graph store, argues that extraction at write time discards content a later query needs, and reports high accuracy on LoCoMo, LongMemEval and BEAM^[13]. A comparison of fifteen memory methods finds long-context baselines highly competitive and no single form of memory best across settings^[14]. A cost study finds long context more accurate on factual recall on two benchmarks, while a fact memory becomes cheaper after about ten turns at 100,000 tokens^[15]. What stays unsettled is which of these layers earns its cost, one at a time, against the simpler memory below it under matched budgets.

The problem is older than language models. A truth maintenance system records the reasons for a program's beliefs so that they can be revised when a discovery contradicts an assumption^[16], and the AGM postulates describe how a theory should be contracted to drop a proposition or revised to admit one^[17].

Definitions

Event: A bounded occurrence in time, kept as an unchanged raw trace. Events are evidence for memory, not memory summaries.
Claim: An assertion with a source, time, scope and status. It guides action only after its status and context have been checked.
Fact: A claim currently treated as strongly supported within a stated scope. It keeps its provenance and validity range, and may be used directly unless contradicted, stale or out of scope.
Belief: A claim held by a named person or agent, not treated as true because it is held. It is stored with its holder, source and revision history, may shape interaction, and does not override stronger evidence.
Contradiction: Two claims or states that cannot both guide present action in the same scope under the same interpretation. Both are kept and the conflict is recorded; the system resolves it by rule, hedges or asks, and never merges the two silently.
Supersession: A later record replaces an earlier one for current use without erasing it. The earlier record stays traceable but inactive, and retrieval prefers the newer one.
Outdated: Once valid, no longer expected to guide present action. Kept with a closed validity interval and used only when history is asked for.
Stale: Possibly still true, but confidence that it is current has decayed. Marked for refresh or confirmation; the system hedges, checks or avoids relying on it.
Valid in context: Applies to the current task, domain, user and policy scope. Records outside the scope are filtered out or ranked lower.
Provenance: The chain showing where a record came from and how it was transformed: source, write path and revisions. Fuller provenance makes a record easier to inspect and correct.
Revision: Any change of state after the first write, such as a correction, a narrowing of scope, a supersession or a contradiction. Revisions are non-destructive where possible, and retrieval prefers the latest valid reading.

The approach

The inquiry is at the design stage. The architecture, the definitions of memory states and the ladder below are drafted; nothing has been built or run, and no pre-registration exists yet.

The ladder. Seven memories will be compared, each adding one layer to the one below.

Rung 0, full context. No external memory; the whole history goes in the prompt.
Rung 1, raw-event retrieval. Verbatim events are stored and retrieved.
Rung 2, typed records. Items gain a type, source, timestamp and scope.
Rung 3, temporal validity. Records gain valid-from and valid-to times and states such as stale, outdated and superseded.
Rung 4, contradiction and supersession. Explicit conflict records and revision rules.
Rung 5, relations. Explicit links among entities, events, claims, contexts and goals.
Rung 6, a light operational ontology. A small set of semantic primitives with scoped meaning, including the separation of belief from fact.

The planned architecture has a few required parts: a raw event store that keeps every source trace unchanged, a typed record store, retrieval that applies validity and revision filters, a brief builder that turns retrieved items into a short operating summary rather than a dump, a provenance layer, and an interface for inspecting and correcting memory. Graph storage and the ontology will stay optional until the ladder shows they help. A domain-specific layer is outside this inquiry.

Matching. Neighbouring rungs will run with the same backbone model, token budget, retrieval latency budget, write-path controls and benchmark slice. One extraction model will write records for every rung that has them.

Benchmarks by rung. Rungs 0 to 2 will be compared on LoCoMo and LongMemEval. Rungs 3 and 4 will be compared on BeliefShift and STALE, which test belief revision and stale memories, and on the dynamic-state-tracking and premise-awareness questions of LongMemEval-V2, a benchmark of environment memory for web agents. Rungs 4 to 6 will be compared on Mem2ActBench, StructMemEval and MemoryArena, which test memory used in action and memory that must be organised. BEAM will serve as a stress test, to check that a gain survives at greater length. BeliefShift and Mem2ActBench will be used only if their data is available under terms that permit the comparison; otherwise those rungs will rest on the remaining benchmarks named for them.

Predictions, stated here before any run. Rung 2 is needed. Rung 3 is very likely needed. Rung 4 is probably needed. Rung 5 is uncertain. Rung 6 is the open dispute. The design is meant to let each prediction fail.

Measures. Outcome quality will be measured on each benchmark's own measure and reported beside cost: tokens per turn, retrieval latency and write cost. Where possible each failure will be assigned to the write, retrieval or use stage, so that a gain can be traced to the layer that produced it. For each retrieved item, each run will also record whether the system could say what it is, where it came from, and whether it is current, in scope and uncontested.

What would count as an answer

These rules will be fixed in a pre-registration before any run. The design sets the form of each rule; the pre-registration will set the numbers (the minimum gain, the cost and latency ceiling, and the interval method) before any benchmark output is read.

Check first. If rung 0, full context, already reaches the ceiling on a slice, that slice cannot separate the rungs and will be dropped before any comparison.
A layer passes only if it beats the rung below it on at least one hard benchmark slice by the pre-registered margin, stays within the pre-registered cost and latency ceiling, and has a mechanism that explains the gain, shown by which failures it removed.
A layer fails if its gain is below the margin, if the gain disappears once results are normalised for cost, or if better retrieval and prompting at the rung below reproduce it.
Inconclusive. If the interval for a layer's gain includes both no gain and the margin, the layer will be reported as inconclusive, not as a pass, and will stay optional.
Stop. If rungs 3 and 4 both fail against rung 2 on the revision benchmarks, the hypothesis that these memory states need explicit representation will be set aside for the kinds of change the benchmarks test and reported as a negative result.

A failed layer will be removed, made optional or moved offline, and the next rung will be compared against the last rung that passed.

What else could explain a success

A richer rung could look better for reasons that have nothing to do with the states it adds.

More text in the prompt. A richer brief may simply carry more evidence. Every pair of rungs will run under the same token budget.
Better prompting, not better structure. The rung below will receive the same brief format and a tuned retrieval and prompt; a layer whose gain this reproduces fails.
A better writer. A stronger extraction step would make records cleaner for that reason alone. The same extraction model and write-path controls will apply to every rung.
Spending more. A layer may win only by using more calls or time. Results will be normalised for cost, and a gain that disappears under normalisation will not count.
The judge or the backbone. A language-model judge can favour a style of answer, and accuracy varies by backbone. The judge will be held fixed, exact scoring will be used where a benchmark allows it, and the backbone will be named in every result.
A slice that cannot separate rungs. If full context already solves a slice, differences among richer rungs are noise. Such slices will be dropped first.
Right labels by luck. A layer may help because its states happen to be assigned correctly on easy items. Where a benchmark supplies the true state of an item, errors in assigning states will be counted apart from errors in using them.

What it would not show

A pass at one rung would show that, on the named published benchmarks and the backbones tested, one layer of structure improved later answers or actions over the simpler memory below it at matched cost. It would not show that the architecture is a general memory, that it carries over to other domains, or that it helps on tasks the benchmarks leave out, such as user corrections over months, deletion and governance, or poisoned writes.

Published benchmarks may sit in the training data of the backbone models, and several are recent preprints that may change. Some of them score answers with a language model as judge. A survey of agent-memory evaluation reports that scores are sensitive to the judge and that accuracy depends on the backbone model^[18], so each result will be reported with its backbone and judge.

A negative result is informative but narrow. If typed records with timestamps match the richer rungs, the extra states were not needed on these benchmarks; that does not show they never matter.

The inquiry concerns memory held outside a model's weights. Whether an edit to the weights reaches the facts that depend on it is taken up in when one fact changes, and whether a memory's failure can be forecast in a separate inquiry. This inquiry does not test human memory.

Next step

Write the pre-registration: the benchmark slices for each pair of rungs, the minimum gain, the cost and latency ceiling and the interval method. Then build rungs 0 to 2 on one backbone and run the ceiling check, to confirm the chosen slices can separate the rungs before any richer layer is built.

What it would build

Revisable memory runtime

References

Maharana, A., Lee, D.-H., Tulyakov, S., Bansal, M., Barbieri, F., Fang, Y. (2024). Evaluating Very Long-Term Conversational Memory of LLM Agents. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 13851–13870.
Wu, D., Wang, H., Yu, W., Zhang, Y., Chang, K.-W., Yu, D. (2025). LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory. International Conference on Learning Representations (ICLR 2025).
Myakala, P. K., Agrawal, M., Manche, R. (2026). BeliefShift: Benchmarking Temporal Belief Consistency and Opinion Drift in LLM Agents. arXiv preprint 2603.23848.
Chao, H., Bai, Y., Sheng, R., Li, T., Sun, Y. (2026). STALE: Can LLM Agents Know When Their Memories Are No Longer Valid? arXiv preprint 2605.06527.
He, Z., Wang, Y., Zhi, C., Hu, Y., Chen, T.-P., Yin, L. et al. (2026). MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks. arXiv preprint 2602.16313 (ICML 2026).
Shen, Y., Li, K., Zhou, W., Hu, S. (2026). Mem2ActBench: A Benchmark for Evaluating Long-Term Memory Utilization in Task-Oriented Autonomous Agents. arXiv preprint 2601.19935.
Shutova, A., Olenina, A., Vinogradov, I., Sinitsin, A. (2026). Evaluating Memory Structure in LLM Agents. arXiv preprint 2602.11243. Introduces the StructMemEval benchmark.
Wu, D., Ji, Z., Kawatkar, A., Kwan, B., Gu, J.-C., Peng, N. et al. (2026). LongMemEval-V2: Evaluating Long-Term Agent Memory Toward Experienced Colleagues. arXiv preprint 2605.12493.
Tavakoli, M., Salemi, A., Ye, C., Abdalla, M., Zamani, H., Mitchell, J. R. (2025). Beyond a Million Tokens: Benchmarking and Enhancing Long-Term Memory in LLMs. arXiv preprint 2510.27246. Introduces the BEAM benchmark.
Rasmussen, P., Paliychuk, P., Beauvais, T., Ryan, J., Chalef, D. (2025). Zep: A Temporal Knowledge Graph Architecture for Agent Memory. arXiv preprint 2501.13956.
Banerjee, P., Moshtaghi, M., Subramanian, S., Misra, A., Chadha, A. (2026). APEX-MEM: Agentic Semi-Structured Memory with Temporal Reasoning for Long-Term Conversational AI. arXiv preprint 2604.14362 (accepted to ACL 2026).
Abtahi, S. M., Rahnema, R., Patel, H., Patel, N., Fekri, M., Khani, T. (2026). Memanto: Typed Semantic Memory with Information-Theoretic Retrieval for Long-Horizon Agents. arXiv preprint 2604.22085.
Adler, J., Zehavi, G. (2026). Storage Is Not Memory: A Retrieval-Centered Architecture for Agent Recall. arXiv technical report 2605.04897.
Wang, Y., Zhang, Z., Chi, M., Yu, K., Li, Y., Peng, M. et al. (2026). EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective. arXiv preprint 2605.18421.
Pollertlam, N., Kornsuwannawit, W. (2026). Beyond the Context Window: A Cost-Performance Analysis of Fact-Based Memory vs. Long-Context LLMs for Persistent Agents. arXiv preprint 2603.04814.
Doyle, J. (1979). A truth maintenance system. Artificial Intelligence, 12(3), 231–272.
Alchourrón, C. E., Gärdenfors, P., Makinson, D. (1985). On the logic of theory change: Partial meet contraction and revision functions. Journal of Symbolic Logic, 50(2), 510–530.
Jiang, D., Li, Y., Wei, S., Yang, J., Kishore, A., Zhao, A. et al. (2026). Anatomy of Agentic Memory: Taxonomy and Empirical Analysis of Evaluation and System Limitations. arXiv preprint 2602.19320.

Cite as

Far Akin Research Labs (2026). What a memory should keep apart. Version 1.0. Far Akin Research Labs. https://farakin.vercel.app/research/what-a-memory-should-keep-apart

BibTeX

@techreport{farakin2026-what-a-memory-should-keep-apart,
  title = {{What a memory should keep apart}},
  author = {Far Akin Research Labs},
  institution = {Far Akin Research Labs},
  type = {Inquiry design},
  year = {2026},
  month = oct,
  note = {Version 1.0},
  url = {https://farakin.vercel.app/research/what-a-memory-should-keep-apart}
}

Noticed an error or a missing source? Send a correction