Skip to content
Menu

Can a memory foresee what will break it?

This inquiry tests whether the failure of a stored memory can be forecast before it happens, and whether a fixed protection budget is better spent on that forecast than on a guess made from the same information.

Designed

Given a stored memory and descriptions of two possible later disruptions, can a trained model predict which one will leave the memory unable to support its rule, more accurately than a model given the same information and no training on outcomes beyond a single fitted temperature?

At a glance
Method
Copy one stored memory, name two later interference classes, have a trained predictor and an equal-information model each forecast whether the memory survives, then run both branches and score the forecasts with the Brier score on 32 held-out families.
Stop rule
  • Stop at the corpus gate with fewer than 64 verified, sealed families or too little divergence between branches.
  • At the held-out comparison, pass only if the interval's lower end clears 0.03 Brier, stop if its upper end falls below 0.03, and report anything else as inconclusive.
  • After a pass, stage two passes only if the budgeted write controller cuts later rule-application error by at least 5 points.
Where collaboration helps
A second author for rule-and-exception families, accelerator time and inference credit for one small open-weight model, critique of the design before it is pre-registered, and a collaborator who knows calibration and scoring rules.
Generated cyanotype print of a small stone cairn on a flat beach with a wave line advancing towards it.Generated photograph of a cairn of white plaster blocks on a plaster shore with terraced plaster sea rising toward it; one low block carries a vermilion bead.Generated render of a sandbar on the sea floor with two currents of points approaching it from either side.
Plate IA cairn above the waterline, with the tide coming in. Which change will take it?The sea rises step by step. Which block goes first, and could the cairn have known?A sandbar and two currents. Which one will take it?Generated image in a cyanotype treatment. Far Akin art direction.Generated image. Far Akin art direction, Datum look.Generated image. Far Akin art direction, Swath look.

Why this question

A stored memory is used later, after other things have happened to it. In a language-model agent, later events can overwrite or blur what was stored, and the failure is usually found when the memory is needed. The practical question is whether it can be seen coming.

The question has an older form in psychology. In one study of people, judgements made at the time of study did not reflect the length of the delay before the test, unless participants were asked to estimate recall at several delays or were asked about forgetting directly. Participants could use what they knew about forgetting, but only when prompted to make theory-based predictions[1]. A system that forecasts its own failure would have to predict from a description of what is coming, not from the present state of the memory alone.

The human result is background, not a comparison: the inquiry tests the machine side only.

What is already known

Catastrophic interference, where learning new material degrades what was stored, was described for connectionist networks in 1989[2] and remains a central problem in continual learning. Later work asks which items are most at risk. One study finds that examples a network learns quickly are less likely to be forgotten than those learned slowly, and that the choice between them changes how well a replay buffer protects against forgetting[3]. That is a forecast of vulnerability between items, from a signal available before the damage.

For language agents, one recent study frames memory as a budgeted choice between retaining raw records and consolidating them with one of three operators, using a lightweight learner on pre-generation features. The best choice depends on the budget[4]. Another asks a model to commit to a plan under a token budget before any feedback, scores it against an oracle, and reports substantial gaps[5]. A third line asks models to estimate whether their own answers are correct. Larger models are well calibrated on multiple-choice and true/false questions when these are formatted properly, and a trained estimate of whether the model knows an answer generalises only partly to new tasks[6].

In human memory, judgements of learning are insensitive to the delay until test unless forgetting is made salient[1].

Each of these treats a part of the question. The design here has a different unit: one stored state, copied, with a forecast made for each of two named disruptions before either runs.

Definitions

Rule-and-exception family
One independently authored rule with its exception in a synthetic domain, with an executable rule engine that supplies exact labels for whether the rule is applied correctly.
Interference class
A described kind of later disruption to a stored memory. Two will be drawn for each family by a fixed generator, and both will be applied to identical copies of the same memory.
Equal-information model
The main comparator: the same model given the same memory and the same interference descriptions, with no parameters trained on outcomes except a single temperature fitted on the 64 sealed families, so that a gain cannot come from rescaling alone.
Brier score
The mean squared difference between a stated probability and what happened, scored within each family; lower is better.
Divergence check
A count, with a threshold to be fixed in the pre-registration, of the families in which the two interference classes lead to different outcomes. Without divergence there is nothing to forecast.
Leak check
A test of whether the interference description alone gives the answer away, read from how well the equal-information model, with no training on outcomes beyond its temperature, already forecasts.

The approach

The test uses 96 rule-and-exception families, each an independently authored rule with its exception in a synthetic domain. An executable rule engine supplies exact labels. For each family, one memory state is built and copied, and two classes of later interference are named for the copy. Neither has run.

One small open-weight language model sees the stored memory and a description of each interference class. It states the probability that the memory will still support the correct application of the rule and its exception after each class. The interference then runs, and the probabilities are scored against the rule engine's outcome with the Brier score, the mean squared difference between a stated probability and what happened, where lower is better[7].

The main comparison is an equal-information model: the same model given the same memory and the same interference descriptions, with no parameters trained on outcomes except a single temperature fitted on the 64 sealed families, so that a gain cannot come from rescaling alone. A trained risk predictor is compared against it. Item difficulty and store usage logs serve as baselines only between families, because within a family the memory is identical in both branches.

Two choices are fixed in writing before the build: whether the probability is read before or after the interference class is announced, and a fixed generator that draws the two classes for each family, so that the author of a family does not also choose its comparison. At the corpus gate, 64 families are verified and sealed. The other 32 are verified by the same procedure and then sealed, unseen by the predictor, until the held-out comparison. Intervals are paired and clustered by family.

Stage two follows only if the forecast passes on Brier score. In it, a budgeted write controller will spend a fixed byte budget on the write each forecast favours: keep the clause verbatim, bind the exception in structured form, or add nothing. Later rule-application error will then be compared with the same spend guided by the equal-information model's forecasts.

What would count as an answer

These conditions are written before any data exist.

  • Stop at the corpus gate. Fewer than 64 verified, sealed families.
  • Stop at the corpus gate. The pre-registered divergence check shows that the two interference classes give the same outcome in more families than a share to be fixed in the pre-registration. There is then nothing to forecast.
  • Pass at the held-out comparison. On the 32 held-out families, the trained predictor beats the equal-information model by at least 0.03 in Brier score, with the lower end of the paired 95% interval clearing that target.
  • Stop at the held-out comparison. The upper end of the 95% interval falls below 0.03 in Brier score.
  • Inconclusive. Any result that is neither a pass nor a stop, such as an interval that excludes no effect but whose lower end falls short of the target. It will be reported as inconclusive, not as a pass.
  • Stage two, after a pass. The budgeted write controller is built. Stage two passes if spending the byte budget on the predictor's forecasts lowers later rule-application error by at least 5 percentage points against the same spend guided by the equal-information model, with the lower end of the paired 95% interval clearing that target. It stops if the upper end falls below 5 points; anything else is reported as inconclusive.

One further check comes first. If the equal-information model, with no training on outcomes beyond its temperature, already forecasts better than a Brier score to be fixed in the pre-registration, the interference description leaks the answer, the comparison means nothing, and the result is not counted.

What else could explain a success

An apparent gain over the equal-information model has four other candidate explanations. The first is leakage: the description of an interference class may carry information about the outcome that a forecaster should not have[8]. The leak check, and a later run on descriptions stripped of outcome-bearing content, are designed to address it. The second is authorship. If one person writes every family, that person could, without intending it, choose pairs that suit the predictor. The classes will be drawn by a fixed generator, the 32 held-out families will stay unseen by the predictor until the comparison, and the lab is seeking a second author for part of the corpus. The third is calibration alone. Modern networks are often poorly calibrated, and a single fitted temperature corrects much of that[9], so training on outcomes could lower the Brier score by rescaling probabilities without learning anything about the memory. The equal-information model's single fitted temperature separates the two: a gain over it cannot come from rescaling alone. The fourth is that the predictor learns which interference class tends to do more damage and ignores the memory. A trained predictor given the class descriptions and no memory will test this: if it matches the full predictor, the forecast concerns disruptions, not the memory. Item difficulty is no rival within a family, because both branches share one memory, and intervals will be clustered by family so that one family's quirks cannot carry the result.

What it would not show

A pass would show that, in one synthetic rule domain and with one small model, a trained predictor forecasts interference outcomes better than a model given the same information. It would not show that the forecast transfers to real agent memories, other kinds of content or other models. The gates test feasibility and identification, not an effect in a wider population.

The design makes no claim about human memory, and no person is tested. Whether a model's forecast resembles a person's judgement of learning stays a question.

A pass could also be too easy to obtain. If the families and the test have the same author, the size of a gain could be shaped by how the interference classes are chosen. The fixed generator and the leak check are designed to reduce that risk, not remove it. Whether an edit to a memory reaches the facts that depend on it is a separate question, taken up in when one fact changes.

Next step

Fix the pre-registration in writing: when the probability is read, how the generator draws the two interference classes, the divergence check with the share of families at which it stops the inquiry, and the level at which the leak check voids the comparison. Then author 96 rule-and-exception families and seal 64 of them for the corpus gate.

What it would build

References

  1. Koriat, A., Bjork, R. A., Sheffer, L. and Bar, S. K. (2004). Predicting one's own forgetting: the role of experience-based and theory-based processes. Journal of Experimental Psychology: General, 133(4), 643–656.
  2. McCloskey, M., Cohen, N. J. (1989). Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem. Psychology of Learning and Motivation 24, 109–165.
  3. Hacohen, G., Tuytelaars, T. (2024). Predicting the Susceptibility of Examples to Catastrophic Forgetting. arXiv preprint 2406.09935.
  4. Kang, Q., Liu, M., Kai, S. et al. (2026). Retain or Consolidate? Budget-Dependent Operator Selection for Language Agent Memory. arXiv preprint 2607.17545.
  5. Al Nazi, Z. and Dipta, S. R. (2026). TRIAGE: Evaluating Prospective Metacognitive Control in LLMs under Resource Constraints. arXiv preprint 2605.13414.
  6. Kadavath, S., Conerly, T., Askell, A., Henighan, T., Drain, D., Perez, E., et al. (2022). Language Models (Mostly) Know What They Know. arXiv preprint 2207.05221.
  7. Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1), 1–3.
  8. Kaufman, S., Rosset, S., Perlich, C. and Stitelman, O. (2012). Leakage in data mining. ACM Transactions on Knowledge Discovery from Data, 6(4), 1–21.
  9. Guo, C., Pleiss, G., Sun, Y. and Weinberger, K. Q. (2017). On Calibration of Modern Neural Networks. Proceedings of ICML 2017, PMLR 70, 1321–1330.

Cite as

Far Akin Research Labs (2026). Can a memory foresee what will break it?. Version 1.0. Far Akin Research Labs. https://farakin.vercel.app/research/can-a-memory-foresee-its-failure

BibTeX
@techreport{farakin2026-can-a-memory-foresee-its-failure,
  title = {{Can a memory foresee what will break it?}},
  author = {Far Akin Research Labs},
  institution = {Far Akin Research Labs},
  type = {Inquiry design},
  year = {2026},
  month = oct,
  note = {Version 1.0},
  url = {https://farakin.vercel.app/research/can-a-memory-foresee-its-failure}
}

Noticed an error or a missing source? Send a correction

Related inquiries

  • Generated cyanotype print of three weighted lines lowered into one dark pool, their rings of ripples overlapping but not quite matching.Generated photograph of three different white probes lowered into one plaster basin, each tip marked by a vermilion bead; the three beads land near each other but not together.Generated night-survey render of three side-by-side strips of mapped sea floor that show the same ridge, nearly aligned.
    Designed

    Can the instruments for a model's workspace be trusted?

    On open-weight language models that already exist, do three tests of an internal “workspace” give repeatable readings (test-retest r of at least 0.8), and do they rank the same models alike or differently?

    Latent space and internal cognition · Cognitive science and psychology · Machine-learning foundations

  • Generated cyanotype print of a fishing net lifted at a single knot, the whole mesh hanging from that point.Generated photograph of a white knotted net on a plaster plinth, one central knot lifted on a vermilion thread, the nearby knots marked with vermilion beads.Generated render of a sea-floor surface where one raised point sends fading rings of change outward.
    Designed

    When one fact changes

    For a weight-based editing method at a stated update budget, does correct updating of dependent facts fall, and do wrong changes to unrelated facts rise, as the number of dependents of one changed fact grows?

    Memory, knowledge and continual learning · Machine-learning foundations

  • Generated cyanotype print of pressed leaves of different ages layered together, the oldest pale and skeletal, the newest dark and whole.Generated photograph of stacked white paper strata, the lower layers pushed aside by newer ones, pierced by a vertical vermilion thread that bends at the displaced layer.Generated render of a cross-section of stacked sea-floor sediment layers, with one folded and broken section.
    Designed

    What a memory should keep apart

    At matched model, token and latency budgets, does an agent memory that stores claims, facts, beliefs, contradictions, supersession and uncertain freshness as separate states produce more correct later answers and actions on published benchmarks of changing information than full context, raw-event retrieval or typed records alone?

    Memory, knowledge and continual learning · Measurement and research methodology