The programmes need instruments as much as findings. This page lists the instruments and systems the inquiries specify: what each takes in, what it returns, the parts it needs and what it will not show.
Every entry carries one of four states. Specified means the design is written. Being built means code, data or a model is in progress. Tested means the system has been run against the check its inquiry wrote in advance. Released means the code, data or model is public.
Where a system is built and tested, its evaluation will be reported with it: the test it was held to, the result and the limits that apply. A negative result will be reported in the same detail as a positive one. Release of code, data or models is a separate state and is not promised for every entry. Each entry names the inquiry that needs it.
A typed, machine-checkable record of a hand-written fictional world and a generator of test questions from it, so that a world model can be graded on whether it keeps a world's facts as well as on how its output looks. The intended output is a report that states a domain of use with error bounds instead of a single score.
A harness that pushes a frozen robot policy along independent scene directions mid-episode in simulation and measures which directions it opposes, separating what the policy holds still from the scene's own physical response. The aim is a signature that predicts which held-out conditions the policy survives.
A planned set of three tests of an internal workspace in open-weight language models, to be frozen with its reliability rule and interval rules fixed in advance, so that later work can measure a model's workspace with instruments whose repeatability is known.
A test bed for holding a simulated interviewee's fixed traits, permitted learning and revealed answers in separate channels, and for checking whether a question-picking planner recovers hidden attributes without winning by changing the person.
A measure of how much of each published survey instrument a language model can reproduce, and a test of whether a model's simulated answers sit closer to the published results as that measured exposure rises, beyond what each item's average answer already explains.
A bench designed to copy a stored memory, name two possible later disruptions, and ask a model for the probability that the memory will still support its rule after each, before either disruption runs. Exact outcomes from a rule engine score the forecast, so a memory's ability to anticipate its own failure can be measured.
A benchmark to be built from a licensed public record of superseded items, designed to grade knowledge editing by how many facts depend on a changed fact, reporting both whether dependents update and whether unrelated facts change wrongly.
A candidate design for a layered memory system for agents. It is designed to keep raw traces of each event, write typed records with origin, time and scope, mark each record's state as conditions change, and assemble a short brief for the task in hand instead of a dump of retrieved text. Each layer above a plain retrieval baseline stays only if an ablation shows it earns its cost.
A candidate design for a small model, to be trained on domain material labelled with a stated scheme of categories and relations and on judgement scenarios, that decides which rules, patterns and rubrics of a domain apply to a given situation; to be tested against a frontier model given the same material through structured retrieval, deterministic validators and a repair loop, and kept only if it adds value over that baseline.