This started by studying other memory systems
Two things sent me back into Orbit’s memory code. The first was OpenClaw’s memory architecture. It separates a small curated core from a much larger searchable history, tracks where memories came from, puts deterministic gates around promotion, and treats memory failure as something that should reduce recall quality rather than block a reply.
The second was Dhravya Shah’s reverse-engineering of Instinct. His reconstruction described a compact user profile and memory one-pager that are available up front, a larger set of linked files the agent can search when needed, and a background process that maintains memory while the answering agent mostly reads it. His evaluation was roughly: explicit recall looked strong, but implicit personalization, multi-step recall, and automatic forgetting still had room to improve.
Orbit already agreed with several of those ideas. Durable memories have evidence, versions, source boundaries, and deterministic admission rules. But reading both systems exposed an imbalance: I had built strong receipts for what entered memory and much weaker evidence for what left memory and reached a model on a particular turn.
That became the question behind this project. Orbit might store the right fact perfectly, but how would I know whether it selected that fact at the moment it mattered?
The wrong memory can win before the model starts
Most memory demos ask whether an assistant can remember something. The harder question is whether it can choose the right memory when years of useful facts are competing for attention.
Before the model answers, Pluto chooses which saved memories to show it. The model never gets one giant dump of a person’s history. It gets a small context for the current turn, with room for at most twelve memories. Inside the code, this selection step is called memory resolution. The live selection rule is labeled assistant-turn/1.
That first rule gave extra weight to memories that are often useful, such as identity, constraints, and preferences. The idea was reasonable. The problem appeared when a person accumulated enough unrelated preferences to fill the available slots. A memory that directly matched the current request could be pushed out before the model ever saw it.
Nothing crashed, and none of the selected memories were fake. The answer could still sound coherent. That is what made the problem so hard to notice: the failure happened before inference, inside a clean-looking list of valid context.
One test failed exactly right
I ran a fun experiment. A synthetic memory set contained one fact about an alpha project, one identity memory, three communication constraints, and eight old preferences that had nothing to do with the project. Then I asked about alpha.
The rule currently in production used all twelve slots on the identity, constraints, and unrelated preferences. It dropped the only project memory. My experimental rule, assistant-turn/2, kept the identity and constraints, included the project fact, and excluded the eight distractions.
request: alpha project
assistant-turn/1
selected: identity + 3 constraints + 8 unrelated preferences
relevant project fact: missing
assistant-turn/2
selected: identity + 3 constraints + relevant project fact
unrelated preferences: excludedWhat changed in v2?
Version two changes how the twelve slots are divided. It first reserves room for up to eight memories that match the current request. It then allows at most four unrelated but generally important memories. If that second group does not use all four slots, more relevant memories can take the remaining room.
Within the unrelated group, identity comes before constraints, and constraints come before preferences. The existing checks for expired memories, conflicting facts, and deterministic ordering stay in place.
The goal is not to make every context smaller. The alpha test selected five memories because only five deserved to be there. The goal is to make sure a label like always-on cannot consume the whole budget before the current request gets a vote.
So I gave version two a shadow lane
One synthetic example can prove that a candidate fixes one known failure. It can't prove that changing production memory is safe. I needed a way to see what version two would do across real turns without letting it answer any of them.
For a sampled turn, Pluto still chooses the real context with assistant-turn/1 first. That result is fixed before version two runs. The candidate receives the same already-loaded request and memory options, creates its alternative selection, and is immediately discarded after the two results are compared.
Version two never reaches the model, changes memory, or sends a reply. It does not trigger another database read or model call. Two policies enter, but only the production policy can influence the answer.
The lane is intentionally small. A low-volume beta can use a five percent sample, and the code will not allow more than ten percent. Turns with more than five hundred memory candidates skip the comparison. Three comparisons that each take more than five milliseconds open a five-minute circuit breaker, and an independent switch can stop shadowing entirely.
There is no background queue or growing in-memory history. If the candidate is slow, storage is unavailable, or the comparison code fails, the real turn continues on the production path.
load the turn once
→ choose production memory with assistant-turn/1
→ freeze that context
├─→ model → real reply
└─→ sampled assistant-turn/2 → compare → discardI measure the difference, not the person
The most useful debugging record would also be the most invasive: a person’s message beside every memory each policy selected. Orbit deliberately does not store that view.
A comparison keeps only measurements such as how many memories each policy selected, how much the selections overlapped, whether one added or removed context, whether conflicts changed, and how long version two took. It does not keep the person, turn, message, reply, memory IDs or values, evidence, sources, prompts, or model output. Raw rows expire after thirty days, and longer-lived rollups contain only aggregates.
Together, those two kinds of evidence answer different questions. The repeatable test suite asks whether version two handles known cases, including recall, contradictions, expiry, injection, isolation, context limits, and the alpha pollution example. Shadow comparisons show how often version two behaves differently under the real shape of production and what that comparison costs.
Neither view is an answer-quality oracle. High overlap can mean both policies made the same mistake, and a smaller context can be cleaner or simply incomplete. The missing personal content is a privacy boundary, but it also means the aggregates cannot tell me whether one specific reply was good.
Where this goes next
The immediate next step is intentionally boring: collect enough shadow comparisons to see whether version two keeps its safety behavior, improves selection consistently, and stays cheap.
After that, the test suite needs harder memory problems. Temporal questions, multi-step recall across conversations, subtle contradictions, changing preferences, and cases where the correct behavior is to admit uncertainty should all become repeatable fixtures.
The architecture can grow in layers too. A very small stable profile can carry identity and durable constraints. The claim ledger can supply memories relevant to the current request. A deeper on-demand path can handle explicit questions about older or multi-step history. That direction borrows the useful separation visible in OpenClaw and Instinct without copying their storage models wholesale.
Version three can then become more thoughtful about recency, conflicts, and different kinds of relevance instead of relying only on a fixed allocation. Whatever changes, the constraints stay: external content cannot promote itself into owner memory, experimental work cannot control a reply, and better recall cannot require dumping a person’s history into another telemetry system.
That is the part I am most excited about. I can now move from a vague bad feeling, to a repeatable example, to a candidate, to bounded real-world evidence, and finally to an explicit promotion or rollback. Orbit’s memory already had receipts for what it stored. Now it has a way to keep improving what it selects.