One Belief Per Fact: Building Agent Memory for Users Who Change Their Minds

Here is a scenario that breaks most agent memory systems.

In session one, a user tells your financial advisory agents she is cautious with money and dislikes individual stocks. The system remembers it. In session two, she has changed her mind: she wants to go aggressive, individual tech stocks. The system remembers that too.

Now session three begins, and an agent asks memory: what is her risk appetite?

If your memory layer stores a row per session, it returns both answers, side by side, with no way to choose. We know because that is exactly what our first version did. This post is about how the semantic tier (L2 in our shorthand) actually works: what gets embedded, how beliefs update, and what happens on a contradiction. We will use one running example throughout, a user named Priya, drawn from the synthetic test scenario we ran the design against.

How we broke it first

The first cut of the semantic tier keyed facts by session. Every session inserted new rows, so after a handful of sessions there were six copies of Priya's risk tolerance, some of them contradicting each other, and no notion of "current." Retrieval surfaced whichever ones ranked highest, which meant a downstream agent could receive "cautious" and "aggressive" in the same context window.

Two other bugs from that first version are worth confessing, because they shaped the fixes. The transaction analysis agent invented income growth it was never told about, which led to a hard grounding rule: only figures explicitly stated in memory, otherwise answer "unknown," never estimate. And our "semantic" search was not semantic at all: the store had been created without an embedding index, so nothing was ever vectorized, and we were doing keyword matching while believing otherwise. The fix included a runtime check that the vector table is populated and that a paraphrased query actually retrieves the right belief.

The structural fix for the main bug is the subject of this post: one belief per fact, current, with the versioned history to prove it. That phrase is the whole design; everything below is what it takes to make it true.

What actually gets embedded

There are two embedding moments, and keeping them separate clears up most confusion about how retrieval works.

The write side. After a session, an asynchronous projector reads the raw episodic log and asks an LLM to break each entry into typed, self-contained fact sentences. From Priya's intake record it extracts something like:

Only one field is embedded: the text sentence. The store is configured to embed that field alone. The sentence goes to an embedding model and the resulting vector is stored in a separate vector table. Everything else, the key, the version, the flags, the provenance, is plain metadata for identity, filtering, and audit. It is never embedded.

The read side. When an agent asks "what is her appetite for investment risk?", the query sentence is embedded with the same model, and the nearest stored vectors win.…