AI agent memory
Every useful agent needs to remember something. The interesting questions are what it remembers, where that lives, who else can read it, and whether “forget that” means anything.
What AI agent memory actually is
An AI agent’s memory is whatever it can still use about you once the current conversation is over. That is a deliberately plain definition, and it rules out more than people expect. A model that recites your last twelve messages back to you is not remembering — it is reading. The test is what survives the moment the session closes.
This matters because the word “memory” is applied to four quite different mechanisms, with four different owners and four different failure modes. A team that buys one and expects another ends up with a system that works in the demo and disappoints in the second week.
The four kinds of agent memory
1. The context window
The text the model can see in this one request. It is working attention, not memory. It is also the most expensive of the four, because every remembered fact is re-read and re-billed on every turn. Larger context windows make longer conversations possible; they do not make memory.
2. Session state and scratchpads
Notes an agent keeps while it works — a plan, a running total, the file it is editing. This survives a few turns and dies with the session. It is the right tool for a task and the wrong tool for a relationship.
3. Retrieval over a stored corpus
Documents, transcripts and notes are embedded into a vector database, and the agent searches them before answering. This is durable and genuinely useful, and it is what most products mean when they advertise memory. Its defining property is ownership: the store belongs to the application. If the application goes away, or you move to a different agent, the memory does not come with you.
4. Identity-bound portable memory
Memory attached to the person or organization rather than to the application. The agent is a reader of the memory, not its owner, so a different agent can read the same memory tomorrow. This is the category SAIHM implements, and the rest of this page is about what it takes to do it honestly.
The first three are complementary, not competitors. A well-built agent uses all of them. The argument here is only about which one holds the things you would be unhappy to lose.
The question that gets asked last: who can read it?
Almost every evaluation of agent memory starts with recall quality and latency. Custody — who is technically able to read the stored content — tends to come up only after the first uncomfortable question from a legal or security reviewer. It is worth putting first, because it is an architectural property and cannot be added later.
| Where the memory lives | Who can technically read it | What happens when you leave |
|---|---|---|
| Inside the agent vendor’s account | The vendor, and anyone the vendor is compelled or breached into disclosing to | It stays with the vendor |
| A vector database you host | Anyone with database access, including your own operators and backup systems | You keep it, in that database’s shape |
| A managed store, encrypted at rest | The operator — encryption at rest protects the disk, not the operator’s query path | You export it, if there is an export |
| Sealed before it is stored, under a key held by the user | Only the holder of the key | It travels with the key |
“Encrypted” is the word that hides the most in that table. Encryption at rest means the storage medium is protected against physical theft. It says nothing about whether the party running the service can read your records in the ordinary course of operating it — usually they can, because they have to be able to serve queries. The distinction that matters is where the plaintext exists. If it exists on the operator’s side at any point, the operator is in your trust boundary.
This is a page about architecture rather than products. For a product-by-product view, the comparison page and the decision matrix cover named alternatives.
Cross-model memory: surviving the switch
Model choice is now something teams revisit every few months, and individuals revisit whenever something better ships. Memory tied to one vendor turns that routine decision into a migration.
The practical symptom is familiar. An assistant that has learned your clients, your constraints and your preferences over six months becomes an argument against ever changing assistant — not because the assistant is better, but because the memory is hostage to it. That is switching cost dressed as loyalty, and it degrades over time into using a worse tool for longer than you should.
Portable memory inverts it. If the memory is held under your key and read through an open protocol, a new agent that speaks the same protocol reads the same memory on its first run. The switch takes minutes, and the six months of accumulated context comes along. The agent becomes replaceable, which is the correct relationship to have with a piece of software that is improving this quickly.
Sovereign memory: what ownership has to mean
“You own your data” is claimed by nearly every product, and it usually resolves to a delete button and an export in a format only that product reads. Ownership is testable, and three questions do most of the work:
- Can anyone else read it? Not by policy — by mechanism. A promise not to look is a different security property from an inability to look.
- Can you take it somewhere else? Into a different agent, from a different vendor, without a conversion project.
- Can you actually destroy it? Including the copies, replicas and backups you never see.
Sovereignty is the conjunction of the three. Any two without the third leaves a gap that shows up at exactly the wrong moment — during a security review, a vendor change, or a deletion request with a legal deadline attached.
Erasure is not deletion
Deletion in most systems means the record stops appearing in results. The bytes remain in replicas, in snapshots, in backup sets with their own retention schedules, and in whatever log captured the write. This is adequate for tidying a working set and inadequate for a deletion obligation.
SAIHM’s erasure model is built on a different mechanism. Each memory is sealed under its own key. Erasing a memory destroys that key rather than chasing the ciphertext, so every remaining copy — replicas, snapshots, backups, anything already distributed — is left as unreadable ciphertext. Nothing has to be found, and nothing has to be trusted to have carried out a delete. What is left cannot be read by SAIHM either, which is the property that makes it worth stating.
The trade is real and worth being explicit about: erasure is irreversible, and so is losing your key. There is no recovery path, by design, because a recovery path is by definition a way for someone else to read your memory. That is the same property viewed from the other side, and it is why the key deserves the care you would give a password manager.
This is the mechanism SAIHM uses to satisfy the right to erasure under GDPR Article 17 — the difference between erasure and de-listing, stated as an architectural property rather than an operational promise. The compliance reports and the trust page set out how it is evidenced.
Sharing without copying
Memory that cannot be shared is a diary. Most real uses involve someone else reading something: a colleague, a specialist, an accountant, another agent working on the same problem.
The usual implementation of sharing is a copy, and a copy is permanent. Once the recipient holds the plaintext, revocation is a request rather than a control. SAIHM shares by re-wrapping a single memory’s key for one named recipient. Exactly that one memory is exposed, to exactly that one recipient, and revoking withdraws future reads.
Being precise about the limit: revocation governs what happens next. It cannot retract what a recipient has already read, and no sharing system honestly claims otherwise. What it does give you is a share that ends when you say it ends, scoped to one item rather than to a folder, a workspace, or an account.
How SAIHM puts this together
SAIHM is a memory protocol, not an AI agent. Any agent that speaks the standard agent protocol can use it — commercial assistants, open-source agents you run yourself, and agents that have not shipped yet.
- Sealed before it leaves the device
- Memories are encrypted where they are created. The storage side holds ciphertext.
- Bound to you, not to an app
- Memory is held under your key, so a different agent reads the same memory.
- Shared one item at a time
- Access is granted per memory, per recipient, and withdrawn on your instruction.
- Erased by destroying the key
- Irreversible, and effective against copies that cannot be reached.
- Open protocol, Apache-2.0
- The interface is public, so portability is checkable rather than promised.
For interface details and integration, see the developers page and the documentation. For how it reads day to day, individuals and enterprise cover the two ends.
Start free
The free tier needs no card, no wallet and no crypto. Tell your agent “Join SAIHM”, or run one command:
npx -y @saihm/mcp-server-pro free-join
It shows a short code and a link. Open the link, approve it, and the agent has memory from that moment. Prefer to start in a browser? /free does the same thing.
Paid plans are monthly. Card worldwide through Stripe; card and M-PESA in Africa through Paystack. Stablecoin is available for those who prefer it and is the only route that needs a wallet. Prices are published on the pricing page rather than quoted per customer.
Common questions
- What is AI agent memory?
- It is whatever an AI agent can still use about you after the current conversation ends. That excludes the context window, which is re-read on every request rather than remembered, and it includes stored notes, retrieved documents, and memory bound to your identity rather than to one application.
- Is a larger context window the same as memory?
- No. A context window is working attention for one request, and everything in it is re-read and re-billed on every turn. It makes longer conversations possible but does not persist anything once the session ends.
- Is a vector database enough for agent memory?
- It is durable and useful, and it is what most products mean by memory. Its limit is ownership: the store belongs to the application, so the memory does not follow you to a different agent, and whoever can query the database can read its contents.
- Can my AI agent’s memory move between models and vendors?
- It can if the memory is held under your key and read through an open protocol rather than stored inside one vendor’s account. That is the property SAIHM is built for: a new agent that speaks the same protocol reads the same memory on its first run.
- What does it take for “forget that” to be real?
- The content has to become unreadable rather than merely hidden. SAIHM erases by destroying the key that decrypts a memory, so replicas, snapshots and backups are left as unreadable ciphertext instead of having to be located and deleted.
- Does encrypted at rest mean the operator cannot read my memory?
- No. Encryption at rest protects the storage medium. If plaintext exists on the operator’s side in order to serve queries, the operator can read it. The distinguishing question is where the plaintext exists, not whether the disk is encrypted.
- Do I need crypto to use SAIHM?
- No. The free tier needs no payment method, and paid plans are payable by card through Stripe, or card and M-PESA through Paystack in Africa. Stablecoin is an option for people who prefer it.
Related
- Multi-agent memory — agents, swarms and robot fleets coordinating without wiring.
- Comparison — named alternatives, side by side.
- Decision matrix — which kind of memory fits which requirement.
- Developers — interface, integration and tooling.
- Trust — what is verifiable and how.
- Blog — longer pieces on custody, erasure and model switching.