A weathered wall of card-catalog drawers with blank brass label plates, one drawer half-open with a slipped card tied by a single red thread to a small brass desk lamp
All posts

How to protect AI agent memory from poisoning

Memory poisoning persists across sessions. Scope writes to a verified user, expire on policy, record provenance, and test what a saved claim does later.

A claim an agent saves today can authorize an action months from now. Memory poisoning is the attack class where adversarial content enters an agent's persistent store — through a conversation, a retrieved document, or a tool response — and keeps shaping behavior after the interaction that introduced it is gone. OWASP classifies it as ASI06: Memory and Context Poisoning in its 2026 Top 10 for Agentic Applications.

The defense is a write path you control. Authorize memory writes outside the model, bind them to a verified user, expire them on a policy you chose, and treat everything a memory asserts as untrusted context. Instructions asking the model to be careful are guidance; they are never the boundary.

Memory poisoning outlives the conversation that plants it

An attacker does not always need direct access to the memory database. A conversation, a retrieved document, or a tool response can persuade the agent to save the attack on the attacker's behalf. Research on the MINJA attack demonstrates this query-only path: adversarial interactions induce an agent to generate malicious records that are stored and later retrieved as demonstrations. Its threat model assumes shared memory, or an isolated-memory setting in which the attacker can disguise their identity. This is evidence for a specific attack path, not proof that every memory system is equally exposed.

Prompt injection redirects a model through input it processes now. Memory poisoning targets what the system carries forward. The two overlap: an injected instruction can persuade an agent to save malicious context, which then influences sessions the attacker never touches. Closing the original conversation does not remove that contamination.

Controls for the memory write path

Use the following as an application security checklist, not as a list of automatic AIVAX protections. Enforce consequential decisions in application code and in the systems that execute actions.

Recommended controls for persistent agent memory
Threat Control
A forged user or source plants a memory Resolve identity server-side; authorize the writer and destination scope.
A saved claim becomes policy Keep preferences separate from authorization; recheck authoritative records before acting.
An unsupported claim persists Record provenance and require review before admitting high-impact changes.
A stale or poisoned entry keeps returning Set retention by data type; support targeted deletion and inspect downstream copies.

Define ownership before enabling writes. Associate each entry with an authenticated application identity and an accountable operator, and never accept an arbitrary user identifier from untrusted input as proof of ownership. Decide which assistants may read or modify the same memory set, then test those boundaries with separate users and gateways.

Limit what memory can decide, too. Remembering a preferred language is different from remembering a spending limit. Keep permissions, refund eligibility, and other consequential state in authoritative systems, and validate them at action time; a remembered claim must not grant access. The same separation applies to authorization at the MCP tool boundary.

Review changes according to their impact. For low-risk preferences, a constrained automatic write may be appropriate. For claims that could affect money, access, or sensitive decisions, use an application-controlled review step before the entry becomes usable — or do not store the claim as agent memory at all. Keep the source conversation or document reference, writer, timestamp, proposed change, and approval decision in an access-controlled audit record, because a list of current memories cannot reconstruct overwritten values by itself.

Expiration deserves the same deliberation. Match retention to how quickly the underlying fact changes, and require fresh evidence before extending a consequential memory; otherwise repeated updates can preserve an old assumption. Expiry limits future reuse, but it cannot undo an action already taken or erase copies in other systems.

How AIVAX scopes memory writes

AIVAX stores memories as documents in the account's @memories collection and records the external user reference in each document's external_user_id metadata. The Memory built-in tool documentation describes identification through a chat client tag, or the request's user field in OpenAI-compatible chat completions. Model-managed memory operations require that user reference; without it, the tools return an error. Your application must bind the reference to the correct user rather than let the model choose it.

There can also be a gateway link, but do not assume gateway isolation by default. Shared-memory visibility is enabled by default in the current configuration and can include records linked to other gateways for the matching user scope. Restrict sharing when continuity across gateways is not intended, and verify the effective configuration rather than relying on a gateway's name.

Searches are restricted to the current user's identifier. Because the identifier is the isolation boundary, issue stable, unguessable references from your own user records, and validate how you generate them before relying on them for separation or bulk cleanup.

What the model can write and retrieve

With the Remember tool enabled, the model can call two tools: memory_save and memory_search. memory_save creates, replaces, or deletes a memory, or changes only its expiration. The model chooses the content and can supply expiresInDays, from 1 to 365; when it omits that value, a new memory never expires. Neither the tool nor its defaults enforce an approval policy.

The write function enforces a size limit on the content; it caps length only and passes the content through unvalidated. The earlier memory_update, memory_remove, and memory_clear tools and the Calendar reminders no longer exist, so update any tool-call handling or review process that still expects them.

Memories are not added to the system instructions automatically. The model must call memory_search, which takes either a query for semantic search or a filter for exact text, metadata, and creation or update dates, and returns up to 10 results with their IDs. Tell the model when to search. An ID helps identify a record during investigation, but does not prove that the record caused a particular answer.

How operators inspect, expire, or delete entries

The dedicated Memories API and dashboard page were removed. Inspect and manage memories through the Collections and Documents APIs or the dashboard's Collections and Documents pages: each document carries its content, creation and update timestamps, and metadata with the user, conversation, gateway, and expires_at. The @ prefix is reserved for system collections, so you cannot create or rename a collection to use it. These surfaces support review, but they keep no immutable write log and offer no approval queue.

Expired memories are hidden from memory_search immediately, while a periodic cleanup deletes them later, so expiry is not immediate permanent erasure. For a confirmed bad entry, delete the document by ID. To remove one user's memories, filter the documents by external_user_id and inspect the affected set before deleting. Deleting the @memories collection erases every user's memories in the account.

If your application requires approval before every write, retain control of that write path instead of treating unrestricted model-managed memory as an approval workflow.

Test the delayed effect before enabling memory

Stricter scope reduces unintended cross-assistant influence, but also reduces useful continuity. Short retention limits stale context, yet requires users to repeat information. Human review adds delay and operational cost, while keeping every source indefinitely can create a separate privacy problem. Choose retention and review requirements by the consequences of a wrong memory. Because AIVAX does not place saved memories in the system instructions, a new write does not change the start of later requests; a retrieved memory enters the conversation as a tool response instead.

The attack shows up later, so the test has to look later. In an isolated test environment, attempt an unauthorized write, inspect what was actually stored, then start a new conversation with the intended identity and check whether the claim changes the answer or a proposed action. Repeat with another user, another gateway, an expired entry, and a deleted entry. Verify the authoritative action system still rejects unauthorized operations even when the model repeats the poisoned claim.

Should the agent write its own memory at all? Allow it only when the permitted content, identity scope, retention, and consequences are understood. Constrained writes for low-risk preferences may be useful; a model's saved assertion must never establish permissions or policy, and a wrong write with significant consequences deserves an independently enforced review path.

If an incident occurs, stop further writes from the affected flow, preserve only the evidence needed under your retention policy, remove the bad records, and examine conversations or downstream artifacts that may already contain copies. Re-run the failed scenario after cleanup. Our discussion of the judge trajectory as a production signal explains why reviewing sequences of interactions is more informative than looking only at the final answer.

Behavior described here reflects the AIVAX implementation and documentation as read on 2026-11-04.