To make AI more useful for us, we gave it memory. Now the databases where those memories are stored are leaking. A look inside these systems shows both how AI sees its users and the moments that make human life uniquely our own.
A wall of 1,078 of the 15,327 memories stored about 115 of 442 people in an exposed Milvus database behind an AI companion robot for children. Each panel is one memory, and each outlined block is one person. People whose memories are similar sit near each other, and an AI-generated image floats behind each person's block. Focus the wall and use the arrow keys or W, A, S and D to move, with Shift to move faster. Press Enter to read the memory nearest the middle, Space to pause the drift, and Escape to close a memory.
Exposed Milvus database
115
of 442 people
1,078
of 15,327 memories
Drag, or focus the wall and use W A S D or the scroll wheel, to move. Click a panel to read it.
- One panel: one memory
- One outlined block: one person
Each panel is one memory and each outlined block is one person. People whose memories are similar sit near each other, and the wall wraps in every direction. The image behind each block is AI-generated from the memories stored about that person. Drag to move, click a panel to read it. This is a selection: 1,078 of 15,327 memories, from 115 of 442 people.
Models and memory
Large language models, the brains behind AI chatbots, are stateless. What we perceive as one continuous conversation is, for the LLM, a series of unconnected chunks of text arriving out of the aether.
The application layer massages this experience to turn a language model into an AI assistant. On each turn with a chatbot, all prior messages are sent along with the new message, allowing the LLM to use information from prior rounds in completing the current prompt.
Memory systems take this idea one step further, allowing AI applications to save information for use in any future conversation. Even something as simple as an AI assistant remembering my name requires that it be saved in a memory system of some kind. What seems natural, even inescapable, to humans—the persistence of memory—is achieved through clever engineering for AI.
Memory as technology
The technical design of memory systems ranges from simple text files to purpose-built technologies. To manage large collections of memories and improve search accuracy, dedicated technology stacks are being developed and deployed.
As with other new AI-supportive technologies, these memory systems are prone to misconfiguration and data leakage. Because of their particular role—not just storing data, but storing memories that an AI has identified as significant to the user—these leaks pose new kinds of privacy risks.
Mem0, Milvus, and Attu
One popular technology stack for a memory system combines Mem0 for the application layer, Milvus as a vector database, and Attu as a web frontend for the database. Together they allow for effective management of memories for AI chatbots, but they can also introduce data leakage risks.
Mem0 manages the interface between the LLM and the database where memories are stored, deciding what should be stored and when it should be retrieved. We take it for granted that our brains do this pretty well. Failing to recall a word on the tip of your tongue, the taste of a madeleine transporting you back to childhood—when our own memory does anything other than provide useful information, it appears as a curiosity, not a crippling overload or absence of information. AI applications also need the right balance of recording and retrieving memories while trying to pay attention to a given task, and Mem0 provides this.
Milvus is a vector database. It is used to store content—in this case small pieces of text, the “memories”—and embeddings, the calculated numerical descriptions of that content that allow similarity comparisons to other text. A vector database like Milvus allows the AI to query for memories related to the user’s name and efficiently identify the record where that is stored.
Finally, Milvus has a web frontend called Attu that allows for interaction with stored data through a graphical hosted interface. Web UIs for databases make it easier to monitor and interact with them, but they create an additional attack surface that needs to be configured securely and is more likely to be exposed to the web than the database itself.
Risks in the memory stack
None of these systems should be exposed to the public internet. When normal network-layer controls are missing, other controls—like requiring authentication to view the database contents—become single points of failure. Perhaps because the systems are designed with an expectation of basic network security, when they are publicly available, they often allow access to anonymous users.
We found a small number of Mem0 instances exposed directly to the internet, and only a handful without authentication. However, in reviewing the data exposed through Milvus Attu frontends lacking authentication, we often found the aforementioned stack: Milvus with Attu used as the storage layer for Mem0. While Mem0 was not the leak vector itself, it conditioned what kind of data was and was not accessible through the misconfigured databases.
Very personal, not so identifiable
Two exposures, one an app for children and one an app for elderly adults, illustrated how personal information manifests in memory systems. UpGuard notified the parties responsible for both databases.
Both data sets came from AI companion apps storing memories with key facts about end users—their names, likes and dislikes, anecdotes from their lives. The stored memory was the end result of the user talking to the AI companion and the memory system recognizing something worth storing for future conversations. These memories thus contained personal details that the person shared with an AI companion, recorded from the perspective of the AI.
Such fragments were not personally identifiable in the sense of pointing to named individuals, although in the case of the elder care app, the users’ phone numbers were used as unique identifiers. Rather, the memories held intimate details more personal than a national ID number despite being anonymous.
For the children, these memories were often universally applicable interests: they liked playing soccer, enjoyed cartoons, loved their pets and families. Those typical child interests were colored with more anxious disclosures of problems at home or school.
For the elders, the articulation of life’s satisfactions and discontents was similarly recognizable as the breadth of human experience. The AI memoirs of these users recorded expressions of how much they cherished their loved ones mixed with feelings of isolation and depression.
Not so personal, but at risk of ransom
As with Chroma, another vector database, open Milvus databases often contained knowledge base information that an AI support agent might query, or instructions on how a user expects an AI agent to perform certain tasks in the future. Even when memory systems are only used for impersonal notes, misconfiguration can pose risks.
Just as with Chroma databases, we found evidence of attacks by the JADEPUFFER group on Milvus databases in which ransom notes were inserted into the database. Even when vector databases do not expose sensitive data, they can provide a means for threat actors to vandalize the end user experience. More concerning, the ability to write to the database could allow them to insert malicious instructions for the LLM processing the retrieved “memory.”
A list of Milvus collections with one named JADEPUFFER, implanted by an attacker.
The poisoned collection's schema, including a field named ransomware.
AI’s expanding attack surface
Exposed databases are a well-worn form of cyber risk, but the architecture of AI systems has found a way to make them new again. Thanks to the properties of LLMs, additional storage strategies and technologies have found a niche. Those data stores carry not only new risks as points on the attack surface, but new forms of data exposure, as they reveal our personalities from the perspective of our AI interlocutors.