From Context to Memory: Your Agents Need a Real Memory Layer — Anders Swanson, Oracle

Hi, I'm Anders. I'm a Hi, I'm Anders. I'm a developer advocate developer advocate developer advocate for Oracle AI Database, for Oracle AI Database, for Oracle AI Database, and I'll talk about how we and I'll talk about how we and I'll talk about how we can approach can approach can approach agent memory as a agent memory as a agent memory as a systems problem, systems problem, systems problem, how to turn how to turn how to turn context into a real-world context into a real-world context into a real-world memory level that will memory level that will memory level that will help your help your help your agents perform agents perform agents perform better. When we better. When we better. When we talk about talk about talk about agent memory, we agent memory, we agent memory, we mean the composition of mean the composition of mean the composition of memory retrieval, memory retrieval, memory retrieval, memory formation, and the memory formation, and the memory formation, and the persistent storage that persistent storage that persistent storage that powers powers powers these memories. these memories. We've probably all We've probably all We've probably all used used used various agents various agents various agents like Claude Code or like Claude Code or like Claude Code or Codex and seen them Codex and seen them Codex and seen them work on a work on a work on a task task task by rereading the by rereading the by rereading the same files over and over again same files over and over again same files over and over again . This is because . This is because . This is because they have no knowledge they have no knowledge they have no knowledge of what of what of what happened in the happened in the happened in the past. Context past. Context past. Context windows are not memory. windows are not memory. windows are not memory. This is input for the This is input for the This is input for the current session, current session, current session, not a permanent experience. not a permanent experience. Let's say I have an Let's say I have an agent session agent session agent session working on working on working on customer support tickets: customer support tickets: customer support tickets: it reads a bunch of it reads a bunch of it reads a bunch of documents, executes documents, executes documents, executes queries, reproduces queries, reproduces queries, reproduces the error, finds the error, finds the error, finds the root cause, and the root cause, and the root cause, and does all this great does all this great does all this great work synthesizing a work synthesizing a work synthesizing a useful procedure. What useful procedure. What useful procedure. What happens happens happens at the end of the session?

at the end of the session? at the end of the session? It just It just It just gets thrown out. And that's gets thrown out. And that's gets thrown out. And that's because there is no because there is no because there is no permanence. After a permanence. After a permanence. After a long time, long time, long time, another another another agent appears, working on a agent appears, working on a agent appears, working on a similar task, and is similar task, and is similar task, and is forced to repeat forced to repeat forced to repeat all the same work. all the same work. He is again searching for He is again searching for documents, performing documents, performing documents, performing queries, which means more queries, which means more queries, which means more opportunities for opportunities for opportunities for errors, hallucinations, errors, hallucinations, errors, hallucinations, and ultimately less and ultimately less and ultimately less accuracy. Memory accuracy. Memory accuracy. Memory cannot begin until cannot begin until cannot begin until this useful experience of this useful experience of this useful experience of agents agents agents is transformed into a is transformed into a is transformed into a controlled state. controlled state. Maximizing Maximizing context windows is context windows is context windows is also not a solution. also not a solution. also not a solution. We just spend more and We just spend more and We just spend more and more more more tokens. The session becomes tokens. The session becomes tokens. The session becomes longer, longer, longer, data compression occurs. A data compression occurs. A data compression occurs. A useful signal for useful signal for useful signal for our agents our agents our agents gets lost in a pile of gets lost in a pile of gets lost in a pile of irrelevant irrelevant irrelevant information. Similarly, " information. Similarly, " drafts" are just drafts" are just drafts" are just workarounds. workarounds. For example, Markdown files : we start : we start : we start building large building large building large file hierarchies, wikis file hierarchies, wikis , prompts. This , prompts. This , prompts. This helps, but it is not a helps, but it is not a helps, but it is not a controlled controlled controlled data replay.

data replay. Files do not Files do not scale in scale in scale in large large large multi-user multi-user multi-user environments. environments. Tools Tools like grep are very like grep are very like grep are very useful for searching, useful for searching, useful for searching, but they don't but they don't but they don't scale well and don't scale well and don't scale well and don't provide provide provide sophisticated hybrid sophisticated hybrid sophisticated hybrid extraction. It seems to me that extraction. It seems to me that extraction. It seems to me that if we if we if we follow through with follow through with the attempt to build a the attempt to build a memory layer on top of memory layer on top of memory layer on top of files, we will eventually files, we will eventually files, we will eventually create our own create our own create our own database. But you can database. But you can database. But you can just use a just use a just use a database. In this database. In this database. In this presentation, we will presentation, we will presentation, we will explore the lessons explore the lessons explore the lessons we have learned as a we have learned as a we have learned as a database vendor database vendor database vendor working with working with working with customers customers customers developing developing developing agent memory, agent memory, agent memory, solving real-world solving real-world solving real-world problems, and problems, and problems, and requiring requiring data reproducibility. Modern Modern agent memory systems agent memory systems agent memory systems converge on a specific converge on a specific converge on a specific model that model that model that involves the involves the involves the formation, formation, formation, reproduction, reproduction, reproduction, development, and development, and development, and evaluation of memory. evaluation of memory. So when we So when we form memories, they form memories, they form memories, they are distilled are distilled are distilled experiences, experiences, experiences, structured facts. structured facts. structured facts. This is not just about This is not just about This is not just about uploading uploading uploading raw raw raw transcripts. When we transcripts. When we transcripts. When we recreate memories, recreate memories, recreate memories, hybrid search becomes the hybrid search becomes the hybrid search becomes the dominant method.

dominant method. dominant method. This is a form of search that This is a form of search that This is a form of search that combines different combines different combines different types of search. These are vector types of search. These are vector types of search. These are vector similarity search, similarity search, similarity search, lexical search, lexical search, lexical search, entity search by entity search by entity search by relational relational relational objects, objects, objects, graph traversal, and various graph traversal, and various graph traversal, and various time time time , relevance, and , relevance, and , relevance, and feedback signals. feedback signals. feedback signals. We also need We also need We also need consolidation in these consolidation in these consolidation in these systems. After all, systems. After all, systems. After all, duplicates may appear. duplicates may appear. duplicates may appear. We need to We need to We need to resolve conflicts resolve conflicts . The system must . The system must . The system must evolve over time. evolve over time. evolve over time. Otherwise, it Otherwise, it Otherwise, it will become outdated, irrelevant will become outdated, irrelevant will become outdated, irrelevant and useless. Memories and useless. Memories and useless. Memories require an element of require an element of temporality. So, temporality. So, when was this created when was this created ? When was this ? When was this ? When was this accessed? How accessed? How accessed? How long does this remain long does this remain long does this remain valid? Maybe it's valid? Maybe it's valid? Maybe it's only true during a only true during a only true during a certain time certain time certain time window. And all of this should window. And all of this should window. And all of this should be part of be part of be part of regular regular regular feedback and feedback and feedback and evaluation. If I don't evaluation. If I don't evaluation. If I don't measure usage measure usage measure usage and provide and provide and provide feedback on memory levels, it feedback on memory levels, it feedback on memory levels, it all fails all fails . Another important . Another important . Another important part is the part is the part is the management controls. management controls. management controls. So, secrets, So, secrets, So, secrets, personal information personal information —this has to be —this has to be —this has to be redacted before it redacted before it redacted before it reaches the reaches the reaches the memory layer. Rights should memory layer. Rights should memory layer. Rights should be limited: for be limited: for be limited: for an agent, for a an agent, for a an agent, for a workflow, a team, or workflow, a team, or workflow, a team, or a user. This is a user. This is a user. This is especially important in especially important in especially important in multi-tenant multi-tenant multi-tenant environments. We environments. We environments. We need appropriate need appropriate need appropriate administrative administrative administrative approvals, verification, approvals, verification, approvals, verification, and the ability to and the ability to and the ability to delete outdated delete outdated delete outdated or corrupted or corrupted or corrupted data. As we data. As we data. As we go through a go through a go through a memory cycle, the agent memory cycle, the agent memory cycle, the agent runs a session, runs a session, runs a session, performs work, reads performs work, reads performs work, reads documents, runs documents, runs documents, runs queries, synthesizes queries, synthesizes queries, synthesizes useful data. After useful data. After useful data. After or during this or during this or during this session, a fact is created session, a fact is created session, a fact is created . We record . We record . We record this fact by this fact by this fact by editing, editing, editing, enriching, and enriching, and enriching, and storing it in the storing it in the storing it in the persistence layer. And persistence layer. And persistence layer. And then, when

then, when then, when another another another session with the appropriate session with the appropriate session with the appropriate context begins, it can context begins, it can context begins, it can extract that fact and extract that fact and extract that fact and use it to use it to use it to reconstruct the reconstruct the reconstruct the context map. context map. context map. So, he So, he So, he uses a fact from the uses a fact from the uses a fact from the previous work previous work previous work to make the to make the to make the next task easier. This next task easier. This next task easier. This reduces the workload reduces the workload reduces the workload for the next for the next for the next agent. And if this agent. And if this agent. And if this agent successfully agent successfully agent successfully uses the fact, we uses the fact, we uses the fact, we can add can add can add feedback points to feedback points to feedback points to that memory. So, we that memory. So, we that memory. So, we adapt over adapt over adapt over time. We get time. We get time. We get related data and related data and related data and new time new time new time attributes. We attributes. We attributes. We turn to it again turn to it again turn to it again and do something with it. and do something with it. Now the memory Now the memory is called as a is called as a is called as a function by the function by the function by the agent runtime. agent runtime. agent runtime. Therefore, it needs Therefore, it needs Therefore, it needs levels of reading, writing, levels of reading, writing, levels of reading, writing, and storage. Your and storage. Your agent runtime can agent runtime can invoke the invoke the invoke the memory layer through hooks, MCPs, memory layer through hooks, MCPs, memory layer through hooks, MCPs, skills, or other skills, or other skills, or other methods. And the memory-level API methods. And the memory-level API exposes read, exposes read, write, write, write, feedback, and feedback, and feedback, and administration APIs, and administration APIs, and administration APIs, and then interacts with then interacts with then interacts with storage or storage or storage or databases. Regarding databases. Regarding databases. Regarding database selection, I find database selection, I find database selection, I find multi-model multi-model multi-model databases particularly databases particularly databases particularly useful because useful because useful because they allow you they allow you they allow you to work with rows, to work with rows, to work with rows, columns, columns, columns, vector vector vector representations, representations, representations, full-text full-text full-text search, relational search, relational search, relational objects, graphs, objects, graphs, objects, graphs, episodes, and facts all episodes, and facts all episodes, and facts all in one place. So in one place. So in one place. So your memory can your memory can your memory can be stored in a be stored in a be stored in a consolidated consolidated consolidated location with a single location with a single location with a single database connection database connection database connection . I can . I can . I can perform various perform various perform various types of hybrid types of hybrid types of hybrid searches and return the searches and return the searches and return the resulting data. What does resulting data. What does resulting data. What does this memory look like?

this memory look like? this memory look like? It will be It will be It will be typed, and you typed, and you typed, and you may have seen this may have seen this may have seen this model in various model in various model in various presentations or in presentations or in presentations or in some form. some form. some form. This is roughly This is roughly This is roughly borrowed from borrowed from borrowed from human memory and human memory and human memory and can be can be can be applied to applied to applied to agents. So, we have agents. So, we have agents. So, we have episodic memory. episodic memory. episodic memory. These are specific past These are specific past These are specific past events or fragments of events or fragments of events or fragments of transcripts that transcripts that agents worked on. We have agents worked on. We have semantic memory. semantic memory. semantic memory. These are specific facts These are specific facts These are specific facts about our about our about our workflows, workflows, workflows, user preferences, and other user preferences, and other user preferences, and other entities. We have entities. We have entities. We have procedural memory. procedural memory. procedural memory. This is how we This is how we This is how we performed a particular performed a particular performed a particular task or routine, task or routine, task or routine, such as an such as an such as an assembly step. These are assembly step. These are reusable rules that reusable rules that agents will agents will agents will call over and over again call over and over again call over and over again if they if they if they repeat similar repeat similar repeat similar work. And we have a work. And we have a work. And we have a working state. Working working state. Working working state. Working state is like state is like state is like short-term memory. short-term memory. short-term memory. This is something we This is something we This is something we are working on at the are working on at the are working on at the moment, and this moment, and this moment, and this temporary data temporary data temporary data can only be valid can only be valid can only be valid for a for a for a certain window of time, certain window of time, certain window of time, after which we can after which we can after which we can move it to move it to move it to another type of memory. another type of memory. another type of memory. This helps us This helps us This helps us organize data into organize data into organize data into different categories, different categories, different categories, and we can and we can and we can apply different apply different apply different search techniques to search techniques to search techniques to these types of memory. What is the these types of memory. What is the memory storage scheme? Memory is memory storage scheme? Memory is typed objects typed objects typed objects plus a search path. plus a search path. plus a search path. So, our So, our So, our memory object will have memory object will have memory object will have an identifier, an identifier, an identifier, such as a primary such as a primary such as a primary key. It will contain key. It will contain key. It will contain text content, JSON text content, JSON tags, and tags, and tags, and document data. It will have a document data. It will have a document data. It will have a vector vector vector representation. We representation. We representation. We create a create a create a content-based vector with a content-based vector with a content-based vector with a vector index vector index vector index for for for similarity search. Ideally, similarity search. Ideally, similarity search. Ideally, we would also have a we would also have a we would also have a text index to text index to text index to facilitate facilitate facilitate full-text full-text full-text keyword searches keyword searches keyword searches . Which is . Which is . Which is combined with combined with combined with vector search. We vector search. We vector search. We can have graph can have graph can have graph connections with different connections with different connections with different edges. Memories edges. Memories edges. Memories can be linked can be linked can be linked through an entity graph.

through an entity graph. through an entity graph. That is, this memory That is, this memory That is, this memory is connected to that is connected to that is connected to that memory, which is memory, which is memory, which is connected to connected to connected to another memory. This is another memory. This is another memory. This is additional context additional context additional context that we can that we can that we can extract during the extract during the extract during the search phase. We search phase. We search phase. We can also have can also have can also have temporary facts. To temporary facts. To temporary facts. To what extent what extent what extent is this memory valid? is this memory valid? is this memory valid? When was the last time we When was the last time we When was the last time we used it? And, used it? And, used it? And, extremely extremely extremely importantly, audit importantly, audit importantly, audit events. Who created events. Who created events. Who created this memory? What was this memory? What was this memory? What was it used for it used for ? A log of how it ? A log of how it ? A log of how it was used and was used and was used and related metadata. related metadata. This should all be This should all be part of your part of your part of your storage scheme when we storage scheme when we storage scheme when we work with memory. work with memory. work with memory. When we create When we create When we create memories, we want memories, we want memories, we want to shape them to shape them to shape them consciously. Good consciously. Good consciously. Good memories are very memories are very memories are very purposeful. So, purposeful. So, purposeful. So, we need to we need to we need to define the define the define the lifecycle, scope, various lifecycle, scope, various lifecycle, scope, various relationships, and relationships, and relationships, and associated metadata. associated metadata. associated metadata. When the When the When the agent runtime agent runtime agent runtime decides to create a decides to create a decides to create a memory, we must memory, we must memory, we must start with editing. start with editing. start with editing. Editing is a very Editing is a very Editing is a very important first important first important first step because before I step because before I step because before I do any embedding or do any embedding or do any embedding or enrichment, I want to enrichment, I want to enrichment, I want to remove personal remove personal remove personal data. I don't want data. I don't want data. I don't want secrets or irrelevant secrets or irrelevant secrets or irrelevant weird information to weird information to weird information to reach the reach the reach the memory level. This is a recipe for memory level. This is a recipe for memory level. This is a recipe for disaster. So we disaster. So we disaster. So we do do do the editing first. Then we the editing first. Then we the editing first. Then we can enrich can enrich can enrich the memory. What is the memory. What is the memory. What is this this this memory connected with? What is the metadata of memory connected with? What is the metadata of memory connected with? What is the metadata of this memory? We this memory? We this memory? We can apply can apply can apply scope to it.

scope to it. scope to it. Which agent worked Which agent worked Which agent worked on this memory? What on this memory? What on this memory? What team is this? What team is this? What team is this? What organization is this? This organization is this? This organization is this? This helps group helps group helps group the data so that I don't the data so that I don't the data so that I don't get get get memories memories memories related to other related to other related to other agents or agents or agents or workflows in other sessions, which creates workflows in other sessions, which creates workflows in other sessions, which creates confusion. We can confusion. We can confusion. We can do embedding, do embedding, do embedding, that is, create a that is, create a that is, create a content vector. And content vector. And content vector. And then we can then we can then we can form what is form what is form what is called an episode. These are called an episode. These are called an episode. These are fragments of fragments of fragments of the transcript from which the the transcript from which the the transcript from which the memory was created. memory was created. memory was created. This is useful for This is useful for This is useful for verifying provenance verifying provenance verifying provenance and additional and additional and additional context. Any organization- context. Any organization- context. Any organization- specific specific specific policies policies policies can also be can also be can also be applied here. Everything applied here. Everything applied here. Everything you need for you need for you need for your specific your specific your specific use case use case . And then we can . And then we can . And then we can save the data to the save the data to the save the data to the storage tier. storage tier. When we create When we create memories, they can memories, they can memories, they can be quite be quite be quite compact: we compact: we compact: we take the transcript and take the transcript and take the transcript and compress it down to compress it down to compress it down to content, metadata, content, metadata, content, metadata, embedding, and embedding, and embedding, and related things, and related things, and related things, and we create those we create those we create those episodes. So, we episodes. So, we episodes. So, we take the take the take the agents' transcripts, filter agents' transcripts, filter agents' transcripts, filter them, find them, find them, find candidates for those candidates for those candidates for those episodes, save a episodes, save a episodes, save a memory record, and memory record, and memory record, and link it link it link it back. You can back. You can back. You can do this do this do this using relationships, using relationships, using relationships, just regular just regular just regular relationships in a database relationships in a database . You can do this . You can do this . You can do this with graphs.

with graphs. with graphs. I especially I especially I especially like graphs like graphs like graphs because they because they because they allow for broader allow for broader allow for broader searching. When we searching. When we searching. When we talk about talk about talk about reliving memories reliving memories , that is, how do we , that is, how do we , that is, how do we remember something? This is a remember something? This is a remember something? This is a search problem, that is, a search problem, that is, a search problem, that is, a query problem to the query problem to the query problem to the storage layer. And storage layer. And storage layer. And when retrieving or when retrieving or when retrieving or reproducing, reproducing, reproducing, hybrid search becomes the hybrid search becomes the hybrid search becomes the dominant method of dominant method of context reconstruction. And this form context reconstruction. And this form of search can of search can of search can combine different combine different combine different methods. So, I can methods. So, I can methods. So, I can use use use vector vector vector similarity search, similarity search, similarity search, full-text full-text full-text keyword search, keyword search, keyword search, relational entities, relational entities, relational entities, graph traversal, other graph traversal, other graph traversal, other feedback signals feedback signals feedback signals like like like importance or importance or importance or recall usefulness recall usefulness recall usefulness to get more to get more to get more comprehensive comprehensive comprehensive results than from a results than from a results than from a single query branch. single query branch. single query branch. For example, if I were to For example, if I were to For example, if I were to only perform a only perform a only perform a similarity search, I would similarity search, I would similarity search, I would only get the only get the only get the most similar most similar most similar results results results according to the according to the according to the embedding model I am embedding model I am embedding model I am using. Um, and using. Um, and using. Um, and overall it wasn't overall it wasn't overall it wasn't enough. So enough. So enough. So we have all these different we have all these different we have all these different query branches, different query branches, different query branches, different types of memory, and you types of memory, and you types of memory, and you can decide which ones to can decide which ones to use. For use. For things like facts, things like facts, things like facts, you may not you may not you may not need need need similarity, similarity, similarity, you may want to do an you may want to do an you may want to do an exact match by exact match by exact match by ID. We ID. We ID. We can take these can take these can take these results, combine results, combine results, combine them, apply them, apply them, apply thresholds, thresholds, thresholds, perform perform perform reranking, and reranking, and reranking, and get a get a get a context card with the context card with the context card with the scored data. We scored data. We scored data. We can say that can say that can say that this memory element this memory element this memory element received a high received a high received a high score in vector score in vector score in vector search and a slightly lower score in search and a slightly lower score in search and a slightly lower score in lexical search. This is the lexical search. This is the lexical search. This is the origin of memory, origin of memory, origin of memory, where it came from and where it came from and where it came from and how we got it. This is how we got it. This is how we got it. This is also important. When also important. When also important. When we consider

we consider we consider hybrid search: hybrid search: hybrid search: vector distance vector distance vector distance is useful for is useful for is useful for semantically similar semantically similar semantically similar things, the " things, the " contains" operator is great for contains" operator is great for contains" operator is great for lexical search, lexical search, lexical search, plus graphical plus graphical plus graphical tables, temporal data, tables, temporal data, tables, temporal data, etc. We take these etc. We take these etc. We take these branches and apply branches and apply branches and apply weights to them. We can weights to them. We can weights to them. We can assign weights assign weights assign weights depending on needs. depending on needs. depending on needs. This is something that can be This is something that can be This is something that can be configured in your configured in your configured in your hybrid hybrid hybrid search algorithm. The results search algorithm. The results search algorithm. The results are combined into a are combined into a are combined into a total score from the total score from the total score from the different branches of the query, different branches of the query, different branches of the query, and then we can and then we can and then we can take the top-K take the top-K take the top-K results to results to results to say: here are the say: here are the say: here are the top three results from top three results from top three results from our hybrid our hybrid our hybrid search. This is roughly how search. This is roughly how search. This is roughly how you can implement you can implement you can implement hybrid search in a hybrid search in a hybrid search in a multi-model multi-model multi-model database that is capable of database that is capable of database that is capable of providing all of these providing all of these providing all of these search methods from a search methods from a search methods from a single interface. single interface. Memory is not just a Memory is not just a storage where we storage where we storage where we write and write and write and read data. To read data. To read data. To be truly be truly be truly useful, memory must useful, memory must useful, memory must develop over time. develop over time. develop over time. We also don't want to We also don't want to We also don't want to constantly constantly constantly load all load all load all the data, otherwise we'll the data, otherwise we'll the data, otherwise we'll end up with a bunch of end up with a bunch of end up with a bunch of garbage memory. garbage memory. garbage memory. This is where we can This is where we can This is where we can close the memory loop close the memory loop close the memory loop through through through consolidation and consolidation and consolidation and error handling.

error handling. error handling. Outdated memory Outdated memory Outdated memory can appear when, can appear when, can appear when, say, my project say, my project say, my project used NPM and used NPM and used NPM and then we decided to then we decided to then we decided to switch to a different switch to a different switch to a different build system. What build system. What build system. What happens to all happens to all happens to all the memory associated with the memory associated with the memory associated with NPM? If the agent NPM? If the agent NPM? If the agent approaches her, it approaches her, it approaches her, it will be problematic and will be problematic and will be problematic and he may try to he may try to he may try to do something wrong. do something wrong. do something wrong. The system must The system must The system must expect expect expect outdated facts to appear outdated facts to appear outdated facts to appear due to due to due to context drift during context drift during context drift during recall. recall. recall. Incorrect Incorrect Incorrect context can context can context can occur when we occur when we occur when we improperly improperly improperly limit limit limit memory boundaries. It's like if you memory boundaries. It's like if you memory boundaries. It's like if you logged into your logged into your logged into your ChatGPT account and saw ChatGPT account and saw someone else's message history. Um, someone else's message history. Um, if you don't if you don't if you don't apply proper apply proper apply proper demarcation, you demarcation, you demarcation, you can get can get can get serious serious serious data leaks. So, this should data leaks. So, this should data leaks. So, this should happen during happen during happen during writing and during writing and during writing and during reading. reading. reading. Memory poisoning is when an Memory poisoning is when an Memory poisoning is when an untrusted source untrusted source untrusted source places a malicious places a malicious places a malicious instruction in your instruction in your instruction in your memory level. This is probably the memory level. This is probably the memory level. This is probably the worst-case worst-case worst-case scenario: a scenario: a scenario: a request injection occurred, request injection occurred, request injection occurred, you had no you had no you had no safeguards, safeguards, safeguards, proper validation, proper validation, proper validation, censorship, or censorship, or censorship, or memory control, and memory control, and memory control, and something very dangerous got in there, something very dangerous got in there, leaking all your data . So be very . So be very . So be very careful to careful to careful to avoid such avoid such avoid such things through things through things through proper proper proper safeguards and safeguards and safeguards and administrative administrative administrative actions.

actions. actions. Memory conflict is when Memory conflict is when Memory conflict is when I have two different I have two different I have two different fragments of memory. fragments of memory. fragments of memory. One says, "Hey, we One says, "Hey, we One says, "Hey, we do the task do the task do the task like this." And the other one says, like this." And the other one says, like this." And the other one says, "Well, that's basically how we "Well, that's basically how we "Well, that's basically how we do it." do it." do it." Since both are Since both are Since both are about the same about the same about the same task, task, task, they will they will they will rank highly in rank highly in rank highly in search. So you'll search. So you'll search. So you'll probably get probably get probably get these same conflicts in these same conflicts in these same conflicts in your context your context your context card. And when the agent card. And when the agent card. And when the agent reads it, he's like, " reads it, he's like, " Well, you told me Well, you told me Well, you told me the restaurant is on the right, the restaurant is on the right, the restaurant is on the right, but it's actually on but it's actually on but it's actually on the left. I don't know what the left. I don't know what the left. I don't know what to do." So to do." So to do." So detection and detection and detection and verification are very verification are very verification are very important here for important here for important here for conflict resolution conflict resolution . Demarcation too. . Demarcation too. . Demarcation too. Sometimes there are different ways Sometimes there are different ways Sometimes there are different ways to accomplish the same to accomplish the same to accomplish the same task. task. task. Over-provisioning and Over-provisioning and Over-provisioning and irrelevance irrelevance irrelevance arise as arise as arise as your your your memory system grows. If memory system grows. If memory system grows. If we just cram we just cram we just cram all the data in there without all the data in there without all the data in there without thinking about thinking about thinking about what we're writing, we can what we're writing, we can what we're writing, we can end up in an end up in an end up in an over-supplied situation over-supplied situation over-supplied situation where we have so many where we have so many where we have so many weak matches that weak matches that weak matches that they overwhelm the they overwhelm the they overwhelm the context card. This is the context card. This is the context card. This is the same problem as same problem as same problem as with a very large with a very large with a very large context window context window context window where we try to where we try to where we try to cram as much cram as much cram as much data as possible. It's data as possible. It's data as possible. It's just confusing. just confusing. just confusing. Irrelevance—these are Irrelevance—these are Irrelevance—these are semantically close semantically close semantically close memories that are highly memories that are highly memories that are highly ranked by hybrid ranked by hybrid ranked by hybrid search but are search but are search but are not actually not actually not actually relevant to the task.

relevant to the task. relevant to the task. They just got a They just got a They just got a high rating. So high rating. So , these are issues of , these are issues of , these are issues of managing and managing and managing and tuning your tuning your tuning your search algorithm search algorithm . A data leak is when . A data leak is when . A data leak is when strange, unexpected strange, unexpected strange, unexpected data enters a data enters a data enters a system's memory. This is a system's memory. This is a system's memory. This is a less serious version of less serious version of less serious version of memory poisoning. memory poisoning. memory poisoning. Proper enrichment Proper enrichment Proper enrichment and filtering of and filtering of and filtering of memory records during memory records during memory records during recording is a way recording is a way recording is a way to solve the to solve the to solve the leakage problem. Over time, we leakage problem. Over time, we leakage problem. Over time, we also get also get also get hidden drift. This is hidden drift. This is hidden drift. This is when the memory system when the memory system when the memory system grows. grows. grows. Maybe it works Maybe it works Maybe it works just fine for just fine for just fine for thousands of records. But thousands of records. But thousands of records. But then when I have then when I have then when I have 50 thousand, a million or 50 thousand, a million or 50 thousand, a million or more, I get all more, I get all more, I get all these weird results these weird results these weird results and it doesn't and it doesn't and it doesn't work at all. So, it is very, work at all. So, it is very, work at all. So, it is very, very important to very important to very important to apply apply apply proper countermeasures: proper countermeasures: proper countermeasures: pre- pre- pre- record editing, record editing, record editing, scope checking, scope checking, scope checking, lifecycle and, lifecycle and, lifecycle and, most importantly, most importantly, most importantly, feedback. feedback. feedback. Feedback Feedback is what closes the is what closes the is what closes the memory loop. The quality of memory loop. The quality of memory loop. The quality of memory should be memory should be memory should be measurable. We measurable. We measurable. We need to need to need to get the data correctly. We get the data correctly. We get the data correctly. We must consider must consider must consider outdated or outdated or outdated or harmful memories and, harmful memories and, harmful memories and, ultimately, the usefulness of the ultimately, the usefulness of the ultimately, the usefulness of the system for system for system for subsequent processes. subsequent processes. subsequent processes. This should influence This should influence This should influence decisions about the decisions about the ranking lifecycle. So, if an ranking lifecycle. So, if an agent receives a agent receives a agent receives a potential memory, potential memory, potential memory, it will either use it will either use it will either use it or reject it.

it or reject it. it or reject it. Maybe if he Maybe if he Maybe if he uses it, everything will go awry uses it, everything will go awry uses it, everything will go awry and the and the and the system will fail. system will fail. system will fail. This This This feedback should feedback should feedback should flow back. And flow back. And flow back. And if the memory was if the memory was if the memory was harmful, the feedback score harmful, the feedback score should go down. If should go down. If it was useful, it was useful, it was useful, maybe we'll up the maybe we'll up the maybe we'll up the rating. So it's rating. So it's rating. So it's not just a storage facility. not just a storage facility. not just a storage facility. He must keep it. He must keep it. He must keep it. Your memory level Your memory level Your memory level should store should store should store information about whether the information about whether the information about whether the memories were memories were memories were useful, useful, useful, outdated, outdated, outdated, irrelevant, irrelevant, irrelevant, harmful, etc. We harmful, etc. We harmful, etc. We really want to really want to really want to design design design measurable measurable measurable data retrieval with data retrieval with data retrieval with controlled controlled controlled memory formation memory formation memory formation built on a built on a built on a multimodal multimodal multimodal storage platform storage platform . Without memory, these . Without memory, these . Without memory, these agents are more agents are more agents are more likely likely likely to restart the same to restart the same tasks. They will tasks. They will create everything again create everything again create everything again and again. They will and again. They will and again. They will read the same files. read the same files. read the same files. They will They will They will execute the same execute the same execute the same queries. They queries. They queries. They will waste a lot of will waste a lot of will waste a lot of time, a lot of tokens, time, a lot of tokens, time, a lot of tokens, work work work much slower, much slower, much slower, and be much and be much and be much more more more error-prone. They will error-prone. They will error-prone. They will repeat the same repeat the same repeat the same mistakes. If the agent is mistakes. If the agent is mistakes. If the agent is unaware of unaware of unaware of previous experience, previous experience, previous experience, it will do the same thing it will do the same thing it will do the same thing again. And if this action again. And if this action again. And if this action was wrong, he was wrong, he was wrong, he will make mistakes again will make mistakes again will make mistakes again and again. And with managed and again. And with managed and again. And with managed memory, agents memory, agents memory, agents can continue where they can continue where they left off. They left off. They can can can use what use what use what came before them and, came before them and, came before them and, ultimately, reduce ultimately, reduce ultimately, reduce the number of the number of the number of repeated failures.

repeated failures. repeated failures. This is the true purpose of This is the true purpose of This is the true purpose of recording, receiving, recording, receiving, recording, receiving, controlling, and controlling, and controlling, and evolving. We want evolving. We want evolving. We want to increase the to increase the to increase the success rate. We want to success rate. We want to success rate. We want to do it faster. We do it faster. We do it faster. We want to do it want to do it want to do it cheaper. We want to cheaper. We want to cheaper. We want to do it better. If do it better. If do it better. If you prefer hands-on you prefer hands-on you prefer hands-on experience, I experience, I experience, I recommend recommend recommend checking out our checking out our checking out our Python SDK for Python SDK for Python SDK for agent memory. This code agent memory. This code agent memory. This code will take you to a will take you to a will take you to a link to GitHub with a link to GitHub with a link to GitHub with a Python notebook Python notebook Python notebook showing how to showing how to showing how to do this using do this using do this using our SDK. This SDK our SDK. This SDK our SDK. This SDK is built on is built on is built on database primitives database primitives database primitives . Everything we're . Everything we're . Everything we're talking about here exists in the talking about here exists in the talking about here exists in the database. We database. We database. We also have our own also have our own also have our own developer center. If you are an developer center. If you are an developer center. If you are an AI engineer or AI engineer or AI engineer or developer and you are developer and you are developer and you are wondering how a modern wondering how a modern wondering how a modern multimodal multimodal multimodal database fits into database fits into database fits into artificial artificial artificial intelligence development? This is the same intelligence development? This is the same intelligence development? This is the same place where you can place where you can place where you can review our review our review our examples, see examples, see examples, see what's possible, and maybe what's possible, and maybe even get inspired even get inspired even get inspired to build something to build something to build something cool with a cool with a cool with a great database. Uh, great database. Uh, great database. Uh, again, I'm Anders, uh, again, I'm Anders, uh, again, I'm Anders, uh, Anders Svensson. If Anders Svensson. If Anders Svensson. If you want to you want to you want to chat or chat or chat or contact me, I'll contact me, I'll contact me, I'll be at the Oracle booth. Well be at the Oracle booth. Well be at the Oracle booth. Well , thank you very much. , thank you very much.

, thank you very much. I hope this was I hope this was I hope this was useful to you, and, uh, useful to you, and, uh, useful to you, and, uh, helped you helped you helped you create great create great create great things. Thank you very much.