Thinner Agents on a Smarter Substrate: The Ontology-based Semantic Layer — Emil Eifrem, Neo4j

All right. At Neo Forj, we work with All right. At Neo Forj, we work with some of the largest companies in the some of the largest companies in the some of the largest companies in the world to help make their data ready for world to help make their data ready for world to help make their data ready for AI agents. And today I want to talk to AI agents. And today I want to talk to AI agents. And today I want to talk to you about a problem that we saw emerging you about a problem that we saw emerging you about a problem that we saw emerging over the last call it six to nine months over the last call it six to nine months over the last call it six to nine months and propose a solution blueprint for and propose a solution blueprint for and propose a solution blueprint for that. So let's say that we work at a big that. So let's say that we work at a big that. So let's say that we work at a big organization a big bank and we want to organization a big bank and we want to organization a big bank and we want to write an agent. Let's say that agent is write an agent. Let's say that agent is write an agent. Let's say that agent is helping automate the opening of a bank helping automate the opening of a bank helping automate the opening of a bank account. Right? You can imagine that's account. Right? You can imagine that's account. Right? You can imagine that's very ripe for automation. You want to be very ripe for automation. You want to be very ripe for automation. You want to be able to orchestrate that process. And able to orchestrate that process. And able to orchestrate that process. And I'm going to use the powers bestowed I'm going to use the powers bestowed I'm going to use the powers bestowed upon me by a short keynote slot to upon me by a short keynote slot to upon me by a short keynote slot to grossly simplify what that agent looks grossly simplify what that agent looks grossly simplify what that agent looks like. I'm going to say there's two like. I'm going to say there's two like. I'm going to say there's two pieces. The first one is let's call it pieces. The first one is let's call it pieces. The first one is let's call it the business logic. Some version of the business logic. Some version of the business logic. Some version of interpreting intent and plan act and we interpreting intent and plan act and we interpreting intent and plan act and we loop around that. It's what your agent loop around that. It's what your agent loop around that. It's what your agent does. And we know that when an agent does. And we know that when an agent does. And we know that when an agent act, it doesn't always operate on data. act, it doesn't always operate on data. act, it doesn't always operate on data. But we equally know that in order for But we equally know that in order for But we equally know that in order for agents to be successful a huge part of agents to be successful a huge part of agents to be successful a huge part of that is giving it access to the right that is giving it access to the right that is giving it access to the right data at the right time. So the second data at the right time. So the second data at the right time. So the second big bucket is let's call it the data big bucket is let's call it the data big bucket is let's call it the data sources need to identify figure out okay sources need to identify figure out okay sources need to identify figure out okay in order to solve my problem I need in order to solve my problem I need in order to solve my problem I need access to these few things and wire them access to these few things and wire them access to these few things and wire them up and make them available to the agent.

up and make them available to the agent. up and make them available to the agent. In the example of our account opening In the example of our account opening In the example of our account opening agent maybe we can imagine that we need agent maybe we can imagine that we need agent maybe we can imagine that we need to be able to validate identity and so to be able to validate identity and so to be able to validate identity and so we might look at two data sources for we might look at two data sources for we might look at two data sources for that. the Department of Motor Vehicles, that. the Department of Motor Vehicles, that. the Department of Motor Vehicles, the DMV registry, and maybe some kind of the DMV registry, and maybe some kind of the DMV registry, and maybe some kind of passport verification service. So, we passport verification service. So, we passport verification service. So, we wire that up into our agent and it wire that up into our agent and it wire that up into our agent and it works. It's great. It's fantastic. And works. It's great. It's fantastic. And works. It's great. It's fantastic. And at the same time, you and other teams in at the same time, you and other teams in at the same time, you and other teams in your organization are building other your organization are building other your organization are building other agents and conceptually they look very agents and conceptually they look very agents and conceptually they look very similar. So, that's great. It's similar. So, that's great. It's similar. So, that's great. It's fantastic. It works. But it has a few fantastic. It works. But it has a few fantastic. It works. But it has a few problems. So first of all, every single problems. So first of all, every single problems. So first of all, every single time a team has to build an agent, they time a team has to build an agent, they time a team has to build an agent, they have to figure out from scratch where have to figure out from scratch where have to figure out from scratch where the data that they require for that the data that they require for that the data that they require for that agent to operate, where it sits, which agent to operate, where it sits, which agent to operate, where it sits, which if you work at a startup and you have if you work at a startup and you have if you work at a startup and you have one application, it sits on top of one one application, it sits on top of one one application, it sits on top of one Postgress database. That's not hard. The Postgress database. That's not hard. The Postgress database. That's not hard. The data is in that Postgress database. But data is in that Postgress database. But data is in that Postgress database. But in an enterprise ecosystem, you don't in an enterprise ecosystem, you don't in an enterprise ecosystem, you don't have one database. You have a hundred have one database. You have a hundred have one database. You have a hundred databases and you have snowflake and databases and you have snowflake and databases and you have snowflake and data bricks probably and you have S3 data bricks probably and you have S3 data bricks probably and you have S3 buckets and so on and so forth. You have buckets and so on and so forth. You have buckets and so on and so forth. You have to do that work manually from scratch to do that work manually from scratch to do that work manually from scratch every single time. And then when you've every single time. And then when you've every single time. And then when you've found the data sources, you know, in an found the data sources, you know, in an found the data sources, you know, in an enterprise there's lots of duplication enterprise there's lots of duplication enterprise there's lots of duplication of data. So then you need to figure out of data. So then you need to figure out of data. So then you need to figure out like is this the right data? Is it the like is this the right data? Is it the like is this the right data? Is it the right version? Can I trust it? Am I right version? Can I trust it? Am I right version? Can I trust it? Am I allowed to access it? So on and so allowed to access it? So on and so allowed to access it? So on and so forth. It also violates one of the core forth. It also violates one of the core forth. It also violates one of the core principles of software engineering, the principles of software engineering, the principles of software engineering, the dry principle. Don't repeat yourself. So dry principle. Don't repeat yourself. So dry principle. Don't repeat yourself. So when something change that cascades when something change that cascades when something change that cascades across all of your agents, you have to across all of your agents, you have to across all of your agents, you have to kind of manually rewire all of them. all

kind of manually rewire all of them. all kind of manually rewire all of them. all the time, which works, but it's just a the time, which works, but it's just a the time, which works, but it's just a lot of work. And then finally, there's lot of work. And then finally, there's lot of work. And then finally, there's no learning around the data sources and no learning around the data sources and no learning around the data sources and how your agents operate on them. So when how your agents operate on them. So when how your agents operate on them. So when your agent wake up wakes up tomorrow, your agent wake up wakes up tomorrow, your agent wake up wakes up tomorrow, it's not smarter than it was today. And it's not smarter than it was today. And it's not smarter than it was today. And there certainly isn't any cross agent there certainly isn't any cross agent there certainly isn't any cross agent learning because all of that wiring learning because all of that wiring learning because all of that wiring between business intent and the data between business intent and the data between business intent and the data sources is encoded in a combination of sources is encoded in a combination of sources is encoded in a combination of code and prompts. code and prompts. code and prompts. So I know what you're all thinking. work So I know what you're all thinking. work So I know what you're all thinking. work on files skills to the rescue and yes on files skills to the rescue and yes on files skills to the rescue and yes and no. Um you can come talk to me and no. Um you can come talk to me and no. Um you can come talk to me afterwards for kind of the full version afterwards for kind of the full version afterwards for kind of the full version of this but we've seen a ton of team of this but we've seen a ton of team of this but we've seen a ton of team that try to solve this problem using that try to solve this problem using that try to solve this problem using just markdown files and the summary is just markdown files and the summary is just markdown files and the summary is it is part of the solution but it is not it is part of the solution but it is not it is part of the solution but it is not the solution. Uh but don't take it from the solution. Uh but don't take it from the solution. Uh but don't take it from me, take it from Switz. A week ago on me, take it from Switz. A week ago on me, take it from Switz. A week ago on the latent space podcast, he said, "Hey the latent space podcast, he said, "Hey the latent space podcast, he said, "Hey guys, you got to learn your databases. guys, you got to learn your databases. guys, you got to learn your databases. You cannot vibe code with just markdown You cannot vibe code with just markdown You cannot vibe code with just markdown files." files." files." So we've been solving this problem at So we've been solving this problem at So we've been solving this problem at scale for some really massive scale for some really massive scale for some really massive organizations recently, including a organizations recently, including a organizations recently, including a Fortune 20 global bank, a massive tech Fortune 20 global bank, a massive tech Fortune 20 global bank, a massive tech platform company based here in the Bay platform company based here in the Bay platform company based here in the Bay Area, and a leading fintech company. And Area, and a leading fintech company. And Area, and a leading fintech company. And the pattern that is emerging is that in the pattern that is emerging is that in the pattern that is emerging is that in order to do agents at scale, we need order to do agents at scale, we need order to do agents at scale, we need thin agents on a smarter shared thin agents on a smarter shared thin agents on a smarter shared substrate. Thin agents on a smarter substrate. Thin agents on a smarter substrate. Thin agents on a smarter shared substrate. And what does that shared substrate. And what does that shared substrate. And what does that look like in practice? There are three

look like in practice? There are three look like in practice? There are three pillars to that. The first pillar is a pillars to that. The first pillar is a pillars to that. The first pillar is a businessfacing ontology. And the word businessfacing ontology. And the word businessfacing ontology. And the word ontology like I grew up in this world. ontology like I grew up in this world. ontology like I grew up in this world. People talked about ontologies forever. People talked about ontologies forever. People talked about ontologies forever. More recently, it's become very hype. More recently, it's become very hype. More recently, it's become very hype. probably thanks to Palunteer but also probably thanks to Palunteer but also probably thanks to Palunteer but also the rise of AI and there's a lot of the rise of AI and there's a lot of the rise of AI and there's a lot of people that want to make ontologies people that want to make ontologies people that want to make ontologies really complex but the core concepts are really complex but the core concepts are really complex but the core concepts are actually super simple what are the key actually super simple what are the key actually super simple what are the key concepts in your organization in our concepts in your organization in our concepts in your organization in our banking example customers accounts um banking example customers accounts um banking example customers accounts um debit cards checks transactions and how debit cards checks transactions and how debit cards checks transactions and how do they all relate but very importantly do they all relate but very importantly do they all relate but very importantly they are expressed in a way that makes they are expressed in a way that makes they are expressed in a way that makes sense to all the human beings working in sense to all the human beings working in sense to all the human beings working in your universe right all the people your universe right all the people your universe right all the people working in your company it's expressed working in your company it's expressed working in your company it's expressed in that name in that way in other words in that name in that way in other words in that name in that way in other words you don't say f_name you don't say f_name you don't say f_name no you have a customer and they have a no you have a customer and they have a no you have a customer and they have a first name so that's the first a first name so that's the first a first name so that's the first a businessf facing ontology the second businessf facing ontology the second businessf facing ontology the second pillar is a technical ontology this is pillar is a technical ontology this is pillar is a technical ontology this is all the metadata of all the data sources all the metadata of all the data sources all the metadata of all the data sources and data assets in your enterprise and data assets in your enterprise and data assets in your enterprise ecosystem I have 14 Oracle databases I ecosystem I have 14 Oracle databases I ecosystem I have 14 Oracle databases I have 15 neo forj databases I have have 15 neo forj databases I have have 15 neo forj databases I have snowflake and data bricks and I have s3 snowflake and data bricks and I have s3 snowflake and data bricks and I have s3 buckets and all of that kind of stuff.

buckets and all of that kind of stuff. buckets and all of that kind of stuff. Where do they sit? What are the schemas? Where do they sit? What are the schemas? Where do they sit? What are the schemas? All of that kind of good stuff. You com All of that kind of good stuff. You com All of that kind of good stuff. You com you construct that technical ontology in you construct that technical ontology in you construct that technical ontology in three key ways that we can talk about three key ways that we can talk about three key ways that we can talk about later though not in this in this talk. later though not in this in this talk. later though not in this in this talk. And then you have a mapping between the And then you have a mapping between the And then you have a mapping between the two. So that customer that has a first two. So that customer that has a first two. So that customer that has a first name that first name has a system of name that first name has a system of name that first name has a system of record and over there there's an Oracle record and over there there's an Oracle record and over there there's an Oracle database with a column called F_name the database with a column called F_name the database with a column called F_name the mapping between the two. And then the mapping between the two. And then the mapping between the two. And then the third pillar is the runtime signals out third pillar is the runtime signals out third pillar is the runtime signals out of your agents. When they walk this of your agents. When they walk this of your agents. When they walk this graph and they execute, they leave the graph and they execute, they leave the graph and they execute, they leave the traces around what have I tried? Was I traces around what have I tried? Was I traces around what have I tried? Was I successful? What was the outcome? The successful? What was the outcome? The successful? What was the outcome? The execution traces those three pillars. execution traces those three pillars. execution traces those three pillars. Okay. So let's look at that in the Okay. So let's look at that in the Okay. So let's look at that in the context of our bank account opening context of our bank account opening context of our bank account opening agent. This is a simplified view, but agent. This is a simplified view, but agent. This is a simplified view, but you can see this graph here. It has a you can see this graph here. It has a you can see this graph here. It has a combination of business concept like combination of business concept like combination of business concept like checks and accounts and credit history checks and accounts and credit history checks and accounts and credit history and stuff like that. This is a process and stuff like that. This is a process and stuff like that. This is a process following agent or a process guided following agent or a process guided following agent or a process guided agent. We want this type of agent to agent. We want this type of agent to agent. We want this type of agent to actually follow a process. So we've also actually follow a process. So we've also actually follow a process. So we've also encoded that in the ontology, a business encoded that in the ontology, a business encoded that in the ontology, a business process. And then if you look at the process. And then if you look at the process. And then if you look at the node that is surrounded by green the node that is surrounded by green the node that is surrounded by green the check compliance one we flip to the check compliance one we flip to the check compliance one we flip to the technical ontology and we've put in the technical ontology and we've put in the technical ontology and we've put in the graph here we've discovered and encoded graph here we've discovered and encoded graph here we've discovered and encoded that in order to do a compliance check that in order to do a compliance check that in order to do a compliance check you might imagine that you need to you might imagine that you need to you might imagine that you need to resolve a governmentissued ID and then resolve a governmentissued ID and then resolve a governmentissued ID and then we say that in this particular we say that in this particular we say that in this particular organization there are two data sources organization there are two data sources organization there are two data sources that can help us with that. It's the that can help us with that. It's the that can help us with that. It's the motor vehicle records and the passport

motor vehicle records and the passport motor vehicle records and the passport verification one. Okay so that's really verification one. Okay so that's really verification one. Okay so that's really great. So then when our agents come in great. So then when our agents come in great. So then when our agents come in here and they realize I'm going to check here and they realize I'm going to check here and they realize I'm going to check compliance, I need a governmentissued compliance, I need a governmentissued compliance, I need a governmentissued ID. Here are the two ways that I can ID. Here are the two ways that I can ID. Here are the two ways that I can resolve that. When they execute and they resolve that. When they execute and they resolve that. When they execute and they try that, they leave the third pillar, try that, they leave the third pillar, try that, they leave the third pillar, the execution traces for that. And the execution traces for that. And the execution traces for that. And they're more sophisticated than what's they're more sophisticated than what's they're more sophisticated than what's on this simplified slide, but involves on this simplified slide, but involves on this simplified slide, but involves things like, okay, where was I? What did things like, okay, where was I? What did things like, okay, where was I? What did I do? What is my context? And was I I do? What is my context? And was I I do? What is my context? And was I successful? And ultimately, it leads out successful? And ultimately, it leads out successful? And ultimately, it leads out to some kind of a score. and you use to some kind of a score. and you use to some kind of a score. and you use that as input. It's like, okay, I've that as input. It's like, okay, I've that as input. It's like, okay, I've been very successful using the DMV been very successful using the DMV been very successful using the DMV lookup, for example, then I'm more lookup, for example, then I'm more lookup, for example, then I'm more likely to choose one if I'm in the right likely to choose one if I'm in the right likely to choose one if I'm in the right context in my next invocation. context in my next invocation. context in my next invocation. Three pillars of the ontology based Three pillars of the ontology based Three pillars of the ontology based semantic layer, a business ontology, a semantic layer, a business ontology, a semantic layer, a business ontology, a technical ontology, the execution traces technical ontology, the execution traces technical ontology, the execution traces taken together, they solve all four of taken together, they solve all four of taken together, they solve all four of the problems. We now have a very easy the problems. We now have a very easy the problems. We now have a very easy way to discover the data sources. We way to discover the data sources. We way to discover the data sources. We know if they're trustworthy or not. We know if they're trustworthy or not. We know if they're trustworthy or not. We know that top down by some kind of human know that top down by some kind of human know that top down by some kind of human curated knowledge, right? An curated knowledge, right? An curated knowledge, right? An administrator of some sort saying it. We administrator of some sort saying it. We administrator of some sort saying it. We also know it bottom up through the also know it bottom up through the also know it bottom up through the execution traces. This is what actually execution traces. This is what actually execution traces. This is what actually worked in reality in practice. We have a worked in reality in practice. We have a worked in reality in practice. We have a single governed place that maps business single governed place that maps business single governed place that maps business intent and the concepts to those data intent and the concepts to those data intent and the concepts to those data sources so we don't repeat ourselves. If sources so we don't repeat ourselves. If sources so we don't repeat ourselves. If something changes that cascades across something changes that cascades across something changes that cascades across all my agents, right? And we have all my agents, right? And we have all my agents, right? And we have self-arning. So my agent that wakes up self-arning. So my agent that wakes up self-arning. So my agent that wakes up tomorrow is slightly smarter than it was tomorrow is slightly smarter than it was tomorrow is slightly smarter than it was today. And not just self-learning on an

today. And not just self-learning on an today. And not just self-learning on an individual agent, but across agents as individual agent, but across agents as individual agent, but across agents as well. well. well. So we're moving from this world, a world So we're moving from this world, a world So we're moving from this world, a world of thick agents with manually wired data of thick agents with manually wired data of thick agents with manually wired data sources into this world where we have sources into this world where we have sources into this world where we have thin agents on a smarter shared ontology thin agents on a smarter shared ontology thin agents on a smarter shared ontology based semantic layer. And this allows us based semantic layer. And this allows us based semantic layer. And this allows us to do a ton more agents without having to do a ton more agents without having to do a ton more agents without having to re-engineer them every time. Thin to re-engineer them every time. Thin to re-engineer them every time. Thin agents agents agents on top of a smarter shared substrate. If on top of a smarter shared substrate. If on top of a smarter shared substrate. If you think this is interesting, there's a you think this is interesting, there's a you think this is interesting, there's a documentation, a web page that outlines documentation, a web page that outlines documentation, a web page that outlines more information about this. If you see more information about this. If you see more information about this. If you see the QR code here, you can also come and the QR code here, you can also come and the QR code here, you can also come and talk to us at the booth. We have a big talk to us at the booth. We have a big talk to us at the booth. We have a big booth here at the expo P3. We love booth here at the expo P3. We love booth here at the expo P3. We love talking about this this kind of stuff. talking about this this kind of stuff. talking about this this kind of stuff. But not just that, this is one pattern, But not just that, this is one pattern, But not just that, this is one pattern, a very exciting pattern that we see a a very exciting pattern that we see a a very exciting pattern that we see a lot of traction around right now for lot of traction around right now for lot of traction around right now for using graphs in AI. But there's hundreds using graphs in AI. But there's hundreds using graphs in AI. But there's hundreds of more interesting patterns that of more interesting patterns that of more interesting patterns that combines graphs and AI. 10 of them is combines graphs and AI. 10 of them is combines graphs and AI. 10 of them is actually in the graph track that is actually in the graph track that is actually in the graph track that is kicking off right now in room 2005. And kicking off right now in room 2005. And kicking off right now in room 2005. And you have some really amazing talks from you have some really amazing talks from you have some really amazing talks from organizations like the Gates Foundation, organizations like the Gates Foundation, organizations like the Gates Foundation, Monday.com, JP Morgan Chase, Berkeley, Monday.com, JP Morgan Chase, Berkeley, Monday.com, JP Morgan Chase, Berkeley, New York Times, and so on and so forth.

New York Times, and so on and so forth. New York Times, and so on and so forth. So go check out that thing. And then So go check out that thing. And then So go check out that thing. And then finally, this was primarily center finally, this was primarily center finally, this was primarily center around organizations where you deal with around organizations where you deal with around organizations where you deal with many data sources and many agents. But many data sources and many agents. But many data sources and many agents. But if you're a startup building on Neo if you're a startup building on Neo if you're a startup building on Neo forj, love you. There is a startup forj, love you. There is a startup forj, love you. There is a startup program for Neo Forj that is phenomenal. program for Neo Forj that is phenomenal. program for Neo Forj that is phenomenal. You get access to free credit, but more You get access to free credit, but more You get access to free credit, but more importantly, we've built up a dedicated importantly, we've built up a dedicated importantly, we've built up a dedicated solution engineering team that spend solution engineering team that spend solution engineering team that spend every day working with startups for every day working with startups for every day working with startups for free, helping them model their data in free, helping them model their data in free, helping them model their data in Neo Forj, tune it for performance, and Neo Forj, tune it for performance, and Neo Forj, tune it for performance, and so on and so forth. So, please sign up so on and so forth. So, please sign up so on and so forth. So, please sign up for our startup program. Thank you very for our startup program. Thank you very for our startup program. Thank you very much. Enjoy the conference. Have a good much. Enjoy the conference. Have a good much. Enjoy the conference. Have a good day, everyone.