Why Your Enterprise Tech Stack Isn’t Ready for AI Agents — Christopher Lovejoy & Saul Howard

>> Okay. >> Okay. Hello everybody. My name is Chris Hello everybody. My name is Chris Hello everybody. My name is Chris Lovejoy and I'm a member of technical Lovejoy and I'm a member of technical Lovejoy and I'm a member of technical staff at Anterior staff at Anterior staff at Anterior and I work as a forward deployed and I work as a forward deployed and I work as a forward deployed engineer. So I embed within enterprise engineer. So I embed within enterprise engineer. So I embed within enterprise organizations and help them get value organizations and help them get value organizations and help them get value from using AI agents. from using AI agents. from using AI agents. And I previously worked at Anterior with And I previously worked at Anterior with And I previously worked at Anterior with Saul. Saul. Saul. >> Hi everybody. I'm Saul. I'm VP of >> Hi everybody. I'm Saul. I'm VP of >> Hi everybody. I'm Saul. I'm VP of engineering at Anterior. We're a New engineering at Anterior. We're a New engineering at Anterior. We're a New York based company selling York based company selling York based company selling AI uh AI uh AI uh agentic AI to US health insurance agentic AI to US health insurance agentic AI to US health insurance companies. companies. companies. Um Um Um Chris and I have spent a lot of time Chris and I have spent a lot of time Chris and I have spent a lot of time building in enterprise and in health building in enterprise and in health building in enterprise and in health care care care enterprises particularly. And health enterprises particularly. And health enterprises particularly. And health care is a very challenging place to care is a very challenging place to care is a very challenging place to develop and deploy AI. develop and deploy AI. develop and deploy AI. Uh health [clears throat] care is so Uh health [clears throat] care is so Uh health [clears throat] care is so challenging because of the challenging because of the challenging because of the requirements around process and uh requirements around process and uh requirements around process and uh compliance, the regulatory requirements compliance, the regulatory requirements compliance, the regulatory requirements that that that are so important. that that that are so important. that that that are so important. Also because of the uh direct real Also because of the uh direct real Also because of the uh direct real impact that your work has on people's impact that your work has on people's impact that your work has on people's lives, which is of course also what lives, which is of course also what lives, which is of course also what makes it so rewarding.

makes it so rewarding. makes it so rewarding. Um I think a lot of the learnings you Um I think a lot of the learnings you Um I think a lot of the learnings you can take from working in enterprise for can take from working in enterprise for can take from working in enterprise for health care, you can take to enterprise health care, you can take to enterprise health care, you can take to enterprise in other regulated industries. Like in other regulated industries. Like in other regulated industries. Like finance, defense, government work, finance, defense, government work, finance, defense, government work, anywhere where process is so important anywhere where process is so important anywhere where process is so important and has to be followed. and has to be followed. and has to be followed. >> In this talk we're going to talk about >> In this talk we're going to talk about >> In this talk we're going to talk about um some of the learnings that we've had um some of the learnings that we've had um some of the learnings that we've had and specifically we're going to talk and specifically we're going to talk and specifically we're going to talk about why enterprise tech stacks aren't about why enterprise tech stacks aren't about why enterprise tech stacks aren't ready for AI agents and some of the ready for AI agents and some of the ready for AI agents and some of the primitives that we've built in the past primitives that we've built in the past primitives that we've built in the past in order to unlock them. in order to unlock them. in order to unlock them. And to make this concrete, let's start And to make this concrete, let's start And to make this concrete, let's start by considering a scenario that might be by considering a scenario that might be by considering a scenario that might be familiar to many of you, which is the familiar to many of you, which is the familiar to many of you, which is the the enterprise proof of proof of the enterprise proof of proof of the enterprise proof of proof of concept, the enterprise POC. concept, the enterprise POC. concept, the enterprise POC. And let's say we have identified a And let's say we have identified a And let's say we have identified a customer that we want to serve, and customer that we want to serve, and customer that we want to serve, and we've identified a priority use case we've identified a priority use case we've identified a priority use case with them. So, obviously we're on the with them. So, obviously we're on the with them. So, obviously we're on the healthcare track here. Let's consider a healthcare track here. Let's consider a healthcare track here. Let's consider a um a large health system and a use case um a large health system and a use case um a large health system and a use case that is some sort of administrative that is some sort of administrative that is some sort of administrative healthcare workflow. healthcare workflow. healthcare workflow. So, you work with them, you scope out a So, you work with them, you scope out a So, you work with them, you scope out a POC, you define the metrics that you are POC, you define the metrics that you are POC, you define the metrics that you are going to care about, you're going to going to care about, you're going to going to care about, you're going to bench benchmark yourselves on.

bench benchmark yourselves on. bench benchmark yourselves on. Um you allocate two engineers, you spend Um you allocate two engineers, you spend Um you allocate two engineers, you spend 4 weeks building it, 4 weeks building it, 4 weeks building it, and um and um and um the actual build-out might look a little the actual build-out might look a little the actual build-out might look a little bit something like this. bit something like this. bit something like this. >> An enterprise stack is very complicated. >> An enterprise stack is very complicated. >> An enterprise stack is very complicated. It's much, much more than we're showing It's much, much more than we're showing It's much, much more than we're showing here, but generally you can have an here, but generally you can have an here, but generally you can have an application layer, a control plane application layer, a control plane application layer, a control plane layer, the data plane. layer, the data plane. layer, the data plane. For your POC, you're going to need some For your POC, you're going to need some For your POC, you're going to need some access to the model provider as well. access to the model provider as well. access to the model provider as well. And your POC is going to need access to And your POC is going to need access to And your POC is going to need access to data across all of these different data across all of these different data across all of these different planes. It may be some in the data lake, planes. It may be some in the data lake, planes. It may be some in the data lake, some directly from the application some directly from the application some directly from the application layer, for example. And so, you're going layer, for example. And so, you're going layer, for example. And so, you're going to deploy it something like this. It's to deploy it something like this. It's to deploy it something like this. It's going to connect to all these different going to connect to all these different going to connect to all these different places. There's going to be some offline places. There's going to be some offline places. There's going to be some offline data pulling. There's going to be some data pulling. There's going to be some data pulling. There's going to be some maybe some online. maybe some online. maybe some online. Generally, you'll get access to the data Generally, you'll get access to the data Generally, you'll get access to the data and push towards the results. >> And so, things go well. You you get >> And so, things go well. You you get great results. The the AI performs you great results. The the AI performs you great results. The the AI performs you know, as you expected. You you hit the know, as you expected. You you hit the know, as you expected. You you hit the performance metrics. Um you know, it's performance metrics. Um you know, it's performance metrics. Um you know, it's fast. It's relatively cheap. And you fast. It's relatively cheap. And you fast. It's relatively cheap. And you hold a meeting, you present this to the hold a meeting, you present this to the hold a meeting, you present this to the relevant stakeholders, and everyone relevant stakeholders, and everyone relevant stakeholders, and everyone seems pretty happy. Um so, you know, seems pretty happy. Um so, you know, seems pretty happy. Um so, you know, your chief of finance um in the in the your chief of finance um in the in the your chief of finance um in the in the company is very excited and wants to company is very excited and wants to company is very excited and wants to understand what's going to be the impact understand what's going to be the impact understand what's going to be the impact on the budget for next year. Uh your on the budget for next year. Uh your on the budget for next year. Uh your chief medical officer is excited to tell chief medical officer is excited to tell chief medical officer is excited to tell his his colleagues, you know, how his his colleagues, you know, how his his colleagues, you know, how accurate his AI is. Um and the head of accurate his AI is. Um and the head of accurate his AI is. Um and the head of sales asks, "Okay, when can we put sales asks, "Okay, when can we put sales asks, "Okay, when can we put powered by AI? When can we put that on powered by AI? When can we put that on powered by AI? When can we put that on the websites?" Um but the problem is the websites?" Um but the problem is the websites?" Um but the problem is that everyone here is assuming that the that everyone here is assuming that the that everyone here is assuming that the the hard part is done, that the AI was the hard part is done, that the AI was the hard part is done, that the AI was was the challenging part. But actually, was the challenging part. But actually, was the challenging part. But actually, as we know, often getting things into

as we know, often getting things into as we know, often getting things into production is really where the challenge production is really where the challenge production is really where the challenge lies. lies. lies. Um and to get a bit more specific on Um and to get a bit more specific on Um and to get a bit more specific on what that challenge looks like, what that challenge looks like, what that challenge looks like, um um um you hold a meeting the next day, you you hold a meeting the next day, you you hold a meeting the next day, you bring in the relevant stakeholders to bring in the relevant stakeholders to bring in the relevant stakeholders to discuss productionizing this proof of discuss productionizing this proof of discuss productionizing this proof of concept application. concept application. concept application. And somebody raises their hand and says, And somebody raises their hand and says, And somebody raises their hand and says, "Um can I see the the audit trail for "Um can I see the the audit trail for "Um can I see the the audit trail for this? Like for us, for compliance, it's this? Like for us, for compliance, it's this? Like for us, for compliance, it's critical that we can see every step, critical that we can see every step, critical that we can see every step, every action that the agent takes, every every action that the agent takes, every every action that the agent takes, every piece of data that it accesses. Can can piece of data that it accesses. Can can piece of data that it accesses. Can can you give that to me?" And you realize you give that to me?" And you realize you give that to me?" And you realize actually, you know, with the way things actually, you know, with the way things actually, you know, with the way things have been implemented in the initial have been implemented in the initial have been implemented in the initial POC, without these um kind of true POC, without these um kind of true POC, without these um kind of true integrations, integrations, integrations, that actually that's going to be quite that actually that's going to be quite that actually that's going to be quite challenging. challenging. challenging. And then somebody else um pops up with And then somebody else um pops up with And then somebody else um pops up with some other questions. So, somebody asks, some other questions. So, somebody asks, some other questions. So, somebody asks, "Okay, well actually, how is data "Okay, well actually, how is data "Okay, well actually, how is data sensitive data being handled here? Um sensitive data being handled here? Um sensitive data being handled here? Um how's that being passed to the agents? how's that being passed to the agents? how's that being passed to the agents? You know, we have a very strict boundary You know, we have a very strict boundary You know, we have a very strict boundary around where our data can go and where around where our data can go and where around where our data can go and where it can't go. Um is it is this respecting it can't go. Um is it is this respecting it can't go. Um is it is this respecting that? How does that look?" that? How does that look?" that? How does that look?" And then your chief medical officer And then your chief medical officer And then your chief medical officer says, again, "Who's approving the says, again, "Who's approving the says, again, "Who's approving the decisions here? Because we know in decisions here? Because we know in decisions here? Because we know in certain scenarios, we have to escalate certain scenarios, we have to escalate certain scenarios, we have to escalate to a clinician who will then uh you to a clinician who will then uh you to a clinician who will then uh you know, approve or or or not agree with know, approve or or or not agree with know, approve or or or not agree with what the agent is saying. Um so, how what the agent is saying. Um so, how what the agent is saying. Um so, how does that happen? Like what's the does that happen? Like what's the does that happen? Like what's the mechanism for that?"

mechanism for that?" mechanism for that?" And over the course of the meetings, you And over the course of the meetings, you And over the course of the meetings, you know, you can imagine you get more and know, you can imagine you get more and know, you can imagine you get more and more questions. So, can untrusted data more questions. So, can untrusted data more questions. So, can untrusted data manipulate the model? How do we know manipulate the model? How do we know manipulate the model? How do we know that the agent continues to perform that the agent continues to perform that the agent continues to perform well? How do we deal with integrations? well? How do we deal with integrations? well? How do we deal with integrations? How do we connect to Epic, to How do we connect to Epic, to How do we connect to Epic, to Salesforce, to the other kind of Salesforce, to the other kind of Salesforce, to the other kind of applications that we care about? applications that we care about? applications that we care about? And for the purposes of this talk, we're And for the purposes of this talk, we're And for the purposes of this talk, we're going to focus on these four, the going to focus on these four, the going to focus on these four, the highlighted ones. For the other two, highlighted ones. For the other two, highlighted ones. For the other two, feel free to come and chat to me and feel free to come and chat to me and feel free to come and chat to me and talk about these later. We're very happy talk about these later. We're very happy talk about these later. We're very happy to talk, but um just in the interest of to talk, but um just in the interest of to talk, but um just in the interest of time, we'll stay focused. And let's time, we'll stay focused. And let's time, we'll stay focused. And let's start with uh this one about the audit start with uh this one about the audit start with uh this one about the audit trail. trail. trail. >> So, this is a question you're guaranteed >> So, this is a question you're guaranteed >> So, this is a question you're guaranteed to get from the security team. They're to get from the security team. They're to get from the security team. They're going to want to see an audit trail. going to want to see an audit trail. going to want to see an audit trail. And for programmers, an audit trail And for programmers, an audit trail And for programmers, an audit trail sounds very like a typical developer log sounds very like a typical developer log sounds very like a typical developer log you might have in DataDog. Surely it's you might have in DataDog. Surely it's you might have in DataDog. Surely it's it's it's a similar kind of thing. But it's it's a similar kind of thing. But it's it's a similar kind of thing. But for security frameworks that that exist for security frameworks that that exist for security frameworks that that exist in the real enterprise world, like SOC in the real enterprise world, like SOC in the real enterprise world, like SOC 2, HITRUST, HIPAA, 2, HITRUST, HIPAA, 2, HITRUST, HIPAA, an audit trail is is a bit more than an audit trail is is a bit more than an audit trail is is a bit more than that. It it has to contain a complete that. It it has to contain a complete that. It it has to contain a complete record of absolutely every action that record of absolutely every action that record of absolutely every action that the agent took. It has to contain the agent took. It has to contain the agent took. It has to contain all of the places where the agent all of the places where the agent all of the places where the agent accessed data, all of the authorization accessed data, all of the authorization accessed data, all of the authorization by which the agent did something.

by which the agent did something. by which the agent did something. It's it's this complete record in in a It's it's this complete record in in a It's it's this complete record in in a much more fundamental way. much more fundamental way. much more fundamental way. And uh you one way of thinking about it And uh you one way of thinking about it And uh you one way of thinking about it is in a legal sense. is in a legal sense. is in a legal sense. It say our agent's decisions came up in It say our agent's decisions came up in It say our agent's decisions came up in a court of law. a court of law. a court of law. Could we show a justifiable chain of Could we show a justifiable chain of Could we show a justifiable chain of evidence for why the particular actions evidence for why the particular actions evidence for why the particular actions were taken by a decision? And that's were taken by a decision? And that's were taken by a decision? And that's something that could easily happen something that could easily happen something that could easily happen within the health care context, for within the health care context, for within the health care context, for example. example. example. When I think about When I think about When I think about architecting systems like this, I think architecting systems like this, I think architecting systems like this, I think often about what do I want to make easy? often about what do I want to make easy? often about what do I want to make easy? What are What are What are But when I'm choosing my constraints, But when I'm choosing my constraints, But when I'm choosing my constraints, I'm saying, "Okay, these are the things I'm saying, "Okay, these are the things I'm saying, "Okay, these are the things I want my system to be easy and let that I want my system to be easy and let that I want my system to be easy and let that drive the trade-offs drive the trade-offs drive the trade-offs that that I'm going to make." that that I'm going to make." that that I'm going to make." And a particular pattern that uh uh And a particular pattern that uh uh And a particular pattern that uh uh is used in lots of different industries, is used in lots of different industries, is used in lots of different industries, for example, in finance, is a for example, in finance, is a for example, in finance, is a transaction log. transaction log. transaction log. An immutable record of events that store An immutable record of events that store An immutable record of events that store all of the transactions that happen all of the transactions that happen all of the transactions that happen throughout the system.

throughout the system. throughout the system. And this is append-only timestamp log. And this is append-only timestamp log. And this is append-only timestamp log. It's complete. So, this is your source It's complete. So, this is your source It's complete. So, this is your source of truth for all of the data of the of truth for all of the data of the of truth for all of the data of the system. system. system. And it's unified. So, there's only one And it's unified. So, there's only one And it's unified. So, there's only one source of truth across all of the source of truth across all of the source of truth across all of the different agents that you might have different agents that you might have different agents that you might have running in parallel, for example. running in parallel, for example. running in parallel, for example. And architecting this way, the making And architecting this way, the making And architecting this way, the making this trade-off, this trade-off, this trade-off, uh uh uh means that auditability becomes trivial. means that auditability becomes trivial. means that auditability becomes trivial. It falls out of your data storage It falls out of your data storage It falls out of your data storage paradigm that you've chosen. It sort of paradigm that you've chosen. It sort of paradigm that you've chosen. It sort of is impossible not to be able to roll is impossible not to be able to roll is impossible not to be able to roll back time and and see exactly the state back time and and see exactly the state back time and and see exactly the state of the system at a a particular point in of the system at a a particular point in of the system at a a particular point in time and be able to uh provide that as time and be able to uh provide that as time and be able to uh provide that as an audit trail for what happened uh uh an audit trail for what happened uh uh an audit trail for what happened uh uh each point in time. each point in time. each point in time. And And And of course, these are trade-offs. So, of course, these are trade-offs. So, of course, these are trade-offs. So, what's the trade-off you're making here? what's the trade-off you're making here? what's the trade-off you're making here? I think we could say that for this kind I think we could say that for this kind I think we could say that for this kind of event logging or sometimes called of event logging or sometimes called of event logging or sometimes called event sourcing pattern, writes become event sourcing pattern, writes become event sourcing pattern, writes become very easy. So, you just drop an event. very easy. So, you just drop an event. very easy. So, you just drop an event. Reads become more difficult because you Reads become more difficult because you Reads become more difficult because you have to have to have to read through all of the events in order read through all of the events in order read through all of the events in order to reconstruct a view of what happened.

to reconstruct a view of what happened. to reconstruct a view of what happened. And there are patterns like caching and And there are patterns like caching and And there are patterns like caching and snapshots so you can bring to to to make snapshots so you can bring to to to make snapshots so you can bring to to to make that that simpler, but there always is that that simpler, but there always is that that simpler, but there always is more effort there. more effort there. more effort there. Although, I have seen in in the Although, I have seen in in the Although, I have seen in in the healthcare context that healthcare context that healthcare context that actually, you're going to want different actually, you're going to want different actually, you're going to want different interpretations of the raw data that interpretations of the raw data that interpretations of the raw data that your agents recorded after the fact. So, your agents recorded after the fact. So, your agents recorded after the fact. So, for example, it might be that for example, it might be that for example, it might be that more events happened and that changes more events happened and that changes more events happened and that changes the interpretation of the healthcare the interpretation of the healthcare the interpretation of the healthcare journey, and you want a different view journey, and you want a different view journey, and you want a different view of the the source of truth at that of the the source of truth at that of the the source of truth at that particular time. particular time. particular time. And this pattern makes that easy because And this pattern makes that easy because And this pattern makes that easy because all of your views of the data are all of your views of the data are all of your views of the data are ephemeral computed projections of the ephemeral computed projections of the ephemeral computed projections of the event log. event log. event log. Um Um Um okay. Next, okay. Next, okay. Next, the compliance officer comes and is the compliance officer comes and is the compliance officer comes and is asking, "How is the sensitive data asking, "How is the sensitive data asking, "How is the sensitive data passed around the system? What's the passed around the system? What's the passed around the system? What's the life cycle of data within our system?" life cycle of data within our system?" life cycle of data within our system?" And within a healthcare context, as we And within a healthcare context, as we And within a healthcare context, as we all know, data all know, data all know, data means a lot. It's PHI, protected or means a lot. It's PHI, protected or means a lot. It's PHI, protected or personal health information.

personal health information. personal health information. It's It's It's has legal restrictions around it. Not has legal restrictions around it. Not has legal restrictions around it. Not just HIPAA, but other legal restrictions just HIPAA, but other legal restrictions just HIPAA, but other legal restrictions about the use of people's data. You about the use of people's data. You about the use of people's data. You cannot have your agent, just as you cannot have your agent, just as you cannot have your agent, just as you cannot have humans, accessing and cannot have humans, accessing and cannot have humans, accessing and reading and utilizing healthcare data reading and utilizing healthcare data reading and utilizing healthcare data that they don't absolutely have a that they don't absolutely have a that they don't absolutely have a necessity to use at that that point in necessity to use at that that point in necessity to use at that that point in time for that particular time for that particular time for that particular journey. journey. journey. And so, And so, And so, again, architecturally, when I think again, architecturally, when I think again, architecturally, when I think about how am I storing data within a about how am I storing data within a about how am I storing data within a particular system, I would like to particular system, I would like to particular system, I would like to think, "What is the shape of the data, think, "What is the shape of the data, think, "What is the shape of the data, what kind of characteristics does the what kind of characteristics does the what kind of characteristics does the data have? data have? data have? For health care data, that might be that For health care data, that might be that For health care data, that might be that it's very complicated. It doesn't follow it's very complicated. It doesn't follow it's very complicated. It doesn't follow strict hierarchical um relationships. strict hierarchical um relationships. strict hierarchical um relationships. It's uh sometimes unstructured and it's It's uh sometimes unstructured and it's It's uh sometimes unstructured and it's sometimes structured. It could be very sometimes structured. It could be very sometimes structured. It could be very large. For example, health care data but large. For example, health care data but large. For example, health care data but one piece of health care data can easily one piece of health care data can easily one piece of health care data can easily be over a megabyte in size or or much be over a megabyte in size or or much be over a megabyte in size or or much more than that. more than that. more than that. Uh it has strict access controls. As Uh it has strict access controls. As Uh it has strict access controls. As we've been saying, the R back comes into we've been saying, the R back comes into we've been saying, the R back comes into play like uh both for humans and and play like uh both for humans and and play like uh both for humans and and then for agents downstream of that.

then for agents downstream of that. then for agents downstream of that. Uh Uh Uh it may even be we I've seen customers it may even be we I've seen customers it may even be we I've seen customers where they're not willing to have their where they're not willing to have their where they're not willing to have their health care data leave their own health care data leave their own health care data leave their own environment, leave their on prem VPC for environment, leave their on prem VPC for environment, leave their on prem VPC for example. So, we have tangential access example. So, we have tangential access example. So, we have tangential access to their to their data. to their to their data. to their to their data. And so, an architectural paradigm I And so, an architectural paradigm I And so, an architectural paradigm I might go to is object storage. might go to is object storage. might go to is object storage. Schema driven object storage, I think is Schema driven object storage, I think is Schema driven object storage, I think is a good fit for this. It's m- matches a good fit for this. It's m- matches a good fit for this. It's m- matches well with the choice of using event well with the choice of using event well with the choice of using event logging logging logging because you can separate the two. So, because you can separate the two. So, because you can separate the two. So, the events we talked about as the record the events we talked about as the record the events we talked about as the record of what the agent is doing at any of what the agent is doing at any of what the agent is doing at any particular time only contain references particular time only contain references particular time only contain references to the schema driven blobs that are the to the schema driven blobs that are the to the schema driven blobs that are the storage of the actual health care data storage of the actual health care data storage of the actual health care data itself. itself. itself. And uh it's important therefore that the And uh it's important therefore that the And uh it's important therefore that the health care data is stored immutably health care data is stored immutably health care data is stored immutably again so that you can always go back in again so that you can always go back in again so that you can always go back in time and reconstruct what data the agent time and reconstruct what data the agent time and reconstruct what data the agent had access to at that particular point had access to at that particular point had access to at that particular point in time. in time. in time. This separation of events for what This separation of events for what This separation of events for what happened and object storage for happened and object storage for happened and object storage for the data that was used at that the data that was used at that the data that was used at that particular point in time particular point in time particular point in time has actually some some very useful has actually some some very useful has actually some some very useful benefits. For example, with a system benefits. For example, with a system benefits. For example, with a system like this, it's possible for developers like this, it's possible for developers like this, it's possible for developers to go back and debug and have to go back and debug and have to go back and debug and have observability over what happened, what observability over what happened, what observability over what happened, what particular steps the agent took, why it particular steps the agent took, why it particular steps the agent took, why it did that, and and retrace the agent's did that, and and retrace the agent's did that, and and retrace the agent's steps steps steps without having access to the without having access to the without having access to the personal health information itself.

personal health information itself. personal health information itself. Although because of the schema driven, Although because of the schema driven, Although because of the schema driven, they can see the shape of that data, they can see the shape of that data, they can see the shape of that data, they they can't and they they can't and they they can't and to be honest, often won't be able to be to be honest, often won't be able to be to be honest, often won't be able to be given access to that health care data. given access to that health care data. given access to that health care data. So, you can separate out observability So, you can separate out observability So, you can separate out observability and orchestration and instrumentation and orchestration and instrumentation and orchestration and instrumentation from the health care data itself. from the health care data itself. from the health care data itself. And this then has another benefit, which And this then has another benefit, which And this then has another benefit, which is zero trust. is zero trust. is zero trust. It it the object storage becomes a place It it the object storage becomes a place It it the object storage becomes a place where you can apply zero trust where you can apply zero trust where you can apply zero trust principles. principles. principles. Your agents can bear tokens and use Your agents can bear tokens and use Your agents can bear tokens and use those tokens to access the data at the those tokens to access the data at the those tokens to access the data at the point of use and not allow data to flow point of use and not allow data to flow point of use and not allow data to flow around the system as it likes. around the system as it likes. around the system as it likes. This then leads into a mitigation for This then leads into a mitigation for This then leads into a mitigation for prompt injection for the lethal prompt injection for the lethal prompt injection for the lethal trifecta. trifecta. trifecta. The way I think about the lethal The way I think about the lethal The way I think about the lethal trifecta is trifecta is trifecta is can I solve for the constraint if I have can I solve for the constraint if I have can I solve for the constraint if I have an agent at point A with access to this an agent at point A with access to this an agent at point A with access to this data? Is it possible within my data? Is it possible within my data? Is it possible within my architecture for the agent to be also architecture for the agent to be also architecture for the agent to be also accessing data over here? accessing data over here? accessing data over here? And zero trust principles, tokens are And zero trust principles, tokens are And zero trust principles, tokens are bad by the agents and object storage bad by the agents and object storage bad by the agents and object storage segregated from the event stream that segregated from the event stream that segregated from the event stream that has your orchestration logic gives you a has your orchestration logic gives you a has your orchestration logic gives you a place to be able to solve for that place to be able to solve for that place to be able to solve for that constraint. It won't be possible for the constraint. It won't be possible for the constraint. It won't be possible for the agent to access data within the same agent to access data within the same agent to access data within the same process that that you've given it the process that that you've given it the process that that you've given it the the previous data.

>> Okay, so then it comes to how do you >> Okay, so then it comes to how do you handle escalation? And handle escalation? And handle escalation? And in many scenarios, you will want to be in many scenarios, you will want to be in many scenarios, you will want to be able to escalate the decision that an able to escalate the decision that an able to escalate the decision that an agent makes or an action that an agent agent makes or an action that an agent agent makes or an action that an agent makes to a human. makes to a human. makes to a human. But one of the challenges here is that But one of the challenges here is that But one of the challenges here is that this is quite dynamic. So, you don't this is quite dynamic. So, you don't this is quite dynamic. So, you don't know in advance when exactly perhaps the know in advance when exactly perhaps the know in advance when exactly perhaps the agent's going to escalate. It could be agent's going to escalate. It could be agent's going to escalate. It could be that you're asking the AI to escalate that you're asking the AI to escalate that you're asking the AI to escalate when it's not sure. when it's not sure. when it's not sure. Um it could be that you define some sort Um it could be that you define some sort Um it could be that you define some sort of rules in your system, maybe in a of rules in your system, maybe in a of rules in your system, maybe in a medical context, the treatments going medical context, the treatments going medical context, the treatments going above a certain threshold means that it above a certain threshold means that it above a certain threshold means that it needs to be escalated uh for an needs to be escalated uh for an needs to be escalated uh for an approval. approval. approval. But [snorts] this makes it very But [snorts] this makes it very But [snorts] this makes it very challenging because of this this challenging because of this this challenging because of this this inability to predict. And a second inability to predict. And a second inability to predict. And a second challenge is also that challenge is also that challenge is also that humans and LLMs ultimately process humans and LLMs ultimately process humans and LLMs ultimately process context differently. You know, LLMs will context differently. You know, LLMs will context differently. You know, LLMs will have no problem if you give them massive have no problem if you give them massive have no problem if you give them massive massive amounts of text, but humans massive amounts of text, but humans massive amounts of text, but humans that's not the case. So, what we've seen that's not the case. So, what we've seen that's not the case. So, what we've seen is that is that is that one pattern that can work very well here one pattern that can work very well here one pattern that can work very well here is is is if in your platform you enforce you kind if in your platform you enforce you kind if in your platform you enforce you kind of a wider definition of agent which of a wider definition of agent which of a wider definition of agent which encompasses both LLMs and humans, then encompasses both LLMs and humans, then encompasses both LLMs and humans, then you can make it such that any action you can make it such that any action you can make it such that any action that can be taken by an LLM could also that can be taken by an LLM could also that can be taken by an LLM could also be taken by a human.

be taken by a human. be taken by a human. And this is helpful because at any point And this is helpful because at any point And this is helpful because at any point in the kind of chain of actions that in the kind of chain of actions that in the kind of chain of actions that your agent is taking, it can escalate to your agent is taking, it can escalate to your agent is taking, it can escalate to a human, the human could perform that a human, the human could perform that a human, the human could perform that action, and then any step downstream action, and then any step downstream action, and then any step downstream doesn't care about whether it was a doesn't care about whether it was a doesn't care about whether it was a human or an LLM that did those actions human or an LLM that did those actions human or an LLM that did those actions upstream. upstream. upstream. Um Um Um and and and on the second point around the context, on the second point around the context, on the second point around the context, what this also what this also what this also makes much easier is that you can define makes much easier is that you can define makes much easier is that you can define methods that take the context, which has methods that take the context, which has methods that take the context, which has some kind of shared definition of some kind of shared definition of some kind of shared definition of context, which is irrespective of context, which is irrespective of context, which is irrespective of whether it's a human or an LLM that's whether it's a human or an LLM that's whether it's a human or an LLM that's going to be accessing it. going to be accessing it. going to be accessing it. And you can take those methods to then And you can take those methods to then And you can take those methods to then map into something that's agent map into something that's agent map into something that's agent friendly, like a prompt, or into friendly, like a prompt, or into friendly, like a prompt, or into something that's more human friendly, something that's more human friendly, something that's more human friendly, for example, a UI. And then on this And then on this fourth and final question that we're fourth and final question that we're fourth and final question that we're going to talk about, going to talk about, going to talk about, um evals. um evals. um evals. Obviously, you know, we hear a lot about Obviously, you know, we hear a lot about Obviously, you know, we hear a lot about evals. We know that evals can be very evals. We know that evals can be very evals. We know that evals can be very helpful, that often they drive helpful, that often they drive helpful, that often they drive decision-making about the types of model decision-making about the types of model decision-making about the types of model you want to use, the type of approach you want to use, the type of approach you want to use, the type of approach you might want to use you might want to use you might want to use within your product. But we also know within your product. But we also know within your product. But we also know that evals can be pretty hard, and that evals can be pretty hard, and that evals can be pretty hard, and there's various factors here. We there's various factors here. We there's various factors here. We [snorts] know that LLMs are not [snorts] know that LLMs are not [snorts] know that LLMs are not deterministic, so it can be quite tricky deterministic, so it can be quite tricky deterministic, so it can be quite tricky to pin down the precise change that led to pin down the precise change that led to pin down the precise change that led to some sort of change in outputs.

to some sort of change in outputs. to some sort of change in outputs. Um Um Um we also know that the data that you we also know that the data that you we also know that the data that you might put in an offline data set might might put in an offline data set might might put in an offline data set might not necessarily represent production not necessarily represent production not necessarily represent production data, and it could be that data, and it could be that data, and it could be that um maybe you sampled from data, but um maybe you sampled from data, but um maybe you sampled from data, but actually that sample isn't truly actually that sample isn't truly actually that sample isn't truly representative. And then you also have representative. And then you also have representative. And then you also have drift of data drift of data drift of data over time, so maybe your offline data over time, so maybe your offline data over time, so maybe your offline data set is now out of date. And what we found is that these three And what we found is that these three primitives that we've described primitives that we've described primitives that we've described described so far in the talk described so far in the talk described so far in the talk actually give you effective privacy actually give you effective privacy actually give you effective privacy preserving evals almost as a byproduct preserving evals almost as a byproduct preserving evals almost as a byproduct without needing to kind of bolt without needing to kind of bolt without needing to kind of bolt something onto the side of your something onto the side of your something onto the side of your architecture. architecture. architecture. So to make that more concrete, so the So to make that more concrete, so the So to make that more concrete, so the immutable ledger, what this means is immutable ledger, what this means is immutable ledger, what this means is that you can replay your actions. So you that you can replay your actions. So you that you can replay your actions. So you can go back to any particular time, you can go back to any particular time, you can go back to any particular time, you know, in this kind of sequence of know, in this kind of sequence of know, in this kind of sequence of events, you can see the complete state events, you can see the complete state events, you can see the complete state of the system at that point in time. And of the system at that point in time. And of the system at that point in time. And if you wanted to, you could then make if you wanted to, you could then make if you wanted to, you could then make very specific tweaks. So you could tweak very specific tweaks. So you could tweak very specific tweaks. So you could tweak a prompt, you could tweak a model, you a prompt, you could tweak a model, you a prompt, you could tweak a model, you could tweak the code, and you can see could tweak the code, and you can see could tweak the code, and you can see the exact direct impact of that because the exact direct impact of that because the exact direct impact of that because you have all of that context.

you have all of that context. you have all of that context. Secondly, you have this human agent Secondly, you have this human agent Secondly, you have this human agent equivalency, equivalency, equivalency, which means that for any task, you could which means that for any task, you could which means that for any task, you could get both the agent, the LLM agent, and get both the agent, the LLM agent, and get both the agent, the LLM agent, and the human to perform it, the human to perform it, the human to perform it, and your difference is your eval, that and your difference is your eval, that and your difference is your eval, that gives you the eval scores. gives you the eval scores. gives you the eval scores. And then finally, what the object And then finally, what the object And then finally, what the object storage enables you to do is to actually storage enables you to do is to actually storage enables you to do is to actually run these evals on production data run these evals on production data run these evals on production data including inside your customer's including inside your customer's including inside your customer's environment without actually ever environment without actually ever environment without actually ever exposing that data. You can get your exposing that data. You can get your exposing that data. You can get your eval results without the sensitive data eval results without the sensitive data eval results without the sensitive data ever needing to come to where your agent ever needing to come to where your agent ever needing to come to where your agent is performing the work. >> Right, so >> Right, so we've gone through four architectural we've gone through four architectural we've gone through four architectural principles that we found useful for principles that we found useful for principles that we found useful for building in healthcare and more building in healthcare and more building in healthcare and more generally in regulated environments for generally in regulated environments for generally in regulated environments for enterprise. The immutable ledger of enterprise. The immutable ledger of enterprise. The immutable ledger of actions, the orchestration adjacent actions, the orchestration adjacent actions, the orchestration adjacent object storage, the human agent object storage, the human agent object storage, the human agent equivalency, and the way that with these equivalency, and the way that with these equivalency, and the way that with these three principles three principles three principles evals can emerge as a first-class evals can emerge as a first-class evals can emerge as a first-class property of the system rather than as property of the system rather than as property of the system rather than as something you attach onto the side.

something you attach onto the side. something you attach onto the side. I think I think I think one of the matters here is that I like one of the matters here is that I like one of the matters here is that I like to think about architecture as to think about architecture as to think about architecture as taking your constraints very seriously taking your constraints very seriously taking your constraints very seriously and thinking about what you want to be and thinking about what you want to be and thinking about what you want to be simple within the system and then simple within the system and then simple within the system and then choosing the trade-offs choosing the trade-offs choosing the trade-offs for that. for that. for that. And of course, alongside that, some And of course, alongside that, some And of course, alongside that, some things will become hard, but it's the things will become hard, but it's the things will become hard, but it's the things that are simple that are most things that are simple that are most things that are simple that are most important to you. important to you. important to you. And that there are patterns that already And that there are patterns that already And that there are patterns that already exist across enterprises that solve for exist across enterprises that solve for exist across enterprises that solve for a lot of these things. And sure, with a lot of these things. And sure, with a lot of these things. And sure, with AI, we need to combine them in new, AI, we need to combine them in new, AI, we need to combine them in new, sometimes radical ways and bring in sometimes radical ways and bring in sometimes radical ways and bring in other way other pieces, other way other pieces, other way other pieces, but there are patterns that have worked but there are patterns that have worked but there are patterns that have worked very well within finance, within very well within finance, within very well within finance, within defense, within big tech that that can defense, within big tech that that can defense, within big tech that that can be applied to this kind of system be applied to this kind of system be applied to this kind of system architecture. architecture. architecture. And I'd say the takeaway is that And I'd say the takeaway is that And I'd say the takeaway is that where I've seen it go wrong is taking where I've seen it go wrong is taking where I've seen it go wrong is taking that initial POC, that um that point that initial POC, that um that point that initial POC, that um that point solution that showed so much promise and solution that showed so much promise and solution that showed so much promise and that that showed the high accuracy, for that that showed the high accuracy, for that that showed the high accuracy, for example, example, example, and then trying to build up from it, and then trying to build up from it, and then trying to build up from it, strapping on the enterprise requirements strapping on the enterprise requirements strapping on the enterprise requirements as you come across them. Okay, we need as you come across them. Okay, we need as you come across them. Okay, we need eval, we need uh security, we need eval, we need uh security, we need eval, we need uh security, we need auditability, and bolting these on as auditability, and bolting these on as auditability, and bolting these on as additions to the the the foundations of additions to the the the foundations of additions to the the the foundations of the POC.

the POC. the POC. You end up with something very brittle, You end up with something very brittle, You end up with something very brittle, something very hard to uh uh externalize something very hard to uh uh externalize something very hard to uh uh externalize and to generalize across different use and to generalize across different use and to generalize across different use cases. cases. cases. But where I've seen it go well is if you But where I've seen it go well is if you But where I've seen it go well is if you take the constraints of a take the constraints of a take the constraints of a production-ready, scaled enterprise production-ready, scaled enterprise production-ready, scaled enterprise uh system seriously from the beginning uh system seriously from the beginning uh system seriously from the beginning and treat those as the architectural and treat those as the architectural and treat those as the architectural principles that you're going to build principles that you're going to build principles that you're going to build everything upon and then build back up everything upon and then build back up everything upon and then build back up towards that POC accuracy using your new towards that POC accuracy using your new towards that POC accuracy using your new primitives. Thank you for your attention. Thank you. Thank you for your attention. Thank you. >> [applause]