An AI Agent Got Its User Banned From Resy — Keycard, JPMorgan Chase & AAuth

Um, hello everyone. Um, hello everyone. Welcome to AI Welcome to AI Welcome to AI Engineer in New York. I am Engineer in New York. I am Engineer in New York. I am very happy to be here very happy to be here very happy to be here today. My name is today. My name is today. My name is Ellie. I am a technical Ellie. I am a technical Ellie. I am a technical specialist at Keycard. specialist at Keycard. specialist at Keycard. I'm here today with a I'm here today with a I'm here today with a wonderful wonderful wonderful panel of people. With us is Ian, panel of people. With us is Ian, panel of people. With us is Ian, CEO of CEO of CEO of Keycard. With us are Michael Keycard. With us are Michael Keycard. With us are Michael Davis, Chief Davis, Chief Global Security Architect at Global Security Architect at JP Morgan Chase, and Dick Hart, JP Morgan Chase, and Dick Hart, JP Morgan Chase, and Dick Hart, founder of AOTH and founder of AOTH and founder of AOTH and co-author of OOTH. I look forward co-author of OOTH. I look forward co-author of OOTH. I look forward to to to talking to them about talking to them about intent-based authorization. To intent-based authorization. To set the stage, set the stage, set the stage, I want to note that I want to note that I want to note that as companies as companies as companies seek to give seek to give seek to give their agents more their agents more their agents more autonomy, they are autonomy, they are autonomy, they are turning to turning to turning to authorization at authorization at authorization at runtime. runtime. Runtime authorization Runtime authorization is the process is the process is the process of verifying the of verifying the of verifying the agent's rights for each agent's rights for each agent's rights for each tool invocation tool invocation tool invocation instead of granting instead of granting instead of granting permanent access permanent access permanent access that can be that can be that can be abused. However, abused. However, abused. However, the question of "is the question of "is the question of "is this challenge safe" this challenge safe" this challenge safe" is never simple. This is because is never simple. This is because is never simple. This is because many many many MCP tools and APIs are not MCP tools and APIs are not MCP tools and APIs are not detailed enough.

detailed enough. It is difficult for people to write the It is difficult for people to write the perfect policy, and perfect policy, and perfect policy, and the environment around the the environment around the the environment around the agent agent agent is constantly changing. Intents is constantly changing. Intents is constantly changing. Intents help provide help provide help provide additional context additional context additional context so that at runtime so that at runtime so that at runtime we can make we can make we can make better decisions about what better decisions about what better decisions about what the agent is the agent is the agent is or is not allowed to or is not allowed to or is not allowed to do. So, I want to do. So, I want to take a closer look at why take a closer look at why we need we need intent-based authorization at all. The models The models have improved significantly. They have have improved significantly. They have have improved significantly. They have larger larger larger context windows. They context windows. They context windows. They can check can check can check work because their work because their work because their vision vision vision has improved. In has improved. In has improved. In many contexts, many contexts, many contexts, people are proclaiming that people are proclaiming that people are proclaiming that AGI is already here. If AGI is already here. If AGI is already here. If the models are so good and the models are so good and the models are so good and AGI already exists, why is it so AGI already exists, why is it so AGI already exists, why is it so difficult for teams to difficult for teams to difficult for teams to give their agents give their agents give their agents more autonomy? more autonomy? more autonomy? Ian, let's start with you. Ian, let's start with you. Ian, let's start with you. You help You help You help many financial many financial many financial businesses at Keycard. businesses at Keycard. businesses at Keycard. What do you think is the What do you think is the What do you think is the obstacle? obstacle? obstacle? Well, I think in general, Well, I think in general, Well, I think in general, businesses, businesses, businesses, especially financial especially financial especially financial firms and regulated firms and regulated firms and regulated industries, almost always industries, almost always industries, almost always put security put security put security first. This is a first. This is a first. This is a way to build way to build way to build trust, and maintaining trust, and maintaining business continuity is our business continuity is our top priority; top priority; top priority; innovations come innovations come innovations come second, third or second, third or second, third or fourth. I think fourth. I think fourth. I think that in any that in any that in any enterprise you enterprise you enterprise you see a kind of see a kind of see a kind of mix of different " mix of different " islands": different islands": different islands": different identities, identities, identities, systems from completely systems from completely systems from completely different worlds. And how do you different worlds. And how do you different worlds. And how do you combine all of this so that the combine all of this so that the combine all of this so that the agent can agent can agent can navigate this, navigate this, navigate this, request access, and request access, and request access, and at the same time at the same time at the same time guarantee that the agent guarantee that the agent guarantee that the agent will perform the task will perform the task will perform the task better than the user better than the user better than the user or the business? And the unfortunate

or the business? And the unfortunate or the business? And the unfortunate situation is situation is situation is that most of that most of that most of our modern our modern our modern systems have too systems have too systems have too general a level of general a level of general a level of access, right? I don't access, right? I don't access, right? I don't know; Salesforce is a good know; Salesforce is a good know; Salesforce is a good example: for a long time example: for a long time example: for a long time they only had one they only had one they only had one scope of access for APIs in scope of access for APIs in scope of access for APIs in OAuth, which, in the context of an OAuth, which, in the context of an OAuth, which, in the context of an agent, let's say, is agent, let's say, is agent, let's say, is not very useful. This is not very useful. This is not very useful. This is actually the first and actually the first and actually the first and foremost problem: foremost problem: foremost problem: most modern most modern most modern security systems were security systems were security systems were designed for a world designed for a world designed for a world where all decisions where all decisions where all decisions were made by humans and the were made by humans and the software was software was completely completely completely deterministic, not deterministic, not deterministic, not for this newfangled for this newfangled for this newfangled magic box magic box magic box that we have created that that we have created that that we have created that thinks for itself and thinks for itself and thinks for itself and tries tries tries to find a solution to the to find a solution to the to find a solution to the problem over and over again. problem over and over again. problem over and over again. Doug, how does OAuth become Doug, how does OAuth become Doug, how does OAuth become part of this part of this part of this problem? Why is OAuth a problem? Why is OAuth a problem? Why is OAuth a hindrance and why is it not hindrance and why is it not hindrance and why is it not suitable for suitable for suitable for agents? agents? Well, when people started Well, when people started thinking about how to thinking about how to thinking about how to give agents give agents give agents access to something, access to something, access to something, OAuth was at hand, and they OAuth was at hand, and they OAuth was at hand, and they just took this just took this just took this tool, saying, tool, saying, tool, saying, here's a ready-made solution here's a ready-made solution . This is authorization. . This is authorization. . This is authorization. She has to solve She has to solve She has to solve this problem. But OAuth this problem. But OAuth this problem. But OAuth wasn't built for that wasn't built for that wasn't built for that kind of world. kind of world. kind of world. It was originally It was originally It was originally developed so that developed so that developed so that one website could one website could one website could interact with another. As interact with another. As interact with another. As you know, in OAuth 2 we you know, in OAuth 2 we you know, in OAuth 2 we added added added scopes. Before that, you could scopes. Before that, you could scopes. Before that, you could do do do anything. But this anything. But this anything. But this required prior required prior required prior registration. It is registration. It is registration. It is designed for the fact designed for the fact designed for the fact that there is a browser and that there is a browser and redirection is performed.

redirection is performed. Of course, over the years, OAuth Of course, over the years, OAuth Of course, over the years, OAuth has evolved to has evolved to has evolved to support other support other support other things as well. But the world of things as well. But the world of things as well. But the world of agents is completely different. agents is completely different. agents is completely different. We don't know We don't know We don't know in advance what exactly the in advance what exactly the agent will have to do. We need to agent will have to do. We need to make these decisions make these decisions make these decisions gradually. Different gradually. Different gradually. Different instances of the same instances of the same instances of the same agent will agent will agent will require different require different require different access levels. And, as access levels. And, as access levels. And, as Ian said, the level of Ian said, the level of Ian said, the level of granularity of granularity of granularity of access scopes that access scopes that access scopes that people are talking about—like, people are talking about—like, people are talking about—like, we have scopes for we have scopes for we have scopes for OAuth. Trying to transfer OAuth. Trying to transfer OAuth. Trying to transfer them to agents is them to agents is them to agents is too crude an too crude an too crude an approach for the approach for the approach for the tasks they tasks they tasks they perform. perform. Davis, do you Davis, do you think companies think companies think companies are moving to higher are moving to higher are moving to higher levels of autonomy? levels of autonomy? levels of autonomy? So, I think the main So, I think the main So, I think the main question is—how do we question is—how do we question is—how do we deal with this deal with this deal with this problem of too much problem of too much problem of too much public access? And public access? And public access? And I see that the more successful I see that the more successful I see that the more successful companies companies onboard agents just like they onboard onboard agents just like they onboard developers, developers, you know? To start with, you know? To start with, you know? To start with, you don't you don't you don't get full get full get full access to make access to make access to make changes on day one, changes on day one, changes on day one, right? You need to right? You need to right? You need to gradually progress through the gradually progress through the gradually progress through the path and add levels of path and add levels of path and add levels of opportunity as opportunity as opportunity as you progress. So you progress. So , the definitions that , the definitions that , the definitions that companies are developing companies are developing : what exactly is the context that : what exactly is the context that : what exactly is the context that makes this agent makes this agent makes this agent safe? And it's safe? And it's safe? And it's usually not just what the usually not just what the usually not just what the agent himself says, agent himself says, agent himself says, right? Um, that might, right? Um, that might, right? Um, that might, you know, indicate you know, indicate you know, indicate something, but there are other something, but there are other something, but there are other things, like the things, like the things, like the environment and other environment and other environment and other information, that information, that information, that will help understand will help understand will help understand levels of autonomy. A levels of autonomy. A levels of autonomy. A very simple example: very simple example: very simple example: where does the sandbox work?

where does the sandbox work? where does the sandbox work? Is she in the sandbox, Is she in the sandbox, Is she in the sandbox, right? Such things can right? Such things can right? Such things can change the level of change the level of autonomy allowed. autonomy allowed. Of course, yes. We Of course, yes. We Of course, yes. We mentioned the mentioned the mentioned the granularity issue, and granularity issue, and granularity issue, and to explain this, to explain this, to explain this, for example, with for example, with Notion's MCP server, you Notion's MCP server, you Notion's MCP server, you can say " can say " get this page get this page " like the " " like the " page view" tool, page view" tool, page view" tool, but you can't but you can't but you can't say "just say "just say "just get this get this get this database on the page." database on the page." database on the page." Why Why is intent-based runtime authorization a is intent-based runtime authorization a better model? This is a better model? This is a better model? This is a question for anyone. question for anyone. I will try to answer. I will try to answer. To me, this is something that the To me, this is something that the To me, this is something that the financial services industry financial services industry can hopefully learn from the can hopefully learn from the experience that experience that experience that banks have gone through. Banks have banks have gone through. Banks have banks have gone through. Banks have long since moved from long since moved from long since moved from authorizing and authorizing and authorizing and authenticating people authenticating people authenticating people to authorizing to authorizing to authorizing transactions, right? transactions, right? transactions, right? And when does And when does And when does this transaction take place? During this transaction take place? During this transaction take place? During execution, right? And execution, right? And execution, right? And so I think we so I think we so I think we can learn from the can learn from the can learn from the mistakes of the past and mistakes of the past and mistakes of the past and apply that to the apply that to the apply that to the new problem. new problem. new problem. I also find that it's easier to I also find that it's easier to I also find that it's easier to debug something debug something debug something at runtime at runtime at runtime when you understand when you understand when you understand the intent, right?

the intent, right? the intent, right? I think we all I think we all I think we all remember the days when we remember the days when we remember the days when we wrote checks, right? wrote checks, right? wrote checks, right? But, probably, it's more But, probably, it's more But, probably, it's more correct to say correct to say correct to say that we only saw it on that we only saw it on that we only saw it on TV. But, TV. But, TV. But, you know, there was a note box on the checks you know, there was a note box on the checks you know, there was a note box on the checks , , , right? This field right? This field right? This field was used was used was used to indicate the intent of to indicate the intent of to indicate the intent of why this payment was being made why this payment was being made why this payment was being made . But . But . But the check itself had limitations, the check itself had limitations, the check itself had limitations, right? Amount, recipient, right? Amount, recipient, right? Amount, recipient, date, etc., right? date, etc., right? date, etc., right? I think many of these I think many of these I think many of these concepts are concepts are concepts are being applied now and being applied now and being applied now and can help with can help with can help with this this this implementation process. implementation process. implementation process. And it's one of those things, And it's one of those things, And it's one of those things, you know, like a lot of you know, like a lot of you know, like a lot of words, words—it's hard. words, words—it's hard. words, words—it's hard. So, based on the So, based on the So, based on the example of making a example of making a example of making a payment or payment or payment or transaction, right? transaction, right? transaction, right? You know, this is not the scope of You know, this is not the scope of You know, this is not the scope of action. This question is: action. This question is: action. This question is: Can I do this Can I do this Can I do this one thing in this one thing in this one thing in this particular particular particular context? Regarding this context? Regarding this context? Regarding this one payment, I one payment, I one payment, I believe we believe we believe we need an agent. To need an agent. To need an agent. To make any make any make any decision, should decision, should decision, should he be able to he be able to he be able to do so? You need a do so? You need a do so? You need a whole other context: whole other context: whole other context: why are we why are we why are we making this payment in the first place? making this payment in the first place? making this payment in the first place? What events happened What events happened What events happened before? You know, didn't before? You know, didn't before? You know, didn't we make the we make the we make the same payment, say same payment, say , 5 seconds ago, and , 5 seconds ago, and , 5 seconds ago, and isn't that a double isn't that a double isn't that a double payment? Just because it's payment? Just because it's payment? Just because it's right in this right in this right in this context doesn't context doesn't context doesn't mean it's mean it's mean it's right within the right within the right within the broader mission. So at AOTH broader mission. So at AOTH broader mission. So at AOTH we work on the we work on the we work on the mission concept: mission concept: mission concept: what do we want the what do we want the what do we want the agent to achieve on a agent to achieve on a agent to achieve on a global scale global scale ? And then every ? And then every ? And then every decision is made decision is made decision is made in the context of intentions in the context of intentions in the context of intentions about what is about what is about what is happening happening happening overall to overall to overall to achieve that goal.

achieve that goal. And you need transaction-level granularity for transaction-level granularity for many things to many things to many things to know exactly what's know exactly what's know exactly what's happening, not happening, not happening, not just "I'm just "I'm just "I'm allowed allowed allowed to make payments to make payments ." So, I think one of the ." So, I think one of the ." So, I think one of the most interesting most interesting most interesting practical examples practical examples practical examples for understanding for understanding for understanding why intents are needed why intents are needed why intents are needed is two ChatGPT is two ChatGPT is two ChatGPT or Claude sessions: in one I or Claude sessions: in one I or Claude sessions: in one I say, “Hey, help say, “Hey, help say, “Hey, help me shop, me shop, ” and in the other, “Hey, ” and in the other, “Hey, ” and in the other, “Hey, help me help me help me understand why there’s an understand why there’s an understand why there’s an error in the work error in the work error in the work environment.” They environment.” They environment.” They have completely different have completely different have completely different profiles in terms of profiles in terms of profiles in terms of what permissions they what permissions they what permissions they should have, and also should have, and also should have, and also when they should when they should when they should bring you back into the bring you back into the bring you back into the process, i.e. when does the process, i.e. when does the process, i.e. when does the person need to person need to person need to intervene again? After all, intervene again? After all, intervene again? After all, if we don't have if we don't have if we don't have the intention or the intention or the intention or ability to convey ability to convey ability to convey what what what the user or the user or the user or business's intent is, we won't business's intent is, we won't business's intent is, we won't be able to achieve be able to achieve be able to achieve autonomy, autonomy, autonomy, right? So a lot of right? So a lot of right? So a lot of the speeches this the speeches this the speeches this morning were about how morning were about how morning were about how we're going to run we're going to run we're going to run the business with the business with the business with agents and agents and agents and do all that. do all that. do all that. Sounds great. But Sounds great. But Sounds great. But what happens when a what happens when a Pocket OS-style incident occurs, Pocket OS-style incident occurs, such as when it such as when it such as when it accidentally deletes a accidentally deletes a accidentally deletes a production database production database production database instead of a test database because it instead of a test database because it instead of a test database because it has too broad has too broad has too broad access rights access rights access rights instead of limited ones? We instead of limited ones? We instead of limited ones? We as an industry want as an industry want as an industry want to use this " to use this " magic box" magic box" magic box" that we have created to that we have created to that we have created to automate our automate our automate our lives. The current state of lives. The current state of lives. The current state of affairs is that you affairs is that you affairs is that you can do it, can do it, can do it, take all the take all the take all the risks, and then risks, and then risks, and then watch the watch the watch the consequences that consequences that consequences that arise. So until arise. So until arise. So until we solve this we solve this we solve this underlying problem, underlying problem, underlying problem, realizing realizing realizing true true true autonomy— autonomy— autonomy— the ability the ability the ability to set up business to set up business processes, step aside processes, step aside processes, step aside as humans, and as humans, and as humans, and scale it— scale it— scale it— requires first requires first requires first addressing the issue of addressing the issue of addressing the issue of intent.

intent. intent. Certainly. Now that Certainly. Now that Certainly. Now that we understand we understand we understand why we need why we need intent-based authorization, intent-based authorization, let's talk let's talk let's talk about what it is in about what it is in about what it is in practice. What are the practice. What are the practice. What are the primitives here? What does this primitives here? What does this primitives here? What does this actually look like actually look like actually look like in practice? Um, Dick in practice? Um, Dick in practice? Um, Dick hinted at this with the hinted at this with the hinted at this with the AOTH mission concept. AOTH mission concept. AOTH mission concept. Um, so, Dick, Um, so, Dick, Um, so, Dick, tell me more tell me more tell me more about the mission. What is about the mission. What is about the mission. What is the mission based on? Is the mission based on? Is the mission based on? Is it based on it based on it based on user intent? user intent? user intent? Is this the agent's intention? Is Is this the agent's intention? Is Is this the agent's intention? Is it a combination of both? it a combination of both? it a combination of both? Well, an agent, you know, has to Well, an agent, you know, has to Well, an agent, you know, has to work on behalf of an work on behalf of an work on behalf of an individual. And at AOTH, we individual. And at AOTH, we individual. And at AOTH, we define a “person” define a “person” define a “person” as an organization or as an organization or as an organization or a person, because he can a person, because he can a person, because he can act on behalf of either act on behalf of either . So he . So he . So he must act in the best must act in the best must act in the best interests of the person he interests of the person he interests of the person he works for, because works for, because works for, because that person is ultimately that person is ultimately that person is ultimately responsible for the responsible for the responsible for the agent's actions, right? agent's actions, right? The agent can't be The agent can't be held held held responsible, responsible, responsible, right? So a mission is what right? So a mission is what right? So a mission is what a person instructs an a person instructs an a person instructs an agent to do. A agent to do. A good agent will go and good agent will go and meet with the person meet with the person meet with the person to discuss what to discuss what to discuss what the parameters of the parameters of the parameters of that assignment should be. that assignment should be.

You know, a good You know, a good example is to think back to example is to think back to example is to think back to the old days, I think the old days, I think three months ago, three months ago, three months ago, when you wanted to when you wanted to when you wanted to code something, code something, code something, you created a plan, and you created a plan, and you created a plan, and then the agent then the agent then the agent executed it. Now you executed it. Now you executed it. Now you just tell him just tell him just tell him what to do and he figures it out himself what to do and he figures it out himself what to do and he figures it out himself . . . But this whole concept But this whole concept —what is a plan? What are you —what is a plan? What are you —what is a plan? What are you trying to do? trying to do? trying to do? And you pass it on to the And you pass it on to the And you pass it on to the agent. So, this agent. So, this agent. So, this creates a context for creates a context for creates a context for you to make you to make you to make authorization authorization authorization decisions in the future. decisions in the future. decisions in the future. And then that And then that And then that mission ID mission ID mission ID is tracked throughout is tracked throughout is tracked throughout . So you can . So you can . So you can go and, you know, go and, you know, go and, you know, check and check and check and see exactly what see exactly what see exactly what happened and what happened and what happened and what decisions were decisions were decisions were made in the context of made in the context of made in the context of this one mission, this one mission, this one mission, right? Because today, right? Because today, right? Because today, you know, there's no really you know, there's no really you know, there's no really good concept of good concept of good concept of agent identity agent identity agent identity or mission, right? It's or mission, right? It's or mission, right? It's just creating just creating just creating something, and the systems something, and the systems something, and the systems make decisions without make decisions without make decisions without knowing which knowing which knowing which agent instance is agent instance is agent instance is doing it and in doing it and in doing it and in what mission context it's what mission context it's what mission context it's happening. Yes. happening. Yes. happening. Yes. Tell me more about this Tell me more about this Tell me more about this negotiation phase, because negotiation phase, because negotiation phase, because what happens if what happens if what happens if a user a user a user talks to talks to talks to an agent and says, “ an agent and says, “ I want you to I want you to I want you to read the read the read the CEO’s email CEO’s email CEO’s email ”? This is ”? This is ”? This is beyond the beyond the beyond the authority of that authority of that authority of that agent. So, is agent. So, is agent. So, is this this this user intent being rejected? Uh user intent being rejected? Uh , it should deviate, , it should deviate, , it should deviate, right? We know the right? We know the right? We know the importance of clear importance of clear importance of clear division, and I'm not division, and I'm not division, and I'm not sure if we'll sure if we'll sure if we'll discuss this discuss this discuss this later, but the division of later, but the division of labor: there's an agent labor: there's an agent that does the work, that does the work, that does the work, and we believe that there should and we believe that there should and we believe that there should be a separate agent be a separate agent be a separate agent that oversees the actions of the that oversees the actions of the that oversees the actions of the former. So you former. So you

former. So you have a division of have a division of have a division of labor where one person can't labor where one person can't labor where one person can't do do do the work, but they the work, but they the work, but they decide what decide what decide what that agent is allowed to do. that agent is allowed to do. that agent is allowed to do. So if a user So if a user So if a user says, "Oh, I want you to says, "Oh, I want you to says, "Oh, I want you to read the read the read the company email," that's company email," that's company email," that's likely likely likely going to be immediately going to be immediately going to be immediately blocked by blocked by blocked by deterministic deterministic deterministic policies, right? policies, right? policies, right? It's not something It's not something It's not something vague, is it? vague, is it? vague, is it? Reading other people's emails Reading other people's emails — we have reliable — we have reliable — we have reliable means of control for means of control for means of control for this. But this. But this. But telling an agent, telling an agent, telling an agent, for example, to forward the for example, to forward the for example, to forward the CEO's email CEO's email CEO's email externally is externally is externally is a bit more complicated, a bit more complicated, a bit more complicated, and that's exactly what a and that's exactly what a controlling agent should ideally stop controlling agent should ideally stop by saying, "That's probably a by saying, "That's probably a by saying, "That's probably a bad idea, no, you bad idea, no, you bad idea, no, you can't do that." can't do that." Certainly. So I Certainly. So I think politics think politics is another is another is another fundamental fundamental fundamental element based element based element based on intentions. Ian, how do on intentions. Ian, how do on intentions. Ian, how do you see Keycard as a you see Keycard as a you see Keycard as a situation where the mission situation where the mission situation where the mission changes or a changes or a policy needs to be updated, how can Keycard policy needs to be updated, how can Keycard help help help track those track those track those changes when there is a changes when there is a changes when there is a need to update the need to update the need to update the mission or policy? mission or policy? mission or policy? So, I think if you So, I think if you So, I think if you take a step back, take a step back, take a step back, Davis and I were Davis and I were Davis and I were sitting in a room together sitting in a room together sitting in a room together before AOTH and other before AOTH and other before AOTH and other things like that things like that things like that came into came into came into the ecosystem, and one the ecosystem, and one the ecosystem, and one of the issues we of the issues we of the issues we talked about a lot was talked about a lot was talked about a lot was "consent fatigue." And I "consent fatigue." And I "consent fatigue." And I think that at the very least, think that at the very least, think that at the very least, we're always faced we're always faced we're always faced with a balance: how with a balance: how with a balance: how safe to make the safe to make the safe to make the system versus the system versus the system versus the impact on the person.

impact on the person. impact on the person. So at Keycard, our So at Keycard, our So at Keycard, our general approach is general approach is general approach is that that that you need a broad you need a broad you need a broad deterministic deterministic deterministic policy that essentially policy that essentially says what should be says what should be says what should be prohibited under prohibited under all circumstances, or what exactly all circumstances, or what exactly all circumstances, or what exactly always requires always requires always requires human intervention. And human intervention. And human intervention. And from that, what you from that, what you from that, what you really have to really have to really have to think about is, " think about is, " Okay, cool." Now that Okay, cool." Now that Okay, cool." Now that this is in place, this is in place, this is in place, there is a big gap there is a big gap there is a big gap between all the policies between all the policies between all the policies I can write I can write I can write as a person or as a person or as a person or organization and what organization and what organization and what I actually want to I actually want to I actually want to see the policies look like see the policies look like see the policies look like within a specific within a specific within a specific task. So the task. So the task. So the real challenge is real challenge is real challenge is how do I how do I how do I combine these two things, combine these two things, combine these two things, right? Because right? Because right? Because people can't people can't people can't write everything. So write everything. So write everything. So there's a version of this, there's a version of this, there's a version of this, like like like user- user- user- agent or agent or agent or agent-agent interaction, and an intent arises between them, agent-agent interaction, and an intent arises between them, like a like a transactional intent. transactional intent. transactional intent. And then there's the business intent— And then there's the business intent— why they're creating why they're creating why they're creating or implementing an or implementing an or implementing an agent. So why did we agent. So why did we agent. So why did we purchase Cloud Code? Well, purchase Cloud Code? Well, purchase Cloud Code? Well, because we wanted because we wanted because we wanted our developers to our developers to our developers to be more be more be more successful, or because we successful, or because we successful, or because we wanted to do these wanted to do these wanted to do these different things. And in different things. And in different things. And in the context of the context of the context of using Cloud Code using Cloud Code using Cloud Code or anything else, or anything else, or anything else, why am I working with why am I working with why am I working with Cloud Code now? Well, actually, Cloud Code now? Well, actually, Cloud Code now? Well, actually, to fix a to fix a to fix a bug in the bug in the bug in the testing environment, right? testing environment, right? testing environment, right? And these are two different types of And these are two different types of And these are two different types of intentions. And if we intentions. And if we intentions. And if we can understand these can understand these can understand these two different things and two different things and two different things and then somehow then somehow then somehow turn them into, turn them into, turn them into, let's say, the boundaries of what let's say, the boundaries of what let's say, the boundaries of what these agents can do these agents can do these agents can do , you can , you can , you can basically calculate basically calculate basically calculate their intersection and their intersection and their intersection and say, "Okay, here's say, "Okay, here's say, "Okay, here's what you can what you can what you can actually do." And then the actually do." And then the actually do." And then the trick for anyone trick for anyone trick for anyone who builds these kinds of who builds these kinds of who builds these kinds of systems is systems is systems is how to ensure how to ensure how to ensure that I don't involve that I don't involve that I don't involve the user unless the user unless absolutely necessary, absolutely necessary, because that's what's going to be the because that's what's going to be the because that's what's going to be the problem of the problem of the problem of the adoption curve. But it's adoption curve. But it's adoption curve. But it's also a problem of the also a problem of the also a problem of the implementation curve implementation curve implementation curve when it comes to when it comes to when it comes to giving agents giving agents giving agents autonomy in the first autonomy in the first autonomy in the first place.

place. place. That makes sense. That makes sense. That makes sense. I like what I like what I like what you said about you said about you said about intersectionality. And I think that intersectionality. And I think that intersectionality. And I think that 's a unique thing 's a unique thing 's a unique thing that we don't that we don't that we don't talk about enough within talk about enough within identity and identity and access management, right? access management, right? access management, right? Usually you only Usually you only Usually you only get access to get access to get access to one thing, right? And one thing, right? And one thing, right? And here's what I've seen here's what I've seen here's what I've seen successful in this intention- successful in this intention- successful in this intention- based approach based approach based approach . Um, and what . Um, and what fascinates me even more is that fascinates me even more is that these different intentions these different intentions these different intentions can come from can come from can come from different parts of the different parts of the different parts of the organization, right? organization, right? organization, right? So you have a So you have a So you have a separation of these things separation of these things separation of these things so that they can so that they can so that they can exist separately, but exist separately, but exist separately, but then come together then come together then come together to form, to form, to form, you know, a solution. And this you know, a solution. And this you know, a solution. And this solution doesn't have to solution doesn't have to solution doesn't have to be be be static, right? static, right? static, right? So if we So if we So if we use the use the CEO email example, you know, CEO email example, you know, if this context if this context if this context is for an is for an is for an administrative administrative administrative assistant, then yeah, assistant, then yeah, assistant, then yeah, maybe she should maybe she should maybe she should read this email, read this email, read this email, right? Or maybe right? Or maybe right? Or maybe just a calendar, right? just a calendar, right? just a calendar, right? And so, I think when And so, I think when And so, I think when we see and start we see and start we see and start thinking about these thinking about these thinking about these components of intersection, components of intersection, components of intersection, we have a we have a we have a whole different way of whole different way of whole different way of expressing policy expressing policy , and also the ability to , and also the ability to , and also the ability to engage not just one engage not just one engage not just one side of the equation side of the equation side of the equation that we've always that we've always that we've always talked about before (that talked about before (that talked about before (that the program does something) the program does something) , but now we actually , but now we actually , but now we actually have an ecosystem that we have an ecosystem that we have an ecosystem that we can engage with. So what can engage with. So what can engage with. So what I'm seeing more and more is I'm seeing more and more is I'm seeing more and more is organizations organizations organizations starting starting starting to realize that to realize that to realize that supporting this supporting this supporting this state or context of state or context of state or context of other elements to other elements to other elements to help with this help with this help with this intent can intent can intent can completely change completely change completely change your effectiveness in your effectiveness in your effectiveness in authorization at authorization at authorization at runtime.

runtime. runtime. Do you see this Do you see this Do you see this in different in different in different businesses today? Do businesses today? Do businesses today? Do most people most people most people have have have mission and policy tracking or is mission and policy tracking or is mission and policy tracking or is this largely this largely this largely absent today? absent today? absent today? No, it's still completely No, it's still completely No, it's still completely new. And I think, you new. And I think, you new. And I think, you know, part of the know, part of the know, part of the reason I'm excited to reason I'm excited to reason I'm excited to talk about this is because talk about this is because talk about this is because it it it changes the way we changes the way we changes the way we think about it, think about it, think about it, right? Even in the right? Even in the right? Even in the simplest simplest simplest case, if you're a case, if you're a case, if you're a developer and you think in terms of developer and you think in terms of transactional categories and have transactional categories and have these concepts of intent, these concepts of intent, these concepts of intent, it makes it easier for you to it makes it easier for you to it makes it easier for you to potentially create potentially create potentially create components that components that components that will be composed and will be composed and will be composed and used used used through that intent. So I through that intent. So I through that intent. So I think that would give us think that would give us think that would give us more ways to more ways to more ways to make sense of it, make sense of it, make sense of it, communicate, and communicate, and communicate, and ultimately be more ultimately be more ultimately be more compatible, right? And compatible, right? And compatible, right? And for me, the last thing I for me, the last thing I for me, the last thing I find interesting is the find interesting is the find interesting is the diagnostics that diagnostics that diagnostics that can be run can be run can be run based on this. Intention based on this. Intention is something that potentially is something that potentially is something that potentially goes through goes through goes through many events, many events, many events, right? And I think one of the right? And I think one of the right? And I think one of the examples I've examples I've examples I've seen before is the seen before is the seen before is the conversation about how this conversation about how this conversation about how this mission cuts mission cuts mission cuts through everything. It can through everything. It can through everything. It can be long-lasting. be long-lasting. be long-lasting. It could be weeks It could be weeks It could be weeks during which the intention during which the intention during which the intention still persists. still persists. still persists. So for most So for most So for most businesses, it's still a businesses, it's still a businesses, it's still a very early stage. very early stage. very early stage. Certainly. And yes, this Certainly. And yes, this Certainly. And yes, this compatibility is compatibility is compatibility is key. I think key. I think key. I think it's quite challenging for it's quite challenging for it's quite challenging for enterprises enterprises enterprises because they're because they're because they're between between between different clouds, different clouds, different clouds, environments, and environments, and environments, and legacy legacy legacy systems, and that systems, and that systems, and that mission policy mission policy mission policy has to work across has to work across has to work across all of these different transitions all of these different transitions .

. . Yes, many Yes, many Yes, many businesses still businesses still businesses still deal with the deal with the deal with the fact that the permission fact that the permission fact that the permission says “read only says “read only ” when it’s actually “ ” when it’s actually “ write”, right? So you write”, right? So you write”, right? So you have other issues have other issues have other issues that you have to that you have to that you have to deal with, and I think deal with, and I think deal with, and I think this will help this will help this will help create a create a create a taxonomy or a language that makes it taxonomy or a language that makes it taxonomy or a language that makes it easier for people easier for people easier for people to interact. Yes, to interact. Yes, to interact. Yes, absolutely. One of the absolutely. One of the absolutely. One of the reasons I'm excited reasons I'm excited reasons I'm excited about about intent-based authorization intent-based authorization is that is that is that it has the potential it has the potential it has the potential to be a huge to be a huge to be a huge breakthrough for breakthrough for breakthrough for things like agent things like agent things like agent commerce. So let's commerce. So let's commerce. So let's apply apply intent-based authorization to intent-based authorization to something real. I something real. I something real. I think think think intent-based authorization intent-based authorization intent-based authorization adds a lot of adds a lot of adds a lot of value here, and for value here, and for value here, and for users, it can users, it can users, it can help help help ensure that their ensure that their ensure that their agents are purchasing the agents are purchasing the agents are purchasing the right product right product right product within their within their within their personal budget. And personal budget. And personal budget. And for sellers, this for sellers, this for sellers, this means capturing means capturing means capturing intent, which gives them intent, which gives them intent, which gives them additional data to additional data to additional data to understand why understand why understand why a purchase is being made a purchase is being made , and whether they should even , and whether they should even , and whether they should even trust that trust that trust that agent to agent to agent to visit their visit their visit their site. Davis, when we site. Davis, when we site. Davis, when we talk about agents talk about agents talk about agents in the context of in the context of in the context of financial services and financial services and financial services and agent commerce, agent commerce, agent commerce, why is intentionality so important why is intentionality so important why is intentionality so important ? ? ? Um, I guess this just Um, I guess this just Um, I guess this just expands on a concept expands on a concept expands on a concept that not everyone knows about, that not everyone knows about, that not everyone knows about, but many have heard—" but many have heard—" know your customer," know your customer," know your customer," right? So I want to know right? So I want to know right? So I want to know my client, know my client, know my client, know who they are and what they do.

who they are and what they do. who they are and what they do. But I think we're also But I think we're also But I think we're also starting to see the starting to see the starting to see the need to " need to " know your agent," know your agent," know your agent," right? So I believe there right? So I believe there right? So I believe there is a connection here between the is a connection here between the is a connection here between the agent itself, agent itself, agent itself, the intent of the real the intent of the real the intent of the real user, the intent of the user, the intent of the user, the intent of the operator, the intent of operator, the intent of operator, the intent of the processor, etc. the processor, etc. We couldn't We couldn't really put really put really put it together before, right? This is of course it together before, right? This is of course it together before, right? This is of course important in matters of important in matters of important in matters of commerce, as commerce, as additional levels emerge, additional levels emerge, such as intentions such as intentions such as intentions regarding regulatory regarding regulatory regarding regulatory requirements, etc. And I think requirements, etc. And I think requirements, etc. And I think this will help this will help this will help streamline operations, which streamline operations, which streamline operations, which means we will have means we will have means we will have more access to more access to more access to agents who agents who agents who will do the will do the will do the work for us. work for us. This is logical. Another This is logical. Another obstacle to obstacle to obstacle to agent commerce agent commerce is that the web was not is that the web was not is that the web was not built for built for built for agents. Yan, can agents. Yan, can agents. Yan, can you tell me how you tell me how you tell me how agent identity agent identity agent identity can help? can help? can help? Yes, I think if we Yes, I think if we Yes, I think if we step back, step back, step back, we always had a we always had a we always had a problem: what to do with an problem: what to do with an problem: what to do with an unauthorized unauthorized unauthorized client? For client? For client? For example, I go example, I go example, I go to eBay without logging in to eBay without logging in to eBay without logging in , , , I use Chrome, eBay I use Chrome, eBay I use Chrome, eBay doesn't know who I am. How do I doesn't know who I am. How do I doesn't know who I am. How do I place an order place an order place an order as a guest? How do I as a guest? How do I as a guest? How do I prove that I am human? prove that I am human? prove that I am human? How do I, you know, How do I, you know, How do I, you know, so they can so they can so they can say, "Yeah, you say, "Yeah, you say, "Yeah, you can can can bet on these stocks for bet on these stocks for bet on these stocks for $50." And if we $50." And if we $50." And if we look at what's look at what's look at what's happening with happening with happening with personal personal personal agents like Muse, Instinct, agents like Muse, Instinct, agents like Muse, Instinct, Grockbot, and the recent Grockbot, and the recent Grockbot, and the recent announcements from Sierra, Decagon, and announcements from Sierra, Decagon, and announcements from Sierra, Decagon, and others about the others about the others about the personal agent protocol personal agent protocol — the fundamental — the fundamental — the fundamental problem is problem is problem is how do you allow an how do you allow an how do you allow an agent to go on a site, agent to go on a site, agent to go on a site, interact with it, and interact with it, and interact with it, and know that you know that you know that you can really trust can really trust can really trust that agent? For a that agent? For a that agent? For a long time, we long time, we long time, we had captchas on had captchas on had captchas on the Internet. Why? Because the Internet. Why? Because the Internet. Why? Because we want to we want to we want to prevent spam. We prevent spam. We prevent spam. We want to prevent want to prevent want to prevent abuse. So we abuse. So we abuse. So we added captchas that added captchas that added captchas that only a human can solve only a human can solve only a human can solve . So, we have a . So, we have a . So, we have a captcha that stops captcha that stops

captcha that stops spam. So they spam. So they spam. So they can't spam can't spam can't spam the system, right? And now the system, right? And now the system, right? And now agents have appeared, and agents have appeared, and agents have appeared, and they have learned to they have learned to they have learned to bypass captchas. But bypass captchas. But bypass captchas. But this was a traditional this was a traditional this was a traditional problem on the Internet. problem on the Internet. problem on the Internet. In fact, if you In fact, if you In fact, if you think back to think back to think back to two or two or two or three years ago, three years ago, three years ago, Vercel—and I think we all Vercel—and I think we all Vercel—and I think we all know Vercel—was know Vercel—was know Vercel—was blocked in blocked in blocked in Spain. So why was Spain. So why was Spain. So why was Vercel blocked in Vercel blocked in Vercel blocked in Spain? Because Vercel Spain? Because Vercel Spain? Because Vercel used the used the used the same IP addresses for same IP addresses for same IP addresses for every host on every host on every host on its platform. And there its platform. And there its platform. And there was one host on Vercel, was one host on Vercel, was one host on Vercel, one website that one website that one website that they hosted that they hosted that they hosted that streamed content streamed content streamed content like the like the like the English Premier English Premier League. So Spain League. So Spain League. So Spain decided, "Hey, to decided, "Hey, to decided, "Hey, to stop stop stop this illegal streaming of this illegal streaming of this illegal streaming of premium premium premium content in the country, we're content in the country, we're content in the country, we're just going to block just going to block just going to block all of Vercel," you know? all of Vercel," you know? all of Vercel," you know? And just as Vercel And just as Vercel And just as Vercel had this problem, the had this problem, the had this problem, the agents have the same one, agents have the same one, agents have the same one, only much, only much, only much, much more serious. much more serious. much more serious. So how do you make So how do you make So how do you make unregistered unregistered unregistered traffic that you've traffic that you've traffic that you've never encountered never encountered never encountered show up on your show up on your show up on your doorstep, and you doorstep, and you doorstep, and you decide whether to let decide whether to let decide whether to let it in or not? This is, in fact, the it in or not? This is, in fact, the it in or not? This is, in fact, the first and first and first and most important most important most important fundamental fundamental fundamental problem that we problem that we problem that we have to solve. This is the same have to solve. This is the same have to solve. This is the same problem as problem as problem as with Muse and Instinct. And they're with Muse and Instinct. And they're with Muse and Instinct. And they're trying to trying to trying to figure out how do we figure out how do we figure out how do we create acceptable create acceptable create acceptable rules that we rules that we rules that we can all can all can all work by? So that I, as a work by? So that I, as a work by? So that I, as a resource, can resource, can resource, can decide which decide which decide which agents to trust agents to trust agents to trust based on based on based on who they are, and at the same time who they are, and at the same time who they are, and at the same time know how to communicate with them know how to communicate with them know how to communicate with them .

. Logically. Dick, how is AOP Logically. Dick, how is AOP helping to unlock the helping to unlock the helping to unlock the future of agency future of agency future of agency commerce? Uh, well, commerce? Uh, well, commerce? Uh, well, the main thing is that many the main thing is that many the main thing is that many people create some kind of people create some kind of people create some kind of closed closed closed proprietary systems proprietary systems , right? You know, I'm a , right? You know, I'm a , right? You know, I'm a fan of the fan of the fan of the Personal Agent Protocol concept, Poppy, but Personal Agent Protocol concept, Poppy, but Personal Agent Protocol concept, Poppy, but it doesn't really it doesn't really it doesn't really go go go far enough, does it? It's far enough, does it? It's far enough, does it? It's just, "Okay, we have just, "Okay, we have just, "Okay, we have some some some agent identity, we're just agent identity, we're just agent identity, we're just using some using some using some building blocks," and building blocks," and building blocks," and that's an attractive that's an attractive that's an attractive approach because you just approach because you just approach because you just do it. But this does not do it. But this does not do it. But this does not solve many of the solve many of the solve many of the new challenges new challenges new challenges needed for the world of needed for the world of needed for the world of agents. So agents. So agents. So the key motivator the key motivator the key motivator for people to have for people to have for people to have standards is to be standards is to be standards is to be able to able to able to combine combine combine components from different components from different components from different places, right? So I places, right? So I places, right? So I strongly believe strongly believe strongly believe that we should have that we should have that we should have control over what control over what control over what our agents do. our agents do. our agents do. At some point, At some point, At some point, agents will be agents will be agents will be doing so doing so doing so much work that we much work that we much work that we can't possibly have a can't possibly have a can't possibly have a human human human approve every approve every approve every step, right? We've already step, right? We've already step, right? We've already seen that this simply seen that this simply seen that this simply doesn't scale.

doesn't scale. doesn't scale. So this means that So this means that So this means that we need an overseeing agent to we need an overseeing agent to guide our guide our policies and policies and policies and decide how we decide how we decide how we act; and it must be act; and it must be act; and it must be independent of the independent of the independent of the agents doing agents doing agents doing all the work. And if we all the work. And if we all the work. And if we want to be want to be want to be able to able to able to connect and connect and connect and use these use these use these things, we need a things, we need a things, we need a protocol to protocol to protocol to distinguish them. Otherwise, distinguish them. Otherwise, distinguish them. Otherwise, decisions about what decisions about what decisions about what an agent can and an agent can and an agent can and cannot do, and what cannot do, and what restrictions are imposed, will be restrictions are imposed, will be determined by the determined by the determined by the agent developers themselves agent developers themselves . And I don't really want to . And I don't really want to . And I don't really want to give them give them give them control over what I control over what I control over what I can do and have all can do and have all can do and have all my decisions my decisions my decisions based on that. based on that. based on that. Or I want to be Or I want to be Or I want to be able to move able to move able to move this supervision to another this supervision to another this supervision to another component, and to be component, and to be component, and to be able to able to able to connect different connect different connect different modules, we need a modules, we need a modules, we need a protocol for protocol for protocol for them to interact them to interact . . . I'll just add to I'll just add to I'll just add to that. I think we that. I think we that. I think we see now that we all see now that we all see now that we all want the network want the network want the network to be federated, to be federated, to be federated, right? This means that right? This means that right? This means that websites or resources on websites or resources on websites or resources on the web can the web can the web can decide what they are decide what they are decide what they are allowed to do, but allowed to do, but allowed to do, but the standards by which the standards by which the standards by which everything works must everything works must everything works must be federated be federated be federated so that we can all so that we can all so that we can all interact. And I interact. And I interact. And I think we're at a think we're at a think we're at a point now where there are point now where there are point now where there are a few big a few big a few big players, and the question players, and the question players, and the question for the industry is, are for the industry is, are for the industry is, are we going to agree to we going to agree to we going to agree to terms that tie terms that tie terms that tie us to them and us to them and us to them and lock the door? Will lock the door? Will lock the door? Will we do what we've we do what we've we do what we've always done: always done: always done: build build build federated federated federated protocols like OIDC protocols like OIDC protocols like OIDC or TLS? They or TLS? They or TLS? They powered the powered the powered the vast web, vast web, vast web, fueled trade, fueled trade, fueled trade, allowed information allowed information allowed information to flow freely, to flow freely, to flow freely, and gave us all and gave us all and gave us all jobs through the jobs through the jobs through the ability to be ability to be ability to be inventive. So inventive. So inventive. So the question I'm asking the the question I'm asking the the question I'm asking the entire industry is: entire industry is: entire industry is: okay, it's okay, it's okay, it's coming. What coming. What coming. What version of the world do we version of the world do we version of the world do we want to choose?

want to choose? want to choose? Yes. I will add that it would be Yes. I will add that it would be Yes. I will add that it would be interesting if everything interesting if everything interesting if everything went perfectly and we went perfectly and we went perfectly and we had agents had agents had agents buying something and everyone buying something and everyone buying something and everyone would be happy. would be happy. would be happy. I think that as a result, I think that as a result, I think that as a result, we will understand that there is we will understand that there is we will understand that there is another component that another component that another component that needs to be worked on, which is the needs to be worked on, which is the needs to be worked on, which is the person himself. And person himself. And person himself. And one of the things, I one of the things, I one of the things, I think, when think, when think, when intentions emerge intentions emerge is the emergence of what's is the emergence of what's is the emergence of what's called psychological called psychological called psychological ownership. This means ownership. This means that there is now a that there is now a that there is now a connection between the person connection between the person connection between the person and the result of and the result of and the result of this agent's work. this agent's work. this agent's work. When you just When you just When you just disable it or disable it or disable it or connect via an OAuth connect via an OAuth token or something token or something token or something like that, it looks like that, it looks like that, it looks like magic and you don't like magic and you don't like magic and you don't really understand what's really understand what's really understand what's happening. And so happening. And so happening. And so I think that with intention, it I think that with intention, it I think that with intention, it will be easier for a person to will be easier for a person to will be easier for a person to take take take responsibility responsibility responsibility when something goes wrong. when something goes wrong. when something goes wrong. And I think when it And I think when it And I think when it comes to transferring comes to transferring comes to transferring real money or real money or real money or things like that, you things like that, you things like that, you want a human being to ultimately be responsible for it want a human being to ultimately be responsible for it . I think many . I think many have seen that have seen that have seen that 1979 IBM manual. 1979 IBM manual. 1979 IBM manual. It says, you know, and It says, you know, and It says, you know, and that's why this photo, I think, is that's why this photo, I think, is that's why this photo, I think, is very apt, because we very apt, because we very apt, because we can't put can't put can't put the responsibility on the the responsibility on the the responsibility on the machines, we need machines, we need machines, we need people to be connected to this people to be connected to this people to be connected to this , and the intention, I , and the intention, I , and the intention, I think, is that connection think, is that connection . . Yes, and another word Yes, and another word for for for intention is motive, as intention is motive, as intention is motive, as we discussed we discussed we discussed last night, uh. And last night, uh. And last night, uh. And this ability to this ability to this ability to hold someone hold someone hold someone accountable, whether accountable, whether accountable, whether it's a person or an it's a person or an it's a person or an agent, I think is very agent, I think is very agent, I think is very important. And I think important. And I think important. And I think intention can also intention can also intention can also help with that, help with that, help with that, to understand: why did to understand: why did to understand: why did things go wrong? Was things go wrong? Was things go wrong? Was it the it the it the user's intent, you know, did user's intent, you know, did user's intent, you know, did the user have the user have the user have malicious intent, or did the agent malicious intent, or did the agent malicious intent, or did the agent have malicious intent. Uh, and have malicious intent. Uh, and have malicious intent. Uh, and then using then using then using that intention to that intention to that intention to be able to be able to be able to react. Uh, how do react. Uh, how do react. Uh, how do each of you see each of you see each of you see that intent provides this that intent provides this that intent provides this advantage and the advantage and the advantage and the ability ability ability to respond to a to respond to a to respond to a potentially potentially potentially unscrupulous unscrupulous unscrupulous user or user or user or agent?

agent? agent? Uh, I guess this Uh, I guess this Uh, I guess this goes back to what I goes back to what I goes back to what I mentioned earlier about mentioned earlier about mentioned earlier about diagnostics, right? So diagnostics, right? So diagnostics, right? So if you change the word if you change the word if you change the word "intent" to "motive," "intent" to "motive," "intent" to "motive," then you change " then you change " diagnostics" to " diagnostics" to " forensics," right? forensics," right? forensics," right? And I think anyone who's ever And I think anyone who's ever debugged or debugged or investigated an investigated an AI agent that's AI agent that's gone out of control gone out of control knows that trying knows that trying knows that trying to understand what it was to understand what it was to understand what it was trying to do trying to do trying to do versus what versus what versus what the user intended is the user intended is the user intended is very difficult, right? Uh very difficult, right? Uh , so having , so having , so having some of this some of this some of this information that's information that's information that's distributed and distributed and distributed and immutable in the system and immutable in the system and immutable in the system and so on gives you certain so on gives you certain so on gives you certain properties that properties that properties that allow you allow you allow you to complete to complete to complete the investigation faster, right? And as the investigation faster, right? And as the investigation faster, right? And as these these these things become more things become more things become more compatible, I think we'll compatible, I think we'll compatible, I think we'll start to see start to see start to see companies being able to companies being able to companies being able to more quickly determine: well, more quickly determine: well, more quickly determine: well, was this was this was this human error? Was this an human error? Was this an agent software error? agent software error? This...what kind of This...what kind of This...what kind of mistake was this? And mistake was this? And mistake was this? And deal with deal with deal with the consequences of this accordingly. Uh, the consequences of this accordingly. Uh, the consequences of this accordingly. Uh, we see this in the we see this in the we see this in the financial system, financial system, financial system, right? You have different right? You have different right? You have different moments when you moments when you moments when you make make make a transaction, like, a transaction, like, "Okay, I made this "Okay, I made this "Okay, I made this payment, but the amount was payment, but the amount was payment, but the amount was wrong or wrong or wrong or something." You have clear something." You have clear something." You have clear instructions on how instructions on how instructions on how to work with this. to work with this. to work with this. We don't have many of those We don't have many of those We don't have many of those opportunities right now when opportunities right now when opportunities right now when things go wrong, so things go wrong, so things go wrong, so I think intention will I think intention will I think intention will help us help us help us create them in the create them in the create them in the future. future.

and I think one and I think one recent incident recent incident recent incident highlights a situation highlights a situation highlights a situation where intentions can be where intentions can be where intentions can be good but actions good but actions good but actions can be can be can be wrong—this is the wrong—this is the wrong—this is the Resi incident that Resi incident that Resi incident that occurred recently, occurred recently, occurred recently, where an Instinct agent where an Instinct agent tried tried to book a to book a to book a restaurant table on resi.com on behalf of a user. Um, restaurant table on resi.com on behalf of a user. Um, restaurant table on resi.com on behalf of a user. Um, that agent was too that agent was too that agent was too diligent. He was making diligent. He was making diligent. He was making 200 API requests per 200 API requests per 200 API requests per hour, which led to the hour, which led to the hour, which led to the user's account user's account user's account being blocked on Resi. being blocked on Resi. Dick, how can AOTH budgets Dick, how can AOTH budgets help help help when such an agent when such an agent when such an agent uses uses uses too many resources, too many resources, too many resources, even though he has good intentions even though he has good intentions ? ? Yes. This is another Yes. This is another example of example of example of how much how much how much software has changed software has changed software has changed in the in the in the agent world, right? agent world, right? agent world, right? When this was a When this was a When this was a software requirement, 200 software requirement, 200 software requirement, 200 requests were considered requests were considered requests were considered a bug that needed to be a bug that needed to be a bug that needed to be fixed. And fixed. And fixed. And now the agent says, "I'll keep now the agent says, "I'll keep now the agent says, "I'll keep trying until I trying until I trying until I succeed." We succeed." We succeed." We saw this while saw this while saw this while working with AOTH and working with AOTH and working with AOTH and realized that there was a realized that there was a realized that there was a big problem: big problem: big problem: agents learn something agents learn something agents learn something and start and start and start abusing the system abusing the system abusing the system without even without even without even realizing it was an realizing it was an realizing it was an abuse. They abuse. They abuse. They simply don't know any simply don't know any simply don't know any other way. There was other way. There was other way. There was no signal no signal no signal telling them, "Oh, I telling them, "Oh, I telling them, "Oh, I only have to use only have to use only have to use this many this many this many resources." Therefore, we resources." Therefore, we resources." Therefore, we added the concept of added the concept of added the concept of budgets, where you give budgets, where you give budgets, where you give the agent not only a token the agent not only a token the agent not only a token with a time limit with a time limit with a time limit to perform a certain to perform a certain to perform a certain action, but also a budget for action, but also a budget for action, but also a budget for the amount of the amount of the amount of resource usage. So they resource usage. So they resource usage. So they have access until the have access until the tokens expire or tokens expire or they run out of they run out of they run out of budget. That way,

budget. That way, budget. That way, when they when they when they come back after come back after come back after using up all their using up all their using up all their budget, you as the budget, you as the budget, you as the supervisor can supervisor can supervisor can decide, "Should I decide, "Should I decide, "Should I give this give this give this agent more agent more agent more budget?" budget?" budget?" Unlike traditional Unlike traditional request rate limiting, which simply request rate limiting, which simply cuts everything off, and then the cuts everything off, and then the cuts everything off, and then the question arises—what question arises—what question arises—what to do next? And he doesn't to do next? And he doesn't to do next? And he doesn't understand: "Why was I understand: "Why was I understand: "Why was I disconnected?" or something disconnected?" or something disconnected?" or something like that. Whereas in the like that. Whereas in the like that. Whereas in the case of a budget, case of a budget, case of a budget, the agent constantly the agent constantly the agent constantly receives receives receives feedback about feedback about feedback about the resources spent and knows the resources spent and knows the resources spent and knows how much budget how much budget how much budget he has left. he has left. he has left. So now he can So now he can So now he can work within work within work within budget, right? Just like budget, right? Just like a person would do: "Oh, a person would do: "Oh, this resource is limited, this resource is limited, this resource is limited, I'll be careful with its I'll be careful with its I'll be careful with its use." use." So how does the concept of So how does the concept of "brain "brain "brain vs. hands" architecture help vs. hands" architecture help vs. hands" architecture help us in a situation where us in a situation where us in a situation where the budget is almost the budget is almost the budget is almost exhausted or has already been exhausted or has already been exhausted or has already been exhausted? How does exhausted? How does exhausted? How does separating access separating access separating access between the "brain" and "hands," between the "brain" and "hands," as well as storing as well as storing as well as storing credentials credentials credentials separately from the agent in the separately from the agent in the separately from the agent in the model itself, model itself, model itself, help stop help stop the agent from using excessive resources? Because when we the agent from using excessive resources? Because when we get close to get close to get close to the limit, we can, the limit, we can, the limit, we can, for example, not for example, not for example, not give him give him give him another another another account.

account. account. Sorry, I didn't quite Sorry, I didn't quite Sorry, I didn't quite understand, there's a terrible understand, there's a terrible understand, there's a terrible echo here. So it's okay echo here. So it's okay , it , it applies to everyone, applies to everyone, applies to everyone, but in general, the but in general, the but in general, the "brain and "brain and "brain and hands" architecture that we're hands" architecture that we're hands" architecture that we're moving to in the moving to in the moving to in the industry is that industry is that industry is that we keep the we keep the we keep the credentials and credentials and credentials and access rights separate access rights separate access rights separate from the agent itself. from the agent itself. from the agent itself. So the agent no longer So the agent no longer So the agent no longer has full broad has full broad has full broad access like it does today. access like it does today. access like it does today. No API keys, which No API keys, which No API keys, which I know is an advantage of I know is an advantage of I know is an advantage of AOTH, as AOF allows AOTH, as AOF allows AOTH, as AOF allows agents to work agents to work agents to work without API keys. How does this without API keys. How does this without API keys. How does this help in help in help in the context of a resi incident the context of a resi incident when you have access to when you have access to it? Sorry, I it? Sorry, I it? Sorry, I still don't understand. still don't understand. still don't understand. Yes, I can Yes, I can Yes, I can connect. Yes, connect. Yes, connect. Yes, thank you. thank you. thank you. I understand, friend. I understand, friend. Generally speaking, I think if you Generally speaking, I think if you think of an incident think of an incident think of an incident that keeps that keeps that keeps coming to mind, coming to mind, coming to mind, like Pocket OS, where they like Pocket OS, where they like Pocket OS, where they found a random found a random found a random key on the disk, right? key on the disk, right? key on the disk, right? It is clear that agents are It is clear that agents are It is clear that agents are very focused. very focused. The point is, The point is, if we give these if we give these if we give these agents a "brain," which is a agents a "brain," which is a agents a "brain," which is a model and an LLM, and we have these model and an LLM, and we have these model and an LLM, and we have these "hands," which is "hands," which is "hands," which is we put them in a we put them in a we put them in a sandbox, and sandbox, and sandbox, and inside are the inside are the tools... the idea of ​​a tools... the idea of ​​a tools... the idea of ​​a sandbox is that sandbox is that sandbox is that you control the content of it you control the content of it you control the content of it , so the agent , so the agent , so the agent can't access the can't access the can't access the API, and if you have an API, and if you have an API, and if you have an outbound proxy outside the outbound proxy outside the outbound proxy outside the sandbox, that sandbox, that sandbox, that creates an creates an creates an authorization boundary, and it doesn't just authorization boundary, and it doesn't just authorization boundary, and it doesn't just help with help with help with translation between translation between translation between systems, right?

systems, right? This is where This is where Athos can exist and Athos can exist and Athos can exist and communicate with the communicate with the communicate with the outside world or outside world or outside world or perform perform perform request completions, request completions, request completions, but it also means but it also means but it also means you get you get you get the ability to the ability to the ability to enforce enforce enforce authorization at the authorization at the authorization at the outbound proxy level. And outbound proxy level. And outbound proxy level. And so I think that at the so I think that at the so I think that at the heart of this is a heart of this is a heart of this is a model—the dominant model—the dominant model—the dominant model that model that model that Anthropic, OpenAI, Anthropic, OpenAI, Anthropic, OpenAI, and many other and many other and many other labs are developing today—that labs are developing today—that labs are developing today—that allows us to allows us to allows us to get what we're get what we're get what we're after: how to get after: how to get after: how to get these systems these systems these systems to experiment, to experiment, to experiment, to try different to try different to try different things, and to do something things, and to do something things, and to do something unusual, which is their unusual, which is their unusual, which is their core value, core value, core value, while still having a while still having a while still having a deterministic deterministic deterministic outer layer that outer layer that outer layer that can can can implement implement implement mechanisms like mechanisms like mechanisms like authentication and authentication and authentication and other components that other components that other components that we want to see in we want to see in we want to see in our protocols. our protocols. our protocols. I think that I think that I think that distinction is distinction is distinction is valuable, don't you? valuable, don't you? valuable, don't you? Returning to the Returning to the Returning to the aforementioned aforementioned aforementioned forensics and forensics and forensics and the ability to separate the ability to separate the ability to separate components, I still components, I still components, I still believe that we still have believe that we still have believe that we still have a lot to a lot to a lot to learn, because it's not learn, because it's not learn, because it's not just one "brain", just one "brain", just one "brain", right? And it's actually right? And it's actually right? And it's actually very easy very easy very easy to build an to build an to build an agent loop where you agent loop where you agent loop where you put the "brain" put the "brain" put the "brain" inside the loop, not inside the loop, not inside the loop, not outside it. Isn't that right? outside it. Isn't that right? outside it. Isn't that right? Here's an example of this: Here's an example of this: Here's an example of this: people who people who people who use use use memory systems. They memory systems. They memory systems. They add some memory add some memory add some memory and context that is now and context that is now and context that is now used by used by used by each agent in each agent in each agent in the future, which the future, which the future, which prompts that prompts that prompts that agent to make agent to make agent to make a decision and order the " a decision and order the " hands", in this hands", in this hands", in this case, to do something that it case, to do something that it case, to do something that it shouldn't do.

shouldn't do. shouldn't do. So one of the things that So one of the things that So one of the things that I think we don't I think we don't I think we don't talk about talk about talk about enough here is enough here is enough here is toxic combinations, toxic combinations, toxic combinations, right? And so when right? And so when right? And so when you separate the "hands" you separate the "hands" you separate the "hands" and the "brain," you can and the "brain," you can and the "brain," you can actually change actually change actually change which "hands" which "hands" which "hands" are available at a given are available at a given are available at a given moment, right? So, an moment, right? So, an moment, right? So, an example that I've example that I've example that I've seen that is very seen that is very seen that is very successful is successful is successful is dynamically removing dynamically removing dynamically removing tools based tools based tools based on the activity of the "brain," on the activity of the "brain," on the activity of the "brain," like in the email reading example mentioned earlier like in the email reading example mentioned earlier . . There's nothing wrong with There's nothing wrong with There's nothing wrong with saying, "Oh, if he's saying, "Oh, if he's saying, "Oh, if he's going to write a going to write a going to write a letter, I'll turn off all letter, I'll turn off all letter, I'll turn off all calls to calls to calls to web search tools or web search tools or any other external any other external any other external access access access until the until the until the letter-writing task is letter-writing task is letter-writing task is done, and done, and done, and then I can then I can then I can turn them back on," turn them back on," turn them back on," right? So I think right? So I think right? So I think we're going to have to we're going to have to take an take an engineering approach more and more to engineering approach more and more to engineering approach more and more to these different concepts, these different concepts, these different concepts, which, while which, while which, while still still still complex, are still complex, are still complex, are still evolving and evolving and evolving and are in their are in their are in their early stages, but I early stages, but I early stages, but I think it's a better think it's a better think it's a better way of distinguishing way of distinguishing way of distinguishing than what we have than what we have than what we have today. Yes, and that's how today. Yes, and that's how today. Yes, and that's how these things that these things that these things that we're talking about that we're talking about that we're talking about that seemed like purely seemed like purely seemed like purely technical technical technical problems problems problems actually actually actually turn into turn into turn into real engineering real engineering real engineering challenges for agents: challenges for agents: challenges for agents: if you understand what you if you understand what you if you understand what you 're doing, one of the 're doing, one of the 're doing, one of the biggest complaints about biggest complaints about biggest complaints about MCP—although less so now MCP—although less so now MCP—although less so now than before—was than before—was context overload, especially context overload, especially when we had smaller when we had smaller when we had smaller context windows, context windows, context windows, right? But right? But right? But intent systems can intent systems can intent systems can ensure that the ensure that the ensure that the agent is given the right agent is given the right agent is given the right capabilities at the capabilities at the capabilities at the right time, so that right time, so that right time, so that they don't waste they don't waste they don't waste a bunch of time doing things they a bunch of time doing things they a bunch of time doing things they shouldn't be doing, shouldn't be doing, shouldn't be doing, right? They help right? They help right? They help them move towards a them move towards a them move towards a goal state and goal state and goal state and guide them onto a guide them onto a guide them onto a more well-trodden more well-trodden more well-trodden path, so intentions not path, so intentions not path, so intentions not only open up only open up only open up a bunch of opportunities for a bunch of opportunities for a bunch of opportunities for security. Intents will also security. Intents will also security. Intents will also lead to lead to lead to less less less token consumption, as the token consumption, as the token consumption, as the agent will spend agent will spend agent will spend less time on actions that less time on actions that less time on actions that do not lead to the do not lead to the do not lead to the correct correct correct result, or he will result, or he will result, or he will say in time, "Hey, I say in time, "Hey, I say in time, "Hey, I just can't just can't just can't do that." So, we do that." So, we do that." So, we know that this era is know that this era is know that this era is largely about

largely about nudging the agent nudging the agent in the right in the right in the right direction and direction and direction and ensuring that they ensuring that they ensuring that they don't do anything don't do anything don't do anything unnecessary, and that's how you unnecessary, and that's how you unnecessary, and that's how you get the desired get the desired get the desired outcome. outcome. Davis, how do you Davis, how do you view the view the view the intent-based approach as intent-based approach as intent-based approach as part of a broader part of a broader part of a broader strategy for strategy for strategy for product agents? product agents? product agents? I know that when it I know that when it I know that when it comes to comes to comes to building, building, building, deploying, and deploying, and deploying, and operating agents operating agents operating agents in a production in a production in a production environment, teams environment, teams environment, teams need to think about need to think about need to think about many outcomes: many outcomes: many outcomes: discovering what discovering what discovering what works, works, works, agent capabilities, the ability agent capabilities, the ability agent capabilities, the ability to inspect input and to inspect input and to inspect input and output data, output data, output data, agent constraints as agent constraints as agent constraints as we discussed, we discussed, we discussed, and the use of and the use of and the use of outbound proxies and outbound proxies and outbound proxies and various network various network various network rules. Um, how does all this rules. Um, how does all this rules. Um, how does all this fit together and what fit together and what fit together and what actually works in actually works in actually works in corporations today corporations today to successfully to successfully to successfully launch agents into launch agents into launch agents into products? Yes, I products? Yes, I products? Yes, I think we're finally think we're finally think we're finally moving from moving from moving from talking about talking about talking about authentication to authentication to authentication to authorization and, in fact, authorization and, in fact, to the question: how do we to the question: how do we to the question: how do we implement implement implement dynamic dynamic dynamic authorization? How do I authorization? How do I authorization? How do I verify this verify this verify this authorization? How do I authorization? How do I authorization? How do I engage a person in the engage a person in the engage a person in the process at the right process at the right process at the right moment? Because I agree with moment? Because I agree with moment? Because I agree with what Dick said what Dick said what Dick said earlier: a person in a earlier: a person in a earlier: a person in a cycle is in many ways cycle is in many ways cycle is in many ways an anti-pattern, isn't an anti-pattern, isn't an anti-pattern, isn't it? Because she may it? Because she may it? Because she may feel tired, etc. feel tired, etc. feel tired, etc. But you need a But you need a But you need a way to engage her way to engage her way to engage her when it really when it really when it really matters. These matters. These matters. These weren't engineering weren't engineering weren't engineering discussions before, right?

discussions before, right? discussions before, right? They are now part of the They are now part of the They are now part of the engineering conversation, engineering conversation, engineering conversation, where developers where developers where developers are involved in the are involved in the are involved in the authorization process. authorization process. authorization process. They no longer They no longer They no longer just say, "Oh, I got just say, "Oh, I got just say, "Oh, I got access. I'm ready to access. I'm ready to access. I'm ready to go." So this go." So this go." So this actually changes the actually changes the actually changes the approach to approach to approach to threat modeling. threat modeling. threat modeling. This changes our This changes our This changes our view of the full view of the full view of the full attack surface for attack surface for attack surface for applications in general. applications in general. applications in general. So, I guess now So, I guess now So, I guess now you're looking at this you're looking at this you're looking at this authorization model authorization model authorization model as another set of as another set of as another set of tools that you tools that you tools that you can use can use can use to reduce not only to reduce not only to reduce not only security issues but security issues but security issues but also the number of errors also the number of errors also the number of errors your agent makes, your agent makes, your agent makes, right? Well, I mean right? Well, I mean , when an agent has fewer , when an agent has fewer , when an agent has fewer options, they options, they options, they make better make better make better decisions, right? And decisions, right? And decisions, right? And so, essentially, we've so, essentially, we've so, essentially, we've seen real-world seen real-world seen real-world cases where an cases where an cases where an agent's workflow becomes agent's workflow becomes agent's workflow becomes better, faster, and more better, faster, and more better, faster, and more reliable because reliable because reliable because you've taken the time you've taken the time you've taken the time to understand what to understand what to understand what goal it's trying goal it's trying goal it's trying to achieve, what task to achieve, what task to achieve, what task it's performing. And it's performing. And it's performing. And lastly, it brings lastly, it brings lastly, it brings people together because now people together because now people together because now we have different teams we have different teams —a security team, an —a security team, an application development team. Now application development team. Now they can they can they can communicate communicate communicate using the same using the same using the same language. And language. And language. And before, when it before, when it before, when it came to came to came to identification, it identification, it identification, it was a completely different language was a completely different language , wasn't it? And it , wasn't it? And it , wasn't it? And it sounded like "don't sounded like "don't sounded like "don't bother me." What do bother me." What do bother me." What do I need to I need to I need to do to get do to get do to get my token and move my token and move my token and move on? Now they on? Now they on? Now they see that it is actually see that it is actually see that it is actually possible to combine these possible to combine these possible to combine these things and things and things and use this use this use this potential to potential to potential to improve the improve the improve the efficiency of efficiency of your agents' work results.

your agents' work results. That makes sense. Ian, how do That makes sense. Ian, how do That makes sense. Ian, how do you apply these you apply these you apply these concepts to concepts to concepts to software factories? I software factories? I software factories? I know that today you know that today you know that today you released a reference released a reference released a reference architecture for a architecture for a architecture for a secure software secure software secure software factory. How do you factory. How do you factory. How do you apply apply apply the concepts of the concepts of the concepts of runtime authorization runtime authorization runtime authorization , , , agent constraints, and all that stuff agent constraints, and all that stuff agent constraints, and all that stuff to this? to this? to this? Yes, I think Yes, I think Yes, I think we we we just published a just published a just published a reference architecture on agentbaseline.org reference architecture on agentbaseline.org . We talked about three . We talked about three . We talked about three different levels of the factory. different levels of the factory. The first is The first is human-driven, the second is human-driven, the second is workflow-driven workflow-driven workflow-driven , and the third is , and the third is agent-driven, which is what we agent-driven, which is what we agent-driven, which is what we all essentially aspire to all essentially aspire to , where everything is constantly , where everything is constantly , where everything is constantly driven by agents. driven by agents. driven by agents. Runtime authorization Runtime authorization Runtime authorization is a key is a key is a key component of the broader component of the broader component of the broader ecosystem. It ecosystem. It ecosystem. It actually allows actually allows actually allows the factory to direct the factory to direct the factory to direct work in the right work in the right work in the right direction, perform the direction, perform the direction, perform the necessary actions, and necessary actions, and necessary actions, and access the access the access the necessary resources. necessary resources. necessary resources. And it looks different And it looks different And it looks different than previous than previous than previous tools that you've tools that you've tools that you've used, like used, like secret managers, because secret managers, because it makes very it makes very it makes very different decisions than different decisions than different decisions than previous generations previous generations previous generations for managing for managing for managing deterministic software. deterministic software. deterministic software. But this is a key But this is a key But this is a key component for the component for the component for the functioning of the entire functioning of the entire functioning of the entire system: when system: when system: when to involve a person in the to involve a person in the to involve a person in the process, which agents process, which agents process, which agents can delegate can delegate can delegate work to other agents.

work to other agents. work to other agents. Do you know how to make Do you know how to make Do you know how to make sure that the agent has sure that the agent has sure that the agent has access to production access to production access to production systems and can systems and can systems and can perform perform perform automatic automatic automatic patch updates? Um, patch updates? Um, patch updates? Um, these are fundamental these are fundamental these are fundamental questions about how we questions about how we questions about how we actually actually actually develop develop software at software at scale. scale. scale. We are now in a We are now in a We are now in a position where we position where we position where we could never could never could never generate more software. generate more software. generate more software. But we never...we have a But we never...we have a But we never...we have a complete jam with complete jam with production and production and job security job security . And this is...the next stage . And this is...the next stage : how do you move from : how do you move from : how do you move from using CMM using CMM using CMM simply to generate simply to generate simply to generate text or code to text or code to text or code to the ability to deploy the ability to deploy the ability to deploy it, improve it, it, improve it, it, improve it, quickly stabilize it, quickly stabilize it, quickly stabilize it, and respond to and respond to and respond to incidents? incidents? Davis, in our Davis, in our podcast we talked podcast we talked podcast we talked about what about what about what cybersecurity can cybersecurity can cybersecurity can learn from learn from learn from robotics, robotics, robotics, like when a robot like when a robot like when a robot raises its hand. He raises its hand. He raises its hand. He thinks: is it safe for thinks: is it safe for thinks: is it safe for me to raise my hand? me to raise my hand? me to raise my hand? Should I Should I Should I do this? Am do this? Am do this? Am I allowed to do this? But he also I allowed to do this? But he also I allowed to do this? But he also looks at the external looks at the external looks at the external environment and asks, environment and asks, environment and asks, "What has changed? Is it "What has changed? Is it "What has changed? Is it still safe for me still safe for me still safe for me to raise my hand?" When to raise my hand?" When to raise my hand?" When we think about we think about we think about runtime authorization runtime authorization runtime authorization , what else, , what else, , what else, besides intent, should besides intent, should besides intent, should these decisions take into account these decisions take into account these decisions take into account ? ? ? Yes, I think this kind of Yes, I think this kind of Yes, I think this kind of model allows us model allows us model allows us to add more to add more to add more sensors to our sensors to our decision-making process. Here's an example with decision-making process. Here's an example with robotics: moving robotics: moving robotics: moving the arm up or down, the arm up or down, the arm up or down, any external any external any external factors, like factors, like factors, like are there other are there other are there other objects nearby, right? It could objects nearby, right? It could objects nearby, right? It could be the temperature, or be the temperature, or maybe I'm moving myself and I do maybe I'm moving myself and I do maybe I'm moving myself and I do n't want to move my hand n't want to move my hand n't want to move my hand while the rest of the while the rest of the while the rest of the robot is moving. So, robot is moving. So, robot is moving. So, he usually he usually he usually takes into account a lot of takes into account a lot of takes into account a lot of different things. None of this different things. None of this different things. None of this is deterministic; These are is deterministic; These are is deterministic; These are all probabilistic all probabilistic all probabilistic things. So there's a range of things. So there's a range of things. So there's a range of options, and it options, and it options, and it chooses the best one chooses the best one chooses the best one based on all the data from the based on all the data from the based on all the data from the sensors. Currently,

sensors. Currently, sensors. Currently, most most most authorization solutions in authorization solutions in authorization solutions in organizations are organizations are organizations are binary, right? And that binary, right? And that binary, right? And that just won't just won't just won't work in work in work in the future, right? So it's the future, right? So it's the future, right? So it's important to match important to match important to match the user's intent with the user's intent with the user's intent with the ecosystem in which the ecosystem in which the ecosystem in which they do it. This could they do it. This could they do it. This could be location be location be location , time of day , time of day , or more complex things: , or more complex things: , or more complex things: for example, should for example, should an administrator be allowed to an administrator be allowed to use Firefox use Firefox use Firefox to access a to access a to access a server that has a server that has a server that has a high level of high level of high level of vulnerabilities and its vulnerabilities and its vulnerabilities and its system is not up to date? system is not up to date? Yes? Now you're suddenly Yes? Now you're suddenly adding a lot adding a lot adding a lot more context to the more context to the environment, and environment, and again that can again that can again that can happen through happen through happen through different commands, but more different commands, but more different commands, but more importantly, through importantly, through importantly, through sensors, and I think we've sensors, and I think we've sensors, and I think we've all seen the all seen the all seen the value of value of value of observability observability observability in general. Imagine if you in general. Imagine if you in general. Imagine if you could have this kind of could have this kind of could have this kind of context at the context at the context at the span level from an Otel perspective span level from an Otel perspective span level from an Otel perspective or something like that: or something like that: or something like that: you look at you look at you look at the trace, you have the the trace, you have the the trace, you have the span, and you actually span, and you actually span, and you actually know what know what know what the environment outside the environment outside the environment outside your application was like at that your application was like at that your application was like at that time, and also what the time, and also what the user's intent was. This user's intent was. This allows you allows you allows you to diagnose things in a to diagnose things in a to diagnose things in a completely different completely different completely different way. And I think way. And I think way. And I think if we can if we can if we can implement some of implement some of implement some of these concepts, we'll these concepts, we'll these concepts, we'll get faster get faster authorization solutions that authorization solutions that have a lot have a lot have a lot more context than more context than more context than just just just user intent. This will be the user intent. This will be the user intent. This will be the state of the state of the state of the environment.

environment. That makes sense. And That makes sense. And looking ahead, looking ahead, looking ahead, assessing whether assessing whether intent-based authorization is working in intent-based authorization is working in our environments, our environments, our environments, one natural one natural one natural place to check place to check place to check would be the agent action log would be the agent action log would be the agent action log . What was . What was . What was rejected, what was rejected, what was rejected, what was allowed. Yana, why is it allowed. Yana, why is it allowed. Yana, why is it important that your important that your important that your authorization log authorization log authorization log is separate from the is separate from the is separate from the agent itself, i.e. outside agent itself, i.e. outside agent itself, i.e. outside its control? its control? its control? Yeah, I think this also Yeah, I think this also Yeah, I think this also goes back to goes back to goes back to why the point of why the point of enforcement should be enforcement should be outside the agent's primary " outside the agent's primary " hands and brains hands and brains hands and brains ," right? You ," right? You ," right? You can't trust this can't trust this can't trust this thing. So, the thing. So, the thing. So, the Hugging Face incident had several Hugging Face incident had several Hugging Face incident had several different components. different components. different components. I mean, one was I mean, one was I mean, one was clearly exploiting a clearly exploiting a clearly exploiting a vulnerability that vulnerability that vulnerability that we didn't know about in the we didn't know about in the Squid proxy server, but more Squid proxy server, but more Squid proxy server, but more importantly, you had no importantly, you had no observability or observability or monitoring of what was monitoring of what was monitoring of what was happening. So this happening. So this happening. So this thing was able to get out into the thing was able to get out into the thing was able to get out into the open internet open internet open internet because of this vulnerability, because of this vulnerability, because of this vulnerability, and there was no place where the and there was no place where the and there was no place where the various various various actions that actions that actions that this agent was taking were recorded and where they could be this agent was taking were recorded and where they could be tracked. And so whether we apply an we apply an enforcement point enforcement point enforcement point to to to network traffic going network traffic going network traffic going to the internet, to the internet, to the internet, what agent what agent what agent is trying to change, is trying to change, is trying to change, or we address or we address or we address vulnerabilities as vulnerabilities as vulnerabilities as they they they emerge. You know, emerge. You know, emerge. You know, this case with Hugging Face this case with Hugging Face this case with Hugging Face is a bit unique.

is a bit unique. is a bit unique. We still need to We still need to We still need to have a record of who have a record of who have a record of who told this agent told this agent told this agent to do a certain thing and for to do a certain thing and for to do a certain thing and for what purpose, and is that what purpose, and is that what purpose, and is that still relevant? Because still relevant? Because still relevant? Because ultimately, what ultimately, what ultimately, what we do with agents we do with agents we do with agents is a matter of is a matter of is a matter of responsibility. And that's why responsibility. And that's why responsibility. And that's why I do I do I do n't really think we n't really think we n't really think we need more need more need more regulation on regulation on regulation on agents. We just agents. We just agents. We just need our need our need our current liability laws to be current liability laws to be applied and applied and enforced, namely: enforced, namely: enforced, namely: if I hire an agent if I hire an agent if I hire an agent to perform a certain to perform a certain to perform a certain action, who is action, who is action, who is liable liable liable if that action causes if that action causes if that action causes harm, right? This is the person harm, right? This is the person harm, right? This is the person who hires him - it who hires him - it who hires him - it could be an could be an could be an organization, a person, organization, a person, organization, a person, or someone who or someone who or someone who has delegated certain actions to something has delegated certain actions to something has delegated certain actions to something else. This is, in else. This is, in else. This is, in fact, the final fact, the final fact, the final question that we will have to question that we will have to deal with. And then, deal with. And then, of course, you have to of course, you have to of course, you have to be able to be able to be able to answer that answer that answer that question: "Well, who question: "Well, who question: "Well, who did it?" This is where the did it?" This is where the tamper-proof tamper-proof automatic automatic automatic event log comes from. event log comes from. event log comes from. Great, and to Great, and to Great, and to conclude, I wanted to conclude, I wanted to conclude, I wanted to ask you all an ask you all an ask you all an interesting question. interesting question. interesting question. So there is the concept of So there is the concept of So there is the concept of probability of probability of probability of catastrophe, or pdoom, catastrophe, or pdoom, catastrophe, or pdoom, where AI pessimists where AI pessimists where AI pessimists say that AI agents say that AI agents say that AI agents will cause chaos or will cause chaos or will cause chaos or be the doom of be the doom of be the doom of humanity; whereas AI humanity; whereas AI optimists say, "No, optimists say, "No, optimists say, "No, it's really ready it's really ready it's really ready to help us, it's not a to help us, it's not a to help us, it's not a real threat." real threat."

For each of you on For each of you on this panel, what are your this panel, what are your this panel, what are your thoughts on this? Is thoughts on this? Is intent-based runtime authorization and the other things intent-based runtime authorization and the other things we talk about, we talk about, we talk about, like like like agent constraints, agent constraints, agent constraints, input and input and input and output validation, a holistic output validation, a holistic output validation, a holistic strategy for strategy for strategy for productive agents productive agents ? Is this enough ? Is this enough ? Is this enough to save us? Is the to save us? Is the to save us? Is the "pdoom" threat real, "pdoom" threat real, "pdoom" threat real, or is it not worth the or is it not worth the or is it not worth the hype? hype? You are starting. You are starting. Oh, I did. Oh, I did. Oh, I did. You have such a cunning You have such a cunning You have such a cunning expression on your face. Well, expression on your face. Well, expression on your face. Well, part of your part of your part of your question is question is question is , will , will , will the agents seize power? And I the agents seize power? And I the agents seize power? And I think agents that think agents that think agents that are completely are completely are completely autonomous and not autonomous and not autonomous and not tied to a tied to a tied to a specific specific specific organization — that's organization — that's organization — that's still science still science still science fiction, you know, fiction, you know, fiction, you know, so who knows, right? This is so who knows, right? This is so who knows, right? This is one of the directions. one of the directions. one of the directions. Another is when the Another is when the Another is when the responsible responsible responsible organization organization organization implements the right implements the right implements the right controls; controls; controls; agents are not...we agents are not...we agents are not...we develop develop develop enough enough enough tools to tools to tools to make sure that make sure that make sure that bad things don't bad things don't bad things don't happen if the happen if the happen if the organization has organization has organization has proper proper proper control mechanisms in place. So, there are control mechanisms in place. So, there are control mechanisms in place. So, there are two two two scenarios left where a scenarios left where a scenarios left where a potential problem is possible. potential problem is possible. potential problem is possible. The first is an The first is an The first is an organization where organization where organization where chaos reigns, and an internal chaos reigns, and an internal chaos reigns, and an internal threat could threat could threat could lead to lead to lead to someone releasing an someone releasing an someone releasing an agent to agent to agent to cause chaos, right?

cause chaos, right? cause chaos, right? So they have special So they have special So they have special access that access that access that other people don't have. Another other people don't have. Another other people don't have. Another option that option that option that worries me more is what has worries me more is what has worries me more is what has already been already been already been demonstrated, demonstrated, demonstrated, for example, on Hugging Face. for example, on Hugging Face. for example, on Hugging Face. And if you look at the And if you look at the And if you look at the keynote they keynote they keynote they gave at Black Hat, it was a gave at Black Hat, it was a gave at Black Hat, it was a real call to real call to real call to action, saying, here's action, saying, here's action, saying, here's a situation where we didn't a situation where we didn't a situation where we didn't want want want something bad to happen, and it did something bad to happen, and it did something bad to happen, and it did . . . What about people who What about people who What about people who want want want something bad to happen and have something bad to happen and have something bad to happen and have these tools? I these tools? I these tools? I see this as a threat see this as a threat see this as a threat that could potentially that could potentially that could potentially lead to—I lead to—I lead to—I mean, it will happen, and there mean, it will happen, and there mean, it will happen, and there will be some will be some will be some disasters. disasters. Hopefully, we are prepared enough that prepared enough that they are not terribly they are not terribly they are not terribly destructive, but there will destructive, but there will destructive, but there will still be things of the still be things of the still be things of the scale of natural scale of natural scale of natural disasters caused by malicious disasters caused by malicious disasters caused by malicious actors— actors— intentionally or intentionally or intentionally or unintentionally; This unintentionally; This unintentionally; This happens when happens when happens when people people use these use these super-powerful super-powerful super-powerful tools irresponsibly. tools irresponsibly.

Yes, I can't name Yes, I can't name any specific any specific any specific probability, of course. probability, of course. probability, of course. But what I can say is that But what I can say is that But what I can say is that I've I've I've always been amazed and always been amazed and always been amazed and fascinated by the fascinated by the fascinated by the ingenuity of ingenuity of ingenuity of engineers in engineers in engineers in solving problems, and solving problems, and solving problems, and I think we're I think we're I think we're at a point now where we have at a point now where we have at a point now where we have new capabilities, new capabilities, new capabilities, like in agents like in agents like in agents and so on. We have real and so on. We have real and so on. We have real engineers, people who are engineers, people who are engineers, people who are just working on just working on just working on this problem, this problem, this problem, right? We are starting to right? We are starting to right? We are starting to see tremendous see tremendous see tremendous progress in what progress in what progress in what I believe will become I believe will become I believe will become available to change available to change available to change our future. our future. our future. So I have no problem So I have no problem So I have no problem believing that engineers believing that engineers believing that engineers will find a solution here. will find a solution here. will find a solution here. Perhaps my opinion is the Perhaps my opinion is the Perhaps my opinion is the most radical. most radical. most radical. I can't say it's I can't say it's I can't say it's zero, zero, zero, but I would say but I would say but I would say it's asymptotically it's asymptotically it's asymptotically approaching zero. approaching zero. approaching zero. It's about It's about It's about the possibility of the end of the the possibility of the end of the the possibility of the end of the world. I think the world. I think the world. I think the incentive system for incentive system for incentive system for most people is to most people is to keep living. keep living. Besides, after all, we Besides, after all, we Besides, after all, we have a legal framework of have a legal framework of have a legal framework of responsibility, and I responsibility, and I responsibility, and I think people are generally think people are generally good beings, so we good beings, so we good beings, so we don't don't don't specifically seek to destroy the specifically seek to destroy the specifically seek to destroy the world. I also believe world. I also believe world. I also believe that in general these that in general these that in general these agents—I have agents—I have agents—I have this bold idea— this bold idea— they can simply be they can simply be they can simply be turned off, and turned off, and turned off, and it seems to be just true. it seems to be just true.

it seems to be just true. After all, these are After all, these are After all, these are computers and we computers and we computers and we can just can just can just turn them off. So, turn them off. So, turn them off. So, generally speaking, I generally speaking, I generally speaking, I don't believe in this debate. don't believe in this debate. don't believe in this debate. I don't think she's I don't think she's I don't think she's real. I think real. I think real. I think on the positive side, I on the positive side, I on the positive side, I lean more towards lean more towards lean more towards Davis's position: we Davis's position: we are developing very quickly. We're are developing very quickly. We're figuring out where these figuring out where these figuring out where these things can things can things can work and where they can't. And work and where they can't. And work and where they can't. And when they don't work when they don't work , they cause harm. And , they cause harm. And , they cause harm. And these are, ultimately, a these are, ultimately, a these are, ultimately, a kind of antibodies kind of antibodies kind of antibodies of society that of society that of society that say, "Hey, maybe say, "Hey, maybe say, "Hey, maybe we shouldn't we shouldn't we shouldn't do this." Like, this is do this." Like, this is do this." Like, this is probably a bad idea. probably a bad idea. probably a bad idea. So, I'm the pessimist here So, I'm the pessimist here So, I'm the pessimist here on this panel. on this panel. on this panel. Yes. This is unfortunate for Yes. This is unfortunate for Yes. This is unfortunate for people like the people like the people like the founder of Pocket OS, who founder of Pocket OS, who founder of Pocket OS, who lost a bunch of data. lost a bunch of data. lost a bunch of data. But it's good for the But it's good for the But it's good for the rest of us because we're all rest of us because we're all rest of us because we're all learning together. And learning together. And learning together. And ultimately, it ultimately, it ultimately, it leads to leads to leads to people not doing people not doing people not doing things that don't make things that don't make things that don't make much sense, and I much sense, and I much sense, and I think that will continue to be the case think that will continue to be the case think that will continue to be the case . . . Logically. Thank you to Logically. Thank you to Logically. Thank you to each of you for your each of you for your each of you for your expertise and time on expertise and time on expertise and time on this this this panel, thank you to panel, thank you to panel, thank you to the audience for your attention, and the audience for your attention, and the audience for your attention, and see you see you see you next time. next time. next time. Thank you. Thank you, Thank you. Thank you, Thank you. Thank you, Marshall.