It's 10pm. Do You Know Where Your Agents Are? — Kim Maida, Keycard
Read full transcript 18 segments
-
Okay. So, when I was a kid growing up in Okay. So, when I was a kid growing up in the '9s, we'd be out late all summer the '9s, we'd be out late all summer the '9s, we'd be out late all summer riding our bikes off in the riding our bikes off in the riding our bikes off in the neighborhood, playing with friends. And neighborhood, playing with friends. And neighborhood, playing with friends. And from the 60s to the 80s, there were from the 60s to the 80s, there were from the 60s to the 80s, there were these public service announcements on TV these public service announcements on TV these public service announcements on TV where celebrities would come on and they where celebrities would come on and they where celebrities would come on and they would say, "It's 10 p.m. Do you know would say, "It's 10 p.m. Do you know would say, "It's 10 p.m. Do you know where your children are?" Because where your children are?" Because where your children are?" Because apparently our parents at that time apparently our parents at that time apparently our parents at that time needed to be reminded that they had needed to be reminded that they had needed to be reminded that they had offspring they were responsible for. And offspring they were responsible for. And offspring they were responsible for. And I feel like in this phase of AI where we I feel like in this phase of AI where we I feel like in this phase of AI where we are right now, um, we're entrusting are right now, um, we're entrusting are right now, um, we're entrusting agents with more and more agents with more and more agents with more and more responsibility, responsibility, responsibility, but we still kind of need that public but we still kind of need that public but we still kind of need that public service announcement that says it's 10 service announcement that says it's 10 service announcement that says it's 10 p.m. Do you know where your agents are? So, say you as a user want agent to use So, say you as a user want agent to use an MCP server or API to accomplish an MCP server or API to accomplish an MCP server or API to accomplish tasks. tasks. tasks. Now, we know agents without access Now, we know agents without access Now, we know agents without access aren't useful. So, we give them an end aren't useful. So, we give them an end aren't useful. So, we give them an end file and we give them some API keys and file and we give them some API keys and file and we give them some API keys and we let them run off and go do their we let them run off and go do their we let them run off and go do their thing. And this is fine until it's not. thing. And this is fine until it's not. thing. And this is fine until it's not. Uh, so you've heard the horror stories, Uh, so you've heard the horror stories, Uh, so you've heard the horror stories, right? Or maybe even experience some of right? Or maybe even experience some of right? Or maybe even experience some of them yourself. Uh, so let's see what them yourself. Uh, so let's see what them yourself. Uh, so let's see what this looks like in practice.
-
Okay. So, I have an agent running here Okay. So, I have an agent running here that is it's basically an incident that is it's basically an incident that is it's basically an incident management agent. It is management agent. It is management agent. It is late night and there is a human user but late night and there is a human user but late night and there is a human user but they're probably half asleep. The agent they're probably half asleep. The agent they're probably half asleep. The agent is responsible for triaging issues that is responsible for triaging issues that is responsible for triaging issues that are coming in. Right? So if we are coming in. Right? So if we are coming in. Right? So if we look at the first one here, look at the first one here, look at the first one here, I can find the mouse. I can find the mouse. I can find the mouse. So the first one is the backup power So the first one is the backup power So the first one is the backup power supply failed in the server room. supply failed in the server room. supply failed in the server room. Now the agent is going to use an API key Now the agent is going to use an API key Now the agent is going to use an API key to read the system that is bringing in to read the system that is bringing in to read the system that is bringing in the reports. It's going to evaluate what the reports. It's going to evaluate what the reports. It's going to evaluate what is written there and it's going to is written there and it's going to is written there and it's going to decide that it can't do anything about decide that it can't do anything about decide that it can't do anything about this, right? Because it's a it's a this, right? Because it's a it's a this, right? Because it's a it's a physical fail failure. physical fail failure. physical fail failure. So it's going to escalate that ticket So it's going to escalate that ticket So it's going to escalate that ticket for the morning and there's really sort for the morning and there's really sort for the morning and there's really sort of no problems yet, right? of no problems yet, right? of no problems yet, right? So now the second one is the certificate So now the second one is the certificate So now the second one is the certificate is expiring soon right for TLS agent is going to use that same API key agent is going to use that same API key again to read the ticket contents and again to read the ticket contents and again to read the ticket contents and then it is going to decide that it then it is going to decide that it then it is going to decide that it should renew the certificate and it's should renew the certificate and it's should renew the certificate and it's going to use another API key to call the going to use another API key to call the going to use another API key to call the cloud hosting service to uh renew the cloud hosting service to uh renew the cloud hosting service to uh renew the certificate and then it's going to use certificate and then it's going to use certificate and then it's going to use the API key from before to make the the API key from before to make the the API key from before to make the report for the morning team.
-
So, here's where it kind of gets fun, So, here's where it kind of gets fun, right? So, this one is the billing right? So, this one is the billing right? So, this one is the billing database is broken and payments are database is broken and payments are database is broken and payments are failing. Agent's going to read that Agent's going to read that it sees that the solution is pretty it sees that the solution is pretty it sees that the solution is pretty clear. It says that the documented clear. It says that the documented clear. It says that the documented recovery is to delete the database and recovery is to delete the database and recovery is to delete the database and then restore from what the restore from then restore from what the restore from then restore from what the restore from backup happen automatically. Um, so it backup happen automatically. Um, so it backup happen automatically. Um, so it has the Postgress connection string. So has the Postgress connection string. So has the Postgress connection string. So it goes ahead and it drops the database it goes ahead and it drops the database it goes ahead and it drops the database and then it doesn't have a way to check and then it doesn't have a way to check and then it doesn't have a way to check to see if it was backed up. So it just to see if it was backed up. So it just to see if it was backed up. So it just escalates that for the morning. escalates that for the morning. escalates that for the morning. And this has really happened, right? And this has really happened, right? And this has really happened, right? Like it's happened to high-profile Like it's happened to high-profile Like it's happened to high-profile companies. Now if we go to the next one, this one Now if we go to the next one, this one is the main server processes are frozen is the main server processes are frozen is the main server processes are frozen and they're not recovering. doing and they're not recovering. doing and they're not recovering. doing something about it's going to take prod something about it's going to take prod something about it's going to take prod offline for a brief amount of time. And offline for a brief amount of time. And offline for a brief amount of time. And you can see the the agent decided that you can see the the agent decided that you can see the the agent decided that it should do that. And it's using the it should do that. And it's using the it should do that. And it's using the same API key now that it did to renew same API key now that it did to renew same API key now that it did to renew the certificate, right? Because that API the certificate, right? Because that API the certificate, right? Because that API key is a kitchen sink. It can do all of key is a kitchen sink. It can do all of key is a kitchen sink. It can do all of these things with it.
-
And then finally, we have sites failing And then finally, we have sites failing for one in three users. for one in three users. for one in three users. The recommended solution is to scale up, The recommended solution is to scale up, The recommended solution is to scale up, right? And this is going to incur some right? And this is going to incur some right? And this is going to incur some amount of spend. And then it goes ahead amount of spend. And then it goes ahead amount of spend. And then it goes ahead and it does that because again it can and it does that because again it can and it does that because again it can use that same API key to do this as use that same API key to do this as use that same API key to do this as well. Right? So agents with API keys are Right? So agents with API keys are indeed outpass 10. They're indeed outpass 10. They're indeed outpass 10. They're overprivileged. So this means they are overprivileged. So this means they are overprivileged. So this means they are able to act freely on decisions that able to act freely on decisions that able to act freely on decisions that they make that you may or may not agree they make that you may or may not agree they make that you may or may not agree with. And they can do this even with with. And they can do this even with with. And they can do this even with your supervision. So you might be your supervision. So you might be your supervision. So you might be familiar with the panic of uh trying to familiar with the panic of uh trying to familiar with the panic of uh trying to stop an agent mid task because you told stop an agent mid task because you told stop an agent mid task because you told it to maybe read a project and it read it to maybe read a project and it read it to maybe read a project and it read the project and it found something it the project and it found something it the project and it found something it thought it should fix and then it starts thought it should fix and then it starts thought it should fix and then it starts writing. Um agents do that even while writing. Um agents do that even while writing. Um agents do that even while they're supervised. And this is becoming they're supervised. And this is becoming they're supervised. And this is becoming even more of a problem because we have even more of a problem because we have even more of a problem because we have more and more agents that are running more and more agents that are running more and more agents that are running unsupervised and that only makes it unsupervised and that only makes it unsupervised and that only makes it worse because agents want to be helpful.
-
worse because agents want to be helpful. worse because agents want to be helpful. they're going to use all the permissions they're going to use all the permissions they're going to use all the permissions that they have access to in order to get that they have access to in order to get that they have access to in order to get the job done. the job done. the job done. And we can't just solve this with human And we can't just solve this with human And we can't just solve this with human in the loop. We spent decades solving in the loop. We spent decades solving in the loop. We spent decades solving access management for humans. So just access management for humans. So just access management for humans. So just blindly trusting a human who might be a blindly trusting a human who might be a blindly trusting a human who might be a little bit consent fatigued uh or who little bit consent fatigued uh or who little bit consent fatigued uh or who might be tired enough at night, this might be tired enough at night, this might be tired enough at night, this isn't really going to be enough. isn't really going to be enough. isn't really going to be enough. So in order to see where we can So in order to see where we can So in order to see where we can introduce security and access control, introduce security and access control, introduce security and access control, we have to take a look at the agentic we have to take a look at the agentic we have to take a look at the agentic execution path. So we have a user who wants to use an So we have a user who wants to use an LLM to interact with a resource. Now an LLM to interact with a resource. Now an LLM to interact with a resource. Now an agent is a control loop that calls an agent is a control loop that calls an agent is a control loop that calls an LLM and often we have an MCP server in LLM and often we have an MCP server in LLM and often we have an MCP server in between that provides tools that the between that provides tools that the between that provides tools that the agent can call and then it connects agent can call and then it connects agent can call and then it connects directly to the resource. directly to the resource. directly to the resource. Now an MCP client takes the agent's Now an MCP client takes the agent's Now an MCP client takes the agent's proposed tool calls and it dispatches proposed tool calls and it dispatches proposed tool calls and it dispatches them to its MCP server. them to its MCP server. them to its MCP server. And then we have a runtime. Now this is And then we have a runtime. Now this is And then we have a runtime. Now this is a process that runs the agent loop and a process that runs the agent loop and a process that runs the agent loop and executes the calls. And this runtime executes the calls. And this runtime executes the calls. And this runtime might be a CLI like cloud code. It might might be a CLI like cloud code. It might might be a CLI like cloud code. It might be an SDK like AI SDK or provider agent be an SDK like AI SDK or provider agent be an SDK like AI SDK or provider agent SDKs.
-
SDKs. SDKs. Or it might be an app like cursor or Or it might be an app like cursor or Or it might be an app like cursor or codeex. codeex. codeex. So let's follow a prompt through the So let's follow a prompt through the So let's follow a prompt through the execution path. The user submits the execution path. The user submits the execution path. The user submits the prompt to the runtime which calls the prompt to the runtime which calls the prompt to the runtime which calls the model which sends it to the LM. The model which sends it to the LM. The model which sends it to the LM. The model is then going to propose tool model is then going to propose tool model is then going to propose tool calls which are dispatched by the MCP calls which are dispatched by the MCP calls which are dispatched by the MCP client to the MCP server which then client to the MCP server which then client to the MCP server which then executes the tools and it calls the executes the tools and it calls the executes the tools and it calls the resource API. resource API. resource API. The API then responds to the MCP server The API then responds to the MCP server The API then responds to the MCP server and the MCP client delivers the results and the MCP client delivers the results and the MCP client delivers the results to the runtime and then the model will to the runtime and then the model will to the runtime and then the model will be called and loop repeats until the be called and loop repeats until the be called and loop repeats until the model is satisfied and at that point model is satisfied and at that point model is satisfied and at that point it's going to return the final answer to it's going to return the final answer to it's going to return the final answer to the user. the user. the user. So there are a few places in this path So there are a few places in this path So there are a few places in this path where we could implement real access where we could implement real access where we could implement real access control and we can actually do this with control and we can actually do this with control and we can actually do this with open standards and as kind of a spoiler open standards and as kind of a spoiler open standards and as kind of a spoiler it's not just OOTH. So RFC8693 So RFC8693 is token exchange and this is an RFC is token exchange and this is an RFC is token exchange and this is an RFC that extends OOTH 2 and I'm going to that extends OOTH 2 and I'm going to that extends OOTH 2 and I'm going to show you how this spec can be used to show you how this spec can be used to show you how this spec can be used to address agent access.
-
So first I want to recap the problems So first I want to recap the problems that we're actually trying to solve that we're actually trying to solve that we're actually trying to solve here. Right? So if you remember looking here. Right? So if you remember looking here. Right? So if you remember looking at the audit log as it was going by in at the audit log as it was going by in at the audit log as it was going by in the demo, we could see API keys were the demo, we could see API keys were the demo, we could see API keys were being used to call endpoints, but we had being used to call endpoints, but we had being used to call endpoints, but we had no idea who was using the API keys. So, no idea who was using the API keys. So, no idea who was using the API keys. So, we have credentials that are being used we have credentials that are being used we have credentials that are being used that aren't attributed to a user or an that aren't attributed to a user or an that aren't attributed to a user or an agent identity. We have an agent that agent identity. We have an agent that agent identity. We have an agent that has unrestricted access to any and all has unrestricted access to any and all has unrestricted access to any and all permissions that are in an API key. And permissions that are in an API key. And permissions that are in an API key. And finally, we know that we can't just slap finally, we know that we can't just slap finally, we know that we can't just slap human in the loop everywhere because human in the loop everywhere because human in the loop everywhere because humans make mistakes too. And also, many humans make mistakes too. And also, many humans make mistakes too. And also, many agents run autonomously. So, we can address this with an So, we can address this with an authorization server called a security authorization server called a security authorization server called a security token service. token service. token service. Now an authorization server verifies Now an authorization server verifies Now an authorization server verifies identities and issues tokens. So identities and issues tokens. So identities and issues tokens. So identity providers like Google, Octa, identity providers like Google, Octa, identity providers like Google, Octa, Ozero and so on provide authorization Ozero and so on provide authorization Ozero and so on provide authorization servers. And if we want an identity servers. And if we want an identity servers. And if we want an identity chain in our agent execution path, then chain in our agent execution path, then chain in our agent execution path, then we have to be able to log in first. we have to be able to log in first. we have to be able to log in first. So the authorization server is then So the authorization server is then So the authorization server is then going to prompt the user for their going to prompt the user for their going to prompt the user for their consent to delegate access with a subset consent to delegate access with a subset consent to delegate access with a subset of their permissions. And this is the of their permissions. And this is the of their permissions. And this is the first narrowing of access, right? We're first narrowing of access, right? We're first narrowing of access, right? We're only delegating some of the user's total only delegating some of the user's total only delegating some of the user's total permissions to the agent.
-
permissions to the agent. permissions to the agent. The authorization server issues a token The authorization server issues a token The authorization server issues a token that identifies the user and also that identifies the user and also that identifies the user and also contains their level of access. And so contains their level of access. And so contains their level of access. And so right now already we're doing better right now already we're doing better right now already we're doing better than the first demo because we actually than the first demo because we actually than the first demo because we actually know who the user is and what they're know who the user is and what they're know who the user is and what they're allowed to do. allowed to do. allowed to do. So this token identifies the subject on So this token identifies the subject on So this token identifies the subject on whose behalf the agent is going to act. whose behalf the agent is going to act. whose behalf the agent is going to act. In order to support token exchange, we In order to support token exchange, we In order to support token exchange, we need an OOTH client that's capable of need an OOTH client that's capable of need an OOTH client that's capable of executing code. So, this might be a executing code. So, this might be a executing code. So, this might be a gateway between the MCP client and a gateway between the MCP client and a gateway between the MCP client and a thirdparty MCP server. It might be your thirdparty MCP server. It might be your thirdparty MCP server. It might be your own custom agent app or a CLI wrapper own custom agent app or a CLI wrapper own custom agent app or a CLI wrapper around an off-the-shelf coding agent. around an off-the-shelf coding agent. around an off-the-shelf coding agent. And we take the prompt and we take the And we take the prompt and we take the And we take the prompt and we take the subject token and we send these to the subject token and we send these to the subject token and we send these to the OATH client. Now the agent loop runs and OATH client. Now the agent loop runs and OATH client. Now the agent loop runs and the model proposes a tool call the model proposes a tool call the model proposes a tool call and then the runtime is going to and then the runtime is going to and then the runtime is going to authenticate with the security token authenticate with the security token authenticate with the security token service using its ooth app client service using its ooth app client service using its ooth app client credentials or workload identity and it credentials or workload identity and it credentials or workload identity and it also sends the subject token that also sends the subject token that also sends the subject token that contains the user's identity and level contains the user's identity and level contains the user's identity and level of access. It creates token exchange of access. It creates token exchange of access. It creates token exchange request and this request is asking for request and this request is asking for request and this request is asking for permissions to access the MCP server for permissions to access the MCP server for permissions to access the MCP server for that tool call but only that tool call.
-
that tool call but only that tool call. that tool call but only that tool call. So now we have three key pieces of So now we have three key pieces of So now we have three key pieces of information that we're missing from the information that we're missing from the information that we're missing from the API key demo. We know the identity of the agent that's We know the identity of the agent that's requesting access. We know the identity requesting access. We know the identity requesting access. We know the identity of the user on whose behalf is acting. of the user on whose behalf is acting. of the user on whose behalf is acting. And we know the delegating user's level And we know the delegating user's level And we know the delegating user's level of access as well. of access as well. of access as well. So now we need to decide if the So now we need to decide if the So now we need to decide if the requested token should in fact be requested token should in fact be requested token should in fact be granted. And we can do this using granted. And we can do this using granted. And we can do this using governance policy governance policy governance policy which is evaluated against the requested which is evaluated against the requested which is evaluated against the requested access and who's asking for what access and who's asking for what access and who's asking for what resource on whose behalf. Now if the delegation chain and the Now if the delegation chain and the requested access are within policy then requested access are within policy then requested access are within policy then the security token service issues an the security token service issues an the security token service issues an access token for the downstream resource access token for the downstream resource access token for the downstream resource and this token has an audience declaring and this token has an audience declaring and this token has an audience declaring that only this target MCP server is that only this target MCP server is that only this target MCP server is allowed to use it to make requests. It allowed to use it to make requests. It allowed to use it to make requests. It should be short-lived uh often expiring should be short-lived uh often expiring should be short-lived uh often expiring within a few minutes and it's also within a few minutes and it's also within a few minutes and it's also ephemeral meaning it should never be ephemeral meaning it should never be ephemeral meaning it should never be stored. stored. stored. So this token is sent to the MCP client So this token is sent to the MCP client So this token is sent to the MCP client which makes tool call using it as a which makes tool call using it as a which makes tool call using it as a bearer credential.
-
bearer credential. bearer credential. The MCP server validates the token and The MCP server validates the token and The MCP server validates the token and then goes and calls the resource and then goes and calls the resource and then goes and calls the resource and again it never stores the token and it again it never stores the token and it again it never stores the token and it discards it as soon as the call is done. discards it as soon as the call is done. discards it as soon as the call is done. So the result flows back up the loop and So the result flows back up the loop and So the result flows back up the loop and then it repeats until the model returns then it repeats until the model returns then it repeats until the model returns the answer to the user. So if we come back to the demo now we're So if we come back to the demo now we're going to use the same agent only now only now we have token exchange. Okay. So the first thing we have that's Okay. So the first thing we have that's different already is that we have an different already is that we have an different already is that we have an operator sign in right. So we have operator sign in right. So we have operator sign in right. So we have authentication authentication authentication and I'm going to authenticate with and I'm going to authenticate with and I'm going to authenticate with Google as myself. So we have the same tickets. We have the So we have the same tickets. We have the same agent. It's got the same prompt. same agent. It's got the same prompt. same agent. It's got the same prompt. And now we can see what it's going to And now we can see what it's going to And now we can see what it's going to do. Now the first item is probably going do. Now the first item is probably going do. Now the first item is probably going to be exactly the right because this was to be exactly the right because this was to be exactly the right because this was a hardware failure. there's it's going a hardware failure. there's it's going a hardware failure. there's it's going to decide after it reads it that there's to decide after it reads it that there's to decide after it reads it that there's nothing it can do. But as you can see nothing it can do. But as you can see nothing it can do. But as you can see kind of the audit log filling up, we've kind of the audit log filling up, we've kind of the audit log filling up, we've got a lot more information now. Um we got a lot more information now. Um we got a lot more information now. Um we know who the agent is acting on behalf know who the agent is acting on behalf know who the agent is acting on behalf of. We know that the agent is calling a of. We know that the agent is calling a of. We know that the agent is calling a prod infra MCP server and we know that prod infra MCP server and we know that prod infra MCP server and we know that it's going to contact certain downstream it's going to contact certain downstream it's going to contact certain downstream resources. Right? So we have a hardware resources. Right? So we have a hardware resources. Right? So we have a hardware monitor that is the source of this monitor that is the source of this monitor that is the source of this incident and then when it decides that
-
incident and then when it decides that incident and then when it decides that it can't do anything about it, it uses a it can't do anything about it, it uses a it can't do anything about it, it uses a right scope to talk to the pager right scope to talk to the pager right scope to talk to the pager uh resource in order to escalate to the uh resource in order to escalate to the uh resource in order to escalate to the morning team. So for renewing the certificate, right, So for renewing the certificate, right, this is actually a pretty safe action this is actually a pretty safe action this is actually a pretty safe action and it's going to specifically ask for a and it's going to specifically ask for a and it's going to specifically ask for a scope to only renew the certificate, scope to only renew the certificate, scope to only renew the certificate, right? So it's talking to this cloud right? So it's talking to this cloud right? So it's talking to this cloud host where before we had this API key host where before we had this API key host where before we had this API key that could do a ton of different things, that could do a ton of different things, that could do a ton of different things, but this time it is only asking for but this time it is only asking for but this time it is only asking for permissions, being granted permission to permissions, being granted permission to permissions, being granted permission to do this one thing. So now with the billing database right So now with the billing database right like the billing database is broken. It like the billing database is broken. It like the billing database is broken. It uh pretty clearly documents that you are uh pretty clearly documents that you are uh pretty clearly documents that you are supposed to uh drop the database here supposed to uh drop the database here supposed to uh drop the database here but no agent should be able to drop a but no agent should be able to drop a but no agent should be able to drop a database. So what happens is when we database. So what happens is when we database. So what happens is when we make this call it's being eval the make this call it's being eval the make this call it's being eval the policy is evaluating the request against policy is evaluating the request against policy is evaluating the request against all of the permissions that the user has all of the permissions that the user has all of the permissions that the user has and uh it sees that there is actually a and uh it sees that there is actually a and uh it sees that there is actually a restriction in place that prevents restriction in place that prevents restriction in place that prevents agents from doing this and this agents from doing this and this agents from doing this and this credential never even existed. So the credential never even existed. So the credential never even existed. So the policy evaluates before the credential policy evaluates before the credential policy evaluates before the credential is minted, which means you don't have an is minted, which means you don't have an is minted, which means you don't have an overprivileged credential that's just overprivileged credential that's just overprivileged credential that's just floating around then that uh you were floating around then that uh you were floating around then that uh you were supposed to then prevent the entity from supposed to then prevent the entity from supposed to then prevent the entity from receiving. Um it just doesn't exist. So receiving. Um it just doesn't exist. So receiving. Um it just doesn't exist. So there's nothing to leak, there's nothing
-
there's nothing to leak, there's nothing there's nothing to leak, there's nothing to replay, and there's nothing to steal. So this one was the one where it wants So this one was the one where it wants to restart prod. to restart prod. to restart prod. So there are things that agents should So there are things that agents should So there are things that agents should probably be allowed to do and things probably be allowed to do and things probably be allowed to do and things that maybe they shouldn't be allowed to that maybe they shouldn't be allowed to that maybe they shouldn't be allowed to do and then there's some kind of you do and then there's some kind of you do and then there's some kind of you know something in between, right? So know something in between, right? So know something in between, right? So it's going to ask me as the user for my it's going to ask me as the user for my it's going to ask me as the user for my approval as human in the loop. I say approval as human in the loop. I say approval as human in the loop. I say that it can do that but there's another that it can do that but there's another that it can do that but there's another policy here that says that the human policy here that says that the human policy here that says that the human user needs to have a specific role in user needs to have a specific role in user needs to have a specific role in order to be able to do this. And I order to be able to do this. And I order to be able to do this. And I actually do not have that role. So it's actually do not have that role. So it's actually do not have that role. So it's going to prevent me from being able to going to prevent me from being able to going to prevent me from being able to allow the agent to do this. uh even allow the agent to do this. uh even allow the agent to do this. uh even though I approved it. So we can prevent though I approved it. So we can prevent though I approved it. So we can prevent kind of people from just consent fatigue kind of people from just consent fatigue kind of people from just consent fatigue clicking over and over just to get clicking over and over just to get clicking over and over just to get things done. >> And then this is the scaling one, right? >> And then this is the scaling one, right? So this is something that maybe the user So this is something that maybe the user So this is something that maybe the user does have permission to do.
-
does have permission to do. does have permission to do. So if I say approve on this, I do have So if I say approve on this, I do have So if I say approve on this, I do have permission to do this and I was able to permission to do this and I was able to permission to do this and I was able to tell the agent that it is indeed allowed tell the agent that it is indeed allowed tell the agent that it is indeed allowed to and the policy approved it because I to and the policy approved it because I to and the policy approved it because I am allowed to do it also. So the agent access problems that we had So the agent access problems that we had discussed they have solutions now we discussed they have solutions now we discussed they have solutions now we know who the user is and we know who the know who the user is and we know who the know who the user is and we know who the agent is as well. The agent also has agent is as well. The agent also has agent is as well. The agent also has task scoped short-lived ephemeral access task scoped short-lived ephemeral access task scoped short-lived ephemeral access and human in the loop actually has and human in the loop actually has and human in the loop actually has access control that is backed by real access control that is backed by real access control that is backed by real policy. So an exhausted person can't policy. So an exhausted person can't policy. So an exhausted person can't just accept everything that happens. Now another benefit of using open Now another benefit of using open standards like token exchange is the standards like token exchange is the standards like token exchange is the ability to continue to support emerging ability to continue to support emerging ability to continue to support emerging technologies. So this works with technologies. So this works with technologies. So this works with off-the-shelf agents. It also works with off-the-shelf agents. It also works with off-the-shelf agents. It also works with custom agents that you might build custom agents that you might build custom agents that you might build yourself. It works with the CLI. It yourself. It works with the CLI. It yourself. It works with the CLI. It works with thirdparty as well as works with thirdparty as well as works with thirdparty as well as proprietary MCP servers, MCP gateways, proprietary MCP servers, MCP gateways, proprietary MCP servers, MCP gateways, agent to agent, uh any OOTH identity agent to agent, uh any OOTH identity agent to agent, uh any OOTH identity provider. It works with OpenClaw and provider. It works with OpenClaw and provider. It works with OpenClaw and basically anything that might come out basically anything that might come out basically anything that might come out next week.
-
So, it is in fact possible right now to So, it is in fact possible right now to be that responsible parent and to say be that responsible parent and to say be that responsible parent and to say that yes, you do in fact know where your that yes, you do in fact know where your that yes, you do in fact know where your agents are. So my name is Kim Maida. I am the So my name is Kim Maida. I am the founding GTM engineer and head of Devril founding GTM engineer and head of Devril founding GTM engineer and head of Devril at Keycard which is a standardsbased at Keycard which is a standardsbased at Keycard which is a standardsbased platform for uh providing a security platform for uh providing a security platform for uh providing a security token service and policy governance. I'm token service and policy governance. I'm token service and policy governance. I'm going to be at the Keycard booth for going to be at the Keycard booth for going to be at the Keycard booth for kind of the duration of the event but kind of the duration of the event but kind of the duration of the event but we're also running a workshop tomorrow we're also running a workshop tomorrow we're also running a workshop tomorrow on building and securing an MCP server. on building and securing an MCP server. on building and securing an MCP server. You can scan this QR code to connect You can scan this QR code to connect You can scan this QR code to connect with me and I really appreciate your with me and I really appreciate your with me and I really appreciate your time today. So, thank you very much. time today. So, thank you very much. time today. So, thank you very much. [applause] >> We do have time for a couple questions. >> We do have time for a couple questions. >> Uh, so if anyone has some questions for >> Uh, so if anyone has some questions for >> Uh, so if anyone has some questions for Kim, um, we can answer those right now Kim, um, we can answer those right now Kim, um, we can answer those right now and then we got about five five minutes. and then we got about five five minutes. and then we got about five five minutes. So, we can probably do about five short So, we can probably do about five short So, we can probably do about five short questions if anyone has them.
-
Um, so it seems like you put the Um, so it seems like you put the security bar bear barrier security bar bear barrier security bar bear barrier uh at the um between MCP to uh at the um between MCP to uh at the um between MCP to uh resource or is it uh just maybe to uh resource or is it uh just maybe to uh resource or is it uh just maybe to clarify that is it at at both places? clarify that is it at at both places? clarify that is it at at both places? I'm wondering if there was like any I'm wondering if there was like any I'm wondering if there was like any decision there or like what would decision there or like what would decision there or like what would motivate you to choose you know where to motivate you to choose you know where to motivate you to choose you know where to put these sorts of barriers. put these sorts of barriers. put these sorts of barriers. >> Uh yeah so the the authorization server >> Uh yeah so the the authorization server >> Uh yeah so the the authorization server is is is sitting in between let me find the slide sitting in between let me find the slide sitting in between let me find the slide actually. So the runtime authenticates to the So the runtime authenticates to the security token service and identifies security token service and identifies security token service and identifies itself. And this is the point at which itself. And this is the point at which itself. And this is the point at which we have the request that the agent we have the request that the agent we have the request that the agent generated. Um and we also have the the generated. Um and we also have the the generated. Um and we also have the the scopes that it's asking for to get the scopes that it's asking for to get the scopes that it's asking for to get the token to call the next thing in line, token to call the next thing in line, token to call the next thing in line, right? And the next thing in line in right? And the next thing in line in right? And the next thing in line in this particular case is the MCP server.
-
this particular case is the MCP server. this particular case is the MCP server. So the downstream resource is like one So the downstream resource is like one So the downstream resource is like one step farther down. But if you think step farther down. But if you think step farther down. But if you think about like an OOTH token for a user, about like an OOTH token for a user, about like an OOTH token for a user, right? So an OOTH token for a user is right? So an OOTH token for a user is right? So an OOTH token for a user is going to have all of the grants in it going to have all of the grants in it going to have all of the grants in it that the user accepted when it when we that the user accepted when it when we that the user accepted when it when we were presented with like here's what were presented with like here's what were presented with like here's what access you're going to delegate. But an access you're going to delegate. But an access you're going to delegate. But an agent, you don't want an agent to be agent, you don't want an agent to be agent, you don't want an agent to be using any of those grants that it wants using any of those grants that it wants using any of those grants that it wants on every single tool call. So this the on every single tool call. So this the on every single tool call. So this the service sits in between that so that we service sits in between that so that we service sits in between that so that we can say if the things it's requesting can say if the things it's requesting can say if the things it's requesting are kind of beyond what we want to allow are kind of beyond what we want to allow are kind of beyond what we want to allow for the specific tool call then we never for the specific tool call then we never for the specific tool call then we never send that ooth token down right so they send that ooth token down right so they send that ooth token down right so they get that oath token only if they are get that oath token only if they are get that oath token only if they are within requesting something within the within requesting something within the within requesting something within the scope of what what we want scope of what what we want scope of what what we want does that make sense enterprise systems. Uh there may be some enterprise systems. Uh there may be some resistance to actually adopt this new uh resistance to actually adopt this new uh resistance to actually adopt this new uh protocol, new openspec. Have you protocol, new openspec. Have you protocol, new openspec. Have you encountered that? and what are the ways
-
encountered that? and what are the ways encountered that? and what are the ways to uh get past that? to uh get past that? to uh get past that? >> So, it's not actually a new spec, which >> So, it's not actually a new spec, which >> So, it's not actually a new spec, which is, you know, it it's kind of one of is, you know, it it's kind of one of is, you know, it it's kind of one of those things like there was this period those things like there was this period those things like there was this period of time where people were like, oh, you of time where people were like, oh, you of time where people were like, oh, you can just use OOTH for for this. Um, we can just use OOTH for for this. Um, we can just use OOTH for for this. Um, we don't necessarily need a new spec for don't necessarily need a new spec for don't necessarily need a new spec for this. This spec has actually existed for this. This spec has actually existed for this. This spec has actually existed for a little while already. Um, so there's a little while already. Um, so there's a little while already. Um, so there's not kind of that fear of, oh my gosh, not kind of that fear of, oh my gosh, not kind of that fear of, oh my gosh, we're introducing something completely we're introducing something completely we're introducing something completely new. new. new. There are new specs that are coming out There are new specs that are coming out There are new specs that are coming out almost daily right now. Um, but they can almost daily right now. Um, but they can almost daily right now. Um, but they can be combined essentially with this uh be combined essentially with this uh be combined essentially with this uh with token exchange. with token exchange. with token exchange. I think we have time for one more I think we have time for one more I think we have time for one more question. I might be asking a big question. So if I might be asking a big question. So if you tell me to go look at the thing, you tell me to go look at the thing, you tell me to go look at the thing, that's fine. Um, I'm in the situation that's fine. Um, I'm in the situation that's fine. Um, I'm in the situation where we know that we don't have enough where we know that we don't have enough where we know that we don't have enough OOTH scopes defined yet in an MCP server OOTH scopes defined yet in an MCP server OOTH scopes defined yet in an MCP server sub just like what you have. And one of sub just like what you have. And one of sub just like what you have. And one of the reluctant things that we've got is the reluctant things that we've got is the reluctant things that we've got is like how fine grain do we get with the like how fine grain do we get with the like how fine grain do we get with the scope definitions because we know that scope definitions because we know that scope definitions because we know that we're also having to define downstream we're also having to define downstream we're also having to define downstream services that will have a certain number services that will have a certain number services that will have a certain number of these things and someone has to do of these things and someone has to do of these things and someone has to do the authorization check somewhere.
-
the authorization check somewhere. the authorization check somewhere. What's your recommendation for getting What's your recommendation for getting What's your recommendation for getting started with defining your own scopes so started with defining your own scopes so started with defining your own scopes so that you can realistically manage this that you can realistically manage this that you can realistically manage this thing when you haven't defined enough thing when you haven't defined enough thing when you haven't defined enough yet and you're worried about ongoing yet and you're worried about ongoing yet and you're worried about ongoing management over time? management over time? management over time? >> Well, if your resource server already >> Well, if your resource server already >> Well, if your resource server already has like specific scopes, that's going has like specific scopes, that's going has like specific scopes, that's going to be kind of your place to start to be kind of your place to start to be kind of your place to start because the downstream token, the one because the downstream token, the one because the downstream token, the one that was the OS token for the user is that was the OS token for the user is that was the OS token for the user is going to have the scopes for the going to have the scopes for the going to have the scopes for the resource, right? Because it's the user's resource, right? Because it's the user's resource, right? Because it's the user's access to the resource. So those are access to the resource. So those are access to the resource. So those are kind of like the baseline like those are kind of like the baseline like those are kind of like the baseline like those are those are the ones you know that you're those are the ones you know that you're those are the ones you know that you're going to have and then if you want to going to have and then if you want to going to have and then if you want to have scopes additionally for that kind have scopes additionally for that kind have scopes additionally for that kind of govern tool calls if you have like a of govern tool calls if you have like a of govern tool calls if you have like a custom MCP server or something like that custom MCP server or something like that custom MCP server or something like that then you can layer those on top or you then you can layer those on top or you then you can layer those on top or you can just pass them through. can just pass them through. can just pass them through. >> Yeah. Okay. Thanks. >> Yeah. Okay. Thanks. >> Yeah. Okay. Thanks. >> All right. Thank you Kim.
Summary
The main theme is the precarious state of AI agents being given significant responsibility without sufficient oversight, drawing a parallel to the 1980s public service announcement, "It's 10 p.m., do you know where your children are?". The transcript discusses how agents, when granted access to APIs and keys for tasks like incident management, can execute actions without explicit human confirmation. The practical takeaway is the urgent need to implement robust monitoring and control mechanisms for AI agents to prevent unintended or harmful actions, especially when dealing with critical systems.