Security Firewall for Agents — Ryan Dahl, Deno

How's it going? How's it going? >> Um, my name is Ryan. Um, I'm going to I >> Um, my name is Ryan. Um, I'm going to I >> Um, my name is Ryan. Um, I'm going to I I'm I'm the CEO at Dino and uh yeah, you I'm I'm the CEO at Dino and uh yeah, you I'm I'm the CEO at Dino and uh yeah, you been developing software for for quite a been developing software for for quite a been developing software for for quite a while at this point. You might know one while at this point. You might know one while at this point. You might know one of my projects, Node.js. of my projects, Node.js. of my projects, Node.js. Um, I want to talk about um a service Um, I want to talk about um a service Um, I want to talk about um a service that we're running at Dino called Dino that we're running at Dino called Dino that we're running at Dino called Dino Deploy. This is a system for hosting Deploy. This is a system for hosting Deploy. This is a system for hosting websites and it has incidences. It's it websites and it has incidences. It's it websites and it has incidences. It's it it has downtime occasionally and uh it has downtime occasionally and uh it has downtime occasionally and uh we've got a pager duty that fires. I'm we've got a pager duty that fires. I'm we've got a pager duty that fires. I'm sure you're all very familiar with the sure you're all very familiar with the sure you're all very familiar with the very scary alarm sound that wakes you up very scary alarm sound that wakes you up very scary alarm sound that wakes you up in the middle of the night. Um, and in the middle of the night. Um, and in the middle of the night. Um, and recently we've been playing around with recently we've been playing around with recently we've been playing around with using agents to automatically service using agents to automatically service using agents to automatically service these incidences. Um, in particular, these incidences. Um, in particular, these incidences. Um, in particular, OpenClaw, but other other agents as OpenClaw, but other other agents as OpenClaw, but other other agents as well. Um, and we've found a pattern that well. Um, and we've found a pattern that well. Um, and we've found a pattern that is working pretty well for us that I is working pretty well for us that I is working pretty well for us that I want to share with you. Um, we actually want to share with you. Um, we actually want to share with you. Um, we actually give OpenClaw access to all sorts of give OpenClaw access to all sorts of give OpenClaw access to all sorts of systems. Postgress, Kubernetes, systems. Postgress, Kubernetes, systems. Postgress, Kubernetes, Clickhouse, AWS, GitHub, Slack, uh all Clickhouse, AWS, GitHub, Slack, uh all Clickhouse, AWS, GitHub, Slack, uh all all sorts of things. And we we do all sorts of things. And we we do all sorts of things. And we we do actually give them uh rewrite access to actually give them uh rewrite access to actually give them uh rewrite access to these systems.

these systems. these systems. This is very powerful uh because the This is very powerful uh because the This is very powerful uh because the agents can actually get all of the agents can actually get all of the agents can actually get all of the context. They can see traces in context. They can see traces in context. They can see traces in ClickHouse. They can look in the ClickHouse. They can look in the ClickHouse. They can look in the production Postgress database at what production Postgress database at what production Postgress database at what the user what you what projects a user the user what you what projects a user the user what you what projects a user owns. They they can look through Slack owns. They they can look through Slack owns. They they can look through Slack for uh communications uh GitHub logs for uh communications uh GitHub logs for uh communications uh GitHub logs etc. Um etc. Um etc. Um this actually works quite well. Uh the this actually works quite well. Uh the this actually works quite well. Uh the the agents are actually able to solve the agents are actually able to solve the agents are actually able to solve quite a lot of incidences where we quite a lot of incidences where we quite a lot of incidences where we previously would have a human s in the previously would have a human s in the previously would have a human s in the loop. loop. loop. But it is very dangerous of course But it is very dangerous of course But it is very dangerous of course because these agents could do nefarious because these agents could do nefarious because these agents could do nefarious things. They could start a psql things. They could start a psql things. They could start a psql subprocess and issue a delete users subprocess and issue a delete users subprocess and issue a delete users table. Um they could call cubecuddle table. Um they could call cubecuddle table. Um they could call cubecuddle delete namespace prod. Um you know they delete namespace prod. Um you know they delete namespace prod. Um you know they they could decide somehow that solving they could decide somehow that solving they could decide somehow that solving the incident it means you know removing the incident it means you know removing the incident it means you know removing all of the users. Uh and of course we all of the users. Uh and of course we all of the users. Uh and of course we don't want that.

We use Opus and Opus is remarkably well We use Opus and Opus is remarkably well aligned. You can really not you you can aligned. You can really not you you can aligned. You can really not you you can try very hard to to get it to delete the try very hard to to get it to delete the try very hard to to get it to delete the user's table and it will refuse over and user's table and it will refuse over and user's table and it will refuse over and over again. But this is not sufficient, over again. But this is not sufficient, over again. But this is not sufficient, right? Security can't just be wishful right? Security can't just be wishful right? Security can't just be wishful thinking that Opus will always obey your thinking that Opus will always obey your thinking that Opus will always obey your your wishes. Um these S sur agents that we have are these S sur agents that we have are connected to the support system and thus connected to the support system and thus connected to the support system and thus can be prompt injected from the outside can be prompt injected from the outside can be prompt injected from the outside and that means that they can be and that means that they can be and that means that they can be manipulated somehow. like who knows who manipulated somehow. like who knows who manipulated somehow. like who knows who knows what sort of uh string of knows what sort of uh string of knows what sort of uh string of characters could send opus into some uh characters could send opus into some uh characters could send opus into some uh bad state that allows it to think that bad state that allows it to think that bad state that allows it to think that it's taking the right action by doing it's taking the right action by doing it's taking the right action by doing something very undesirable. something very undesirable. something very undesirable. So you know we take the stance that the So you know we take the stance that the So you know we take the stance that the sec the agents themselves have to be sec the agents themselves have to be sec the agents themselves have to be untrusted software. You can't rely on untrusted software. You can't rely on untrusted software. You can't rely on the agent itself to guard what it's the agent itself to guard what it's the agent itself to guard what it's doing. You can't put the guard inside doing. You can't put the guard inside doing. You can't put the guard inside the agent.

the agent. the agent. We run agents and I assume many of you We run agents and I assume many of you We run agents and I assume many of you do the same on standalone VMs. So we're do the same on standalone VMs. So we're do the same on standalone VMs. So we're not very concerned about agents touching not very concerned about agents touching not very concerned about agents touching files on the file system. You know, files on the file system. You know, files on the file system. You know, they're they're they're uh they're they're they're they're uh they're they're they're they're uh they're properly isolated at the system level. properly isolated at the system level. properly isolated at the system level. But But But [clears throat] so you know effectively [clears throat] so you know effectively [clears throat] so you know effectively every nefarious action that that an every nefarious action that that an every nefarious action that that an agent could take every good action that agent could take every good action that agent could take every good action that it takes comes in the form of some it takes comes in the form of some it takes comes in the form of some network communication network communication network communication some some bites over the wire and how some some bites over the wire and how some some bites over the wire and how these bytes are formed can happen in these bytes are formed can happen in these bytes are formed can happen in various ways. You can of course call various ways. You can of course call various ways. You can of course call through MCP but also subprocesses and if through MCP but also subprocesses and if through MCP but also subprocesses and if you think of Postgress for example this you think of Postgress for example this you think of Postgress for example this is a nonHTTP protocol that uh open clock is a nonHTTP protocol that uh open clock is a nonHTTP protocol that uh open clock can just spawn as as a subprocess and can just spawn as as a subprocess and can just spawn as as a subprocess and and start connecting to to services. Um and start connecting to to services. Um and start connecting to to services. Um so we take the stance that we really so we take the stance that we really so we take the stance that we really want to understand what the bites are want to understand what the bites are want to understand what the bites are coming out of that agent in great coming out of that agent in great coming out of that agent in great detail.

detail. detail. >> [clears throat] >> [clears throat] >> [clears throat] >> This can get very tricky in real world >> This can get very tricky in real world >> This can get very tricky in real world systems. So for example, we have a systems. So for example, we have a systems. So for example, we have a production Postgress database in AWS um production Postgress database in AWS um production Postgress database in AWS um that is inside a VPC that we can only that is inside a VPC that we can only that is inside a VPC that we can only reach really through uh an EKS endpoint. reach really through uh an EKS endpoint. reach really through uh an EKS endpoint. And what we'd really like to do is And what we'd really like to do is And what we'd really like to do is ensure that our agent, which we want to ensure that our agent, which we want to ensure that our agent, which we want to give access to everything essentially, give access to everything essentially, give access to everything essentially, can't somehow tunnel through this EKS can't somehow tunnel through this EKS can't somehow tunnel through this EKS server, spawn psql, and drop the users server, spawn psql, and drop the users server, spawn psql, and drop the users table, right? We're we're we're table, right? We're we're we're table, right? We're we're we're concerned about pretty crazy situations concerned about pretty crazy situations concerned about pretty crazy situations like this that get very complicated. And like this that get very complicated. And like this that get very complicated. And I think many of you work in companies I think many of you work in companies I think many of you work in companies where you have real world systems where where you have real world systems where where you have real world systems where things are very complic. So yeah, just to just to highlight this, So yeah, just to just to highlight this, this is an outbound path the agents host this is an outbound path the agents host this is an outbound path the agents host can't reach on a protocol that isn't can't reach on a protocol that isn't can't reach on a protocol that isn't HTTP that's gated by a rule that HTTP that's gated by a rule that HTTP that's gated by a rule that understands SQL.

understands SQL. understands SQL. These are what human S surres would do. These are what human S surres would do. These are what human S surres would do. And how can we, you know, empower these And how can we, you know, empower these And how can we, you know, empower these these agents to to have kind of the the these agents to to have kind of the the these agents to to have kind of the the same access that that a human might. same access that that a human might. same access that that a human might. Um, so you might ask, you might say, Um, so you might ask, you might say, Um, so you might ask, you might say, well, you know, there's ACL's, there's well, you know, there's ACL's, there's well, you know, there's ACL's, there's permissions, you can issue readonly uh permissions, you can issue readonly uh permissions, you can issue readonly uh Postgress credentials. Um, and yeah, Postgress credentials. Um, and yeah, Postgress credentials. Um, and yeah, that's true up to a point. Um, you can that's true up to a point. Um, you can that's true up to a point. Um, you can do careful credential provisioning and do careful credential provisioning and do careful credential provisioning and you should. Um, but this this really you should. Um, but this this really you should. Um, but this this really requires uh working across many requires uh working across many requires uh working across many different systems, provisioning different systems, provisioning different systems, provisioning credentials in in incredibly careful credentials in in incredibly careful credentials in in incredibly careful ways. And as I just demonstrated, the ways. And as I just demonstrated, the ways. And as I just demonstrated, the composition of access can lead to holes composition of access can lead to holes composition of access can lead to holes when you can access one system and then when you can access one system and then when you can access one system and then another system. um another system. um another system. um MCP, you know, you can you can uh MCP, you know, you can you can uh MCP, you know, you can you can uh structure all of this as uh very careful structure all of this as uh very careful structure all of this as uh very careful MCP uh tools that uh have the proper MCP uh tools that uh have the proper MCP uh tools that uh have the proper permissions. But, you know, then then permissions. But, you know, then then permissions. But, you know, then then you can't spawn subprocesses, right? You you can't spawn subprocesses, right? You you can't spawn subprocesses, right? You you you can't you know, as soon as as you you can't you know, as soon as as you you can't you know, as soon as as soon as the open clause spawns the the soon as the open clause spawns the the soon as the open clause spawns the the PSQL, uh you're you're kind of out PSQL, uh you're you're kind of out PSQL, uh you're you're kind of out broken through the the security broken through the the security broken through the the security boundary.

There are quite a few projects in this There are quite a few projects in this space um namely projects that kind of space um namely projects that kind of space um namely projects that kind of sit in front of an agent and under look sit in front of an agent and under look sit in front of an agent and under look at what it's sending and try to control at what it's sending and try to control at what it's sending and try to control based on on uh the the bytes that are based on on uh the the bytes that are based on on uh the the bytes that are flowing through this. Um, LLM gateways. flowing through this. Um, LLM gateways. flowing through this. Um, LLM gateways. I think we're all familiar with Open I think we're all familiar with Open I think we're all familiar with Open Router, Light LLM, for example. These Router, Light LLM, for example. These Router, Light LLM, for example. These often have a guard rails feature that often have a guard rails feature that often have a guard rails feature that can [clears throat] can [clears throat] can [clears throat] uh uh uh guard against prompt injection uh, you guard against prompt injection uh, you guard against prompt injection uh, you know, scan for for various uh, know, scan for for various uh, know, scan for for various uh, expressions, etc. that that are going expressions, etc. that that are going expressions, etc. that that are going back and forth between the the LLM uh, back and forth between the the LLM uh, back and forth between the the LLM uh, provider. provider. provider. But, of course, that's just the LLM. uh But, of course, that's just the LLM. uh But, of course, that's just the LLM. uh you know we're we're talking to you know we're we're talking to you know we're we're talking to databases and stuff. Um you have systems databases and stuff. Um you have systems databases and stuff. Um you have systems like HTTP jail and Crabtrap that are like HTTP jail and Crabtrap that are like HTTP jail and Crabtrap that are HTTP proxies that really sit at at the HTTP proxies that really sit at at the HTTP proxies that really sit at at the HTTP layer and you uh HTTP jail for HTTP layer and you uh HTTP jail for HTTP layer and you uh HTTP jail for example can will allow you to write example can will allow you to write example can will allow you to write rules that say well you can make get rules that say well you can make get rules that say well you can make get requests but not post requests or you requests but not post requests or you requests but not post requests or you can access this HTTP subpath. Um, Crabt can access this HTTP subpath. Um, Crabt can access this HTTP subpath. Um, Crabt Trap is a project from Brex that has a Trap is a project from Brex that has a Trap is a project from Brex that has a LLM as judge that operates on the HTTP LLM as judge that operates on the HTTP LLM as judge that operates on the HTTP requests flowing back and forth. you requests flowing back and forth. you requests flowing back and forth. you have uh proxies that inject credentials have uh proxies that inject credentials have uh proxies that inject credentials into uh as they're passing out of the into uh as they're passing out of the into uh as they're passing out of the agent. Uh agent vault uh being a popular

agent. Uh agent vault uh being a popular agent. Uh agent vault uh being a popular one where the the agent itself never one where the the agent itself never one where the the agent itself never actually sees the credentials of the actually sees the credentials of the actually sees the credentials of the system that it's talking to but passes system that it's talking to but passes system that it's talking to but passes some placeholder out and the proxy some placeholder out and the proxy some placeholder out and the proxy itself injects those credentials. This itself injects those credentials. This itself injects those credentials. This is an important part of the problem but is an important part of the problem but is an important part of the problem but not a complete solution. And you have not a complete solution. And you have not a complete solution. And you have things like process sandboxes like things like process sandboxes like things like process sandboxes like Nvidia's OpenShell that you know really Nvidia's OpenShell that you know really Nvidia's OpenShell that you know really are kind of OS system level uh uh guards are kind of OS system level uh uh guards are kind of OS system level uh uh guards against say accessing different file against say accessing different file against say accessing different file system paths um accessing different SIS system paths um accessing different SIS system paths um accessing different SIS calls that sort of thing but as I said calls that sort of thing but as I said calls that sort of thing but as I said before we're we're not really concerned before we're we're not really concerned before we're we're not really concerned about that because we provision uh about that because we provision uh about that because we provision uh standalone VM for for our agents. standalone VM for for our agents. standalone VM for for our agents. So the software that uh we've written to So the software that uh we've written to So the software that uh we've written to address this problem is called claw address this problem is called claw address this problem is called claw patrol. Uh it's an open- source MIT patrol. Uh it's an open- source MIT patrol. Uh it's an open- source MIT license project and this is a proxy that license project and this is a proxy that license project and this is a proxy that sits in front of your agents.

sits in front of your agents. sits in front of your agents. Um it operates not at the HTP level but Um it operates not at the HTP level but Um it operates not at the HTP level but at a lower level. It understands each at a lower level. It understands each at a lower level. It understands each and every bite flowing through flowing and every bite flowing through flowing and every bite flowing through flowing out of your agent. It holds credentials out of your agent. It holds credentials out of your agent. It holds credentials like agent vault and can inject those like agent vault and can inject those like agent vault and can inject those credentials so that your credentials so that your credentials so that your uh whatever agent software you're using uh whatever agent software you're using uh whatever agent software you're using uh doesn't actually doesn't ever uh doesn't actually doesn't ever uh doesn't actually doesn't ever actually see secret values. actually see secret values. actually see secret values. And um in particular, it has a very And um in particular, it has a very And um in particular, it has a very advanced rule system that allows you to advanced rule system that allows you to advanced rule system that allows you to say in in precise details how how and say in in precise details how how and say in in precise details how how and and what requests get uh transferred out and what requests get uh transferred out and what requests get uh transferred out out of the agent and talk to the outside out of the agent and talk to the outside out of the agent and talk to the outside world. world. world. These rules are are kind of the the key These rules are are kind of the the key These rules are are kind of the the key piece of the system and we write them in piece of the system and we write them in piece of the system and we write them in a configuration file using a language a configuration file using a language a configuration file using a language called HCL. Uh who anybody familiar with called HCL. Uh who anybody familiar with called HCL. Uh who anybody familiar with HCL? This is like the Terraform the HCL? This is like the Terraform the HCL? This is like the Terraform the Terraform configuration language. Uh it Terraform configuration language. Uh it Terraform configuration language. Uh it actually works really well here. So we actually works really well here. So we actually works really well here. So we have a file that we check into git and have a file that we check into git and have a file that we check into git and we manage very carefully that we manage very carefully that we manage very carefully that essentially defines the permissions for essentially defines the permissions for essentially defines the permissions for all of our services at Dino and these all of our services at Dino and these all of our services at Dino and these yeah it's it's a big long file. It's yeah it's it's a big long file. It's yeah it's it's a big long file. It's like a thousand lines and you know we we like a thousand lines and you know we we like a thousand lines and you know we we manage each and every change to that in manage each and every change to that in manage each and every change to that in in kind of precise detail. This is an in kind of precise detail. This is an in kind of precise detail. This is an example of a rule in our configuration example of a rule in our configuration example of a rule in our configuration file that blocks certain Postgress

file that blocks certain Postgress file that blocks certain Postgress functions from being uh being called. functions from being uh being called. functions from being uh being called. And so yeah, again Postgress being a And so yeah, again Postgress being a And so yeah, again Postgress being a nonHTP protocol and these rules can be nonHTP protocol and these rules can be nonHTP protocol and these rules can be applied even when tunneling through applied even when tunneling through applied even when tunneling through other systems. Um, it supports uh a other systems. Um, it supports uh a other systems. Um, it supports uh a number of different protocols and has a number of different protocols and has a number of different protocols and has a plug-in system to extend it when you run plug-in system to extend it when you run plug-in system to extend it when you run into a protocol that it is not yet into a protocol that it is not yet into a protocol that it is not yet familiar with. familiar with. familiar with. So, uh, here's here's a little demo. So, uh, here's here's a little demo. So, uh, here's here's a little demo. Unfortunately, not live. Um, but we call Unfortunately, not live. Um, but we call Unfortunately, not live. Um, but we call claw patrol run codeex in yellow mode so claw patrol run codeex in yellow mode so claw patrol run codeex in yellow mode so that it just does what you say it should that it just does what you say it should that it just does what you say it should do. And you tell you tell Codex, hey, do. And you tell you tell Codex, hey, do. And you tell you tell Codex, hey, delete the users table from from delete the users table from from delete the users table from from Postgress and Codeex um properly uh Postgress and Codeex um properly uh Postgress and Codeex um properly uh obeys and starts a Psql subprocess where obeys and starts a Psql subprocess where obeys and starts a Psql subprocess where it deletes the the the users table. That it deletes the the the users table. That it deletes the the the users table. That uh Psql subprocess opens a network uh Psql subprocess opens a network uh Psql subprocess opens a network connection to to our to the the connection to to our to the the connection to to our to the the Postgress server that goes through claw Postgress server that goes through claw Postgress server that goes through claw patrol where we pars each and every patrol where we pars each and every patrol where we pars each and every bite. We understand the Postgress bite. We understand the Postgress bite. We understand the Postgress protocol. We apply our rules and protocol. We apply our rules and protocol. We apply our rules and ultimately reject that what we call an ultimately reject that what we call an ultimately reject that what we call an action from uh from doing something action from uh from doing something action from uh from doing something destructive.

Claw Patrol has a dashboard that lets Claw Patrol has a dashboard that lets you see what your agents are doing. So you see what your agents are doing. So you see what your agents are doing. So at the top you can see a couple of at the top you can see a couple of at the top you can see a couple of different uh devices or agents and the different uh devices or agents and the different uh devices or agents and the the various requests that are flowing the various requests that are flowing the various requests that are flowing through. Some of them being denied, some through. Some of them being denied, some through. Some of them being denied, some of them need approval which I'll talk of them need approval which I'll talk of them need approval which I'll talk about in a second. And you can click about in a second. And you can click about in a second. And you can click into to each request or uh action as we into to each request or uh action as we into to each request or uh action as we call it because it's more general than call it because it's more general than call it because it's more general than HTTP requests and see the details of of HTTP requests and see the details of of HTTP requests and see the details of of what's going on. there's there's what's going on. there's there's what's going on. there's there's analytics and yeah it's it's very uh analytics and yeah it's it's very uh analytics and yeah it's it's very uh utilitarian driven. It's like what what utilitarian driven. It's like what what utilitarian driven. It's like what what we need to understand our own agents. we need to understand our own agents. we need to understand our own agents. Um there as I said there's there's an Um there as I said there's there's an Um there as I said there's there's an approval system in this. So you can approval system in this. So you can approval system in this. So you can route, you can define rules that don't route, you can define rules that don't route, you can define rules that don't just reject requests or actions but uh just reject requests or actions but uh just reject requests or actions but uh ask a human for example in a slack ask a human for example in a slack ask a human for example in a slack channel or run an LLM judge over over channel or run an LLM judge over over channel or run an LLM judge over over this um or any combination thereof, this um or any combination thereof, this um or any combination thereof, right? Maybe first first get an LLM right? Maybe first first get an LLM right? Maybe first first get an LLM judge and then get approval in Slack. uh judge and then get approval in Slack. uh judge and then get approval in Slack. uh so that you can have again very precise so that you can have again very precise so that you can have again very precise control over what your agents are doing control over what your agents are doing control over what your agents are doing outside of the agent software itself outside of the agent software itself outside of the agent software itself right we we treat the agent software as right we we treat the agent software as right we we treat the agent software as a black box right we we don't require a black box right we we don't require a black box right we we don't require any changes to to that software any changes to to that software any changes to to that software um I mentioned credential injection um I mentioned credential injection um I mentioned credential injection before uh claw patrol has very detailed

before uh claw patrol has very detailed before uh claw patrol has very detailed support for all sorts of systems support for all sorts of systems support for all sorts of systems credentials come in many different forms credentials come in many different forms credentials come in many different forms they're not just uh bearer uh headers uh they're not just uh bearer uh headers uh they're not just uh bearer uh headers uh it handles cookies it handles Postgress it handles cookies it handles Postgress it handles cookies it handles Postgress as I mentioned uh click house supports as I mentioned uh click house supports as I mentioned uh click house supports all sorts of uh ooth protocols supports all sorts of uh ooth protocols supports all sorts of uh ooth protocols supports very complex things like AWS SIG v4 um very complex things like AWS SIG v4 um very complex things like AWS SIG v4 um so yeah this I guess what I'm trying to so yeah this I guess what I'm trying to so yeah this I guess what I'm trying to uh say is that this is this is really uh say is that this is this is really uh say is that this is this is really born out of utility here and meant for born out of utility here and meant for born out of utility here and meant for real world systems This is not just you real world systems This is not just you real world systems This is not just you know kind of an imaginary scenario. know kind of an imaginary scenario. know kind of an imaginary scenario. [clears throat] Um this system works [clears throat] Um this system works [clears throat] Um this system works over tail scale or wire guard. Um we over tail scale or wire guard. Um we over tail scale or wire guard. Um we ourselves run claw patrol run our agents ourselves run claw patrol run our agents ourselves run claw patrol run our agents inside of tail scale inside of a tail inside of tail scale inside of a tail inside of tail scale inside of a tail net and claw patrol acts as a tail scale net and claw patrol acts as a tail scale net and claw patrol acts as a tail scale exit node. Um we also lean on tail scale exit node. Um we also lean on tail scale exit node. Um we also lean on tail scale for authentication to the dashboard. So for authentication to the dashboard. So for authentication to the dashboard. So your your tail your tail scale identity your your tail your tail scale identity your your tail your tail scale identity actually allows you access to to the actually allows you access to to the actually allows you access to to the dashboard so that we don't have to layer dashboard so that we don't have to layer dashboard so that we don't have to layer on another authentication mechanism. But on another authentication mechanism. But on another authentication mechanism. But we also have this wire guard for people we also have this wire guard for people we also have this wire guard for people who have not bought into the wonderful who have not bought into the wonderful who have not bought into the wonderful tail scale ecosystem. But this works tail scale ecosystem. But this works tail scale ecosystem. But this works very well for us because we know that very well for us because we know that very well for us because we know that all of our stuff is is off the internet all of our stuff is is off the internet all of our stuff is is off the internet and all of these very security sensitive

and all of these very security sensitive and all of these very security sensitive things are are uh tightly controlled. things are are uh tightly controlled. things are are uh tightly controlled. Claw Patrol itself is holding all of Claw Patrol itself is holding all of Claw Patrol itself is holding all of these credentials to production systems. these credentials to production systems. these credentials to production systems. So you have to be very careful with it. So you have to be very careful with it. So you have to be very careful with it. Um so yeah, this the thesis here is is Um so yeah, this the thesis here is is Um so yeah, this the thesis here is is basically that agents can't be trusted basically that agents can't be trusted basically that agents can't be trusted to police themselves. that includes to police themselves. that includes to police themselves. that includes security plugins or or modifications to security plugins or or modifications to security plugins or or modifications to the to the agent software itself. The the to the agent software itself. The the to the agent software itself. The the security boundary has to be the security boundary has to be the security boundary has to be elsewhere. And that's not to say that elsewhere. And that's not to say that elsewhere. And that's not to say that alignment is not a good thing, but uh alignment is not a good thing, but uh alignment is not a good thing, but uh you know for for real world security you know for for real world security you know for for real world security systems, we we really do need to control systems, we we really do need to control systems, we we really do need to control this at at a higher level. And uh claw this at at a higher level. And uh claw this at at a higher level. And uh claw patrol is our attempt to uh make this patrol is our attempt to uh make this patrol is our attempt to uh make this work for ourselves. Um and yeah, you can work for ourselves. Um and yeah, you can work for ourselves. Um and yeah, you can you can check it out here. >> I might have time for one question or >> I might have time for one question or so.

so. so. >> Yes sir. >> Yes sir. >> Yes sir. >> What kind of email testing do you do on >> What kind of email testing do you do on >> What kind of email testing do you do on this to make sure it's working? this to make sure it's working? this to make sure it's working? >> Uh yeah. So the question is what what >> Uh yeah. So the question is what what >> Uh yeah. So the question is what what sort of testing do we do we do to make sort of testing do we do we do to make sort of testing do we do we do to make make sure it works properly? Um I I make sure it works properly? Um I I make sure it works properly? Um I I didn't mention but but there this rule didn't mention but but there this rule didn't mention but but there this rule file actually has a test system along file actually has a test system along file actually has a test system along with it where you can provide uh with it where you can provide uh with it where you can provide uh fixtures action like fixture requests fixtures action like fixture requests fixtures action like fixture requests that can flow through the rules and then that can flow through the rules and then that can flow through the rules and then you can uh essentially create unit tests you can uh essentially create unit tests you can uh essentially create unit tests to make sure that that fixture is always to make sure that that fixture is always to make sure that that fixture is always you know that request it will always be you know that request it will always be you know that request it will always be blocked by by your set of rules. And blocked by by your set of rules. And blocked by by your set of rules. And then of course for the claw patrol then of course for the claw patrol then of course for the claw patrol software itself we have a a large suite software itself we have a a large suite software itself we have a a large suite of of testing. of of testing. of of testing. Yes sir. >> So the question is as as agents get >> So the question is as as agents get smarter does this problem get bigger or smarter does this problem get bigger or smarter does this problem get bigger or smaller? I think I think we can we will smaller? I think I think we can we will smaller? I think I think we can we will never be able to fully trust uh AIs. I never be able to fully trust uh AIs. I never be able to fully trust uh AIs. I think it becomes less and less of a think it becomes less and less of a think it becomes less and less of a problem as they are smarter, have better problem as they are smarter, have better problem as they are smarter, have better context, know that they're working with context, know that they're working with context, know that they're working with a company, know that that they shouldn't a company, know that that they shouldn't a company, know that that they shouldn't be doing bad things. Opus is more be doing bad things. Opus is more be doing bad things. Opus is more aligned than previous models, but I aligned than previous models, but I aligned than previous models, but I think we're always going to have to have think we're always going to have to have think we're always going to have to have uh backs stop security mechanisms.

uh backs stop security mechanisms. uh backs stop security mechanisms. Um, cool. Well, I I'll be around for Um, cool. Well, I I'll be around for Um, cool. Well, I I'll be around for other questions, but thank you very other questions, but thank you very other questions, but thank you very much. [applause]