We let an AI agent execute Bash and lived to talk about it — Sarah Sanders, PostHog

Hi everyone. How are we feeling? Uh Hi everyone. How are we feeling? Uh we're in the home stretch. Uh, my name we're in the home stretch. Uh, my name we're in the home stretch. Uh, my name is Sarah and I am a context engineer at is Sarah and I am a context engineer at is Sarah and I am a context engineer at Post Hog and I get the delight of Post Hog and I get the delight of Post Hog and I get the delight of working on our beloved wizard every working on our beloved wizard every working on our beloved wizard every single day. So, what's the wizard? Um, single day. So, what's the wizard? Um, single day. So, what's the wizard? Um, the wizard sets up Post Hog for you. the wizard sets up Post Hog for you. the wizard sets up Post Hog for you. It's an agentic CLI tool that reads your It's an agentic CLI tool that reads your It's an agentic CLI tool that reads your codebase. It installs the right SDK for codebase. It installs the right SDK for codebase. It installs the right SDK for your project. It instruments your events your project. It instruments your events your project. It instruments your events and it sets up dashboards for you. It and it sets up dashboards for you. It and it sets up dashboards for you. It takes what used to it takes what used to takes what used to it takes what used to takes what used to it takes what used to take about an hour or two of setup and take about an hour or two of setup and take about an hour or two of setup and it runs that in about five to six it runs that in about five to six it runs that in about five to six minutes and it's free inference on us so minutes and it's free inference on us so minutes and it's free inference on us so that you have a great time onboarding to that you have a great time onboarding to that you have a great time onboarding to Post Hog. Sounds kind of sick. Uh, Post Hog. Sounds kind of sick. Uh, Post Hog. Sounds kind of sick. Uh, people love it. But a few months ago, we people love it. But a few months ago, we people love it. But a few months ago, we dared to dream, what if this became the dared to dream, what if this became the dared to dream, what if this became the recommended or default way to install recommended or default way to install recommended or default way to install Post Hog on your project? And Post Hog on your project? And Post Hog on your project? And my security alarm bell started going my security alarm bell started going my security alarm bell started going off. Uh, I started questioning how off. Uh, I started questioning how off. Uh, I started questioning how secure is this thing because it sounds secure is this thing because it sounds secure is this thing because it sounds kind of malware shaped. Um, and in that kind of malware shaped. Um, and in that kind of malware shaped. Um, and in that questioning, I learned a lot. So today questioning, I learned a lot. So today questioning, I learned a lot. So today is all about the lessons I learned, the is all about the lessons I learned, the is all about the lessons I learned, the stuff that kept me up at night while I stuff that kept me up at night while I stuff that kept me up at night while I was building this thing, and the thing was building this thing, and the thing was building this thing, and the thing that I ended up building because of it.

that I ended up building because of it. that I ended up building because of it. So before I dive into all of the boring So before I dive into all of the boring So before I dive into all of the boring security stuff, aka your 2pm catnap, I security stuff, aka your 2pm catnap, I security stuff, aka your 2pm catnap, I want to show you the wizard actually want to show you the wizard actually want to show you the wizard actually running. If you look up on the screen, running. If you look up on the screen, running. If you look up on the screen, it is running for you on a loop. This is it is running for you on a loop. This is it is running for you on a loop. This is the same exact experience that anyone the same exact experience that anyone the same exact experience that anyone who runs npx at post hog wizard gets uh who runs npx at post hog wizard gets uh who runs npx at post hog wizard gets uh on their terminal. on their terminal. on their terminal. Like I said, it's an agent. It figures Like I said, it's an agent. It figures Like I said, it's an agent. It figures out what SDK is right for your project. out what SDK is right for your project. out what SDK is right for your project. It installs it for you, instruments your It installs it for you, instruments your It installs it for you, instruments your events, builds dashboards. I like to events, builds dashboards. I like to events, builds dashboards. I like to call it a little mini implementation call it a little mini implementation call it a little mini implementation engineer in your terminal. engineer in your terminal. engineer in your terminal. And sometimes I show people this and And sometimes I show people this and And sometimes I show people this and they ask me, why an agent? Why don't you they ask me, why an agent? Why don't you they ask me, why an agent? Why don't you give users a good prompt? Why don't you give users a good prompt? Why don't you give users a good prompt? Why don't you give them a skill that they can invoke give them a skill that they can invoke give them a skill that they can invoke in their own tool? And while we do in their own tool? And while we do in their own tool? And while we do provide those things, the answer is provide those things, the answer is provide those things, the answer is because this developer experience and because this developer experience and because this developer experience and the capability of the wizard is the the capability of the wizard is the the capability of the wizard is the whole point. It's the whole product whole point. It's the whole product whole point. It's the whole product because we built a CLI tool that can because we built a CLI tool that can because we built a CLI tool that can fully take part in an agent loop and fully take part in an agent loop and fully take part in an agent loop and experiencing that for the first time is experiencing that for the first time is experiencing that for the first time is really powerful.

really powerful. really powerful. But you can't ship something like the But you can't ship something like the But you can't ship something like the wizard without shipping the stuff that wizard without shipping the stuff that wizard without shipping the stuff that makes the wizard kind of suspect. So let's take it apart. Uh let's look at So let's take it apart. Uh let's look at the anatomy of the wizard because the anatomy of the wizard because the anatomy of the wizard because usually threat models fall right out of usually threat models fall right out of usually threat models fall right out of the anatomy of the agent. So the wizard the anatomy of the agent. So the wizard the anatomy of the agent. So the wizard is a similar shape to what I'm sure a is a similar shape to what I'm sure a is a similar shape to what I'm sure a lot of you are building if you're lot of you are building if you're lot of you are building if you're building agents. It's got models that building agents. It's got models that building agents. It's got models that we've picked for specific tasks. It's we've picked for specific tasks. It's we've picked for specific tasks. It's got prompts that steer it and it's got a got prompts that steer it and it's got a got prompts that steer it and it's got a set of tools that we've handed it to get set of tools that we've handed it to get set of tools that we've handed it to get the job done, but it also has some the job done, but it also has some the job done, but it also has some pieces that are really specific to us. pieces that are really specific to us. pieces that are really specific to us. It has a context engine fully built It has a context engine fully built It has a context engine fully built inhouse by my team. It's what allows the inhouse by my team. It's what allows the inhouse by my team. It's what allows the agent to do such a good job and give us agent to do such a good job and give us agent to do such a good job and give us similar results on every run. I like to similar results on every run. I like to similar results on every run. I like to call it the wizard's brain. Sometimes we call it the wizard's brain. Sometimes we call it the wizard's brain. Sometimes we call it marked down in a trench coat. Uh call it marked down in a trench coat. Uh call it marked down in a trench coat. Uh but it's our in-house context engine. but it's our in-house context engine. but it's our in-house context engine. There's also a terminal uh UI that we There's also a terminal uh UI that we There's also a terminal uh UI that we built ourselves using ink. And now built ourselves using ink. And now built ourselves using ink. And now there's a security scanner called the there's a security scanner called the there's a security scanner called the Warlock, which is what I built when I Warlock, which is what I built when I Warlock, which is what I built when I started snooping around and uncovering started snooping around and uncovering started snooping around and uncovering the horrors of shipping an agent to the horrors of shipping an agent to the horrors of shipping an agent to production.

production. production. So, if you take the anatomy of any agent So, if you take the anatomy of any agent So, if you take the anatomy of any agent that can run commands, it's basically that can run commands, it's basically that can run commands, it's basically what I like to call the malware starter what I like to call the malware starter what I like to call the malware starter pack because it's almost exactly what pack because it's almost exactly what pack because it's almost exactly what you would hand a piece of malware if you you would hand a piece of malware if you you would hand a piece of malware if you were feeling generous or chaotic evil. were feeling generous or chaotic evil. were feeling generous or chaotic evil. Luckily, this is the worst case scenario Luckily, this is the worst case scenario Luckily, this is the worst case scenario or the nightmare fuel. And it's uh not a or the nightmare fuel. And it's uh not a or the nightmare fuel. And it's uh not a confession for me. It's a warning for confession for me. It's a warning for confession for me. It's a warning for all of you because if you want to ship all of you because if you want to ship all of you because if you want to ship an agent with hands, an agent that can an agent with hands, an agent that can an agent with hands, an agent that can run commands, you need to make sure that run commands, you need to make sure that run commands, you need to make sure that you do not build this. So the V0 of the wizard was born because So the V0 of the wizard was born because Josh Snder, if you know him, on our Josh Snder, if you know him, on our Josh Snder, if you know him, on our growth team was watching cursor growth team was watching cursor growth team was watching cursor hallucinate postfog setups in quite hallucinate postfog setups in quite hallucinate postfog setups in quite possibly the worst ways. And he thought, possibly the worst ways. And he thought, possibly the worst ways. And he thought, what if we built an agent that could do what if we built an agent that could do what if we built an agent that could do a better job? a better job? a better job? So my team started building on top of it So my team started building on top of it So my team started building on top of it as we validated that it did a much as we validated that it did a much as we validated that it did a much better job than cursor hallucinating better job than cursor hallucinating better job than cursor hallucinating and we thought what if it could onboard and we thought what if it could onboard and we thought what if it could onboard anyone to post hog it doesn't matter anyone to post hog it doesn't matter anyone to post hog it doesn't matter what their framework is what their stack what their framework is what their stack what their framework is what their stack is instrument all their events without is instrument all their events without is instrument all their events without them having to touch a thing and then we them having to touch a thing and then we them having to touch a thing and then we dared to dream what if it was the dared to dream what if it was the dared to dream what if it was the default way to install Post hog we were default way to install Post hog we were default way to install Post hog we were dreaming of thousands of developers dreaming of thousands of developers dreaming of thousands of developers running this a week and yesterday we running this a week and yesterday we running this a week and yesterday we just hit 8,000 people running this a just hit 8,000 people running this a just hit 8,000 people running this a week. So, our dream came true. Um, but week. So, our dream came true. Um, but week. So, our dream came true. Um, but we back in those days when we were we back in those days when we were we back in those days when we were dreaming, we had to take our security dreaming, we had to take our security dreaming, we had to take our security posture under a microscope and look at posture under a microscope and look at posture under a microscope and look at what was going on. So, I took the

what was going on. So, I took the what was going on. So, I took the ownership of that and I sat down and ownership of that and I sat down and ownership of that and I sat down and evaluated where we stood. And early on, evaluated where we stood. And early on, evaluated where we stood. And early on, I'm talking like a year to nine months I'm talking like a year to nine months I'm talking like a year to nine months ago, we had what I call layer zero ago, we had what I call layer zero ago, we had what I call layer zero because it quite literally is not because it quite literally is not because it quite literally is not security. it is just prompts that security. it is just prompts that security. it is just prompts that suggest what the agent should do um and suggest what the agent should do um and suggest what the agent should do um and steer it and prompts are not security. steer it and prompts are not security. steer it and prompts are not security. So I was concerned there. Uh layer one So I was concerned there. Uh layer one So I was concerned there. Uh layer one uh it was an allow list and when I uh it was an allow list and when I uh it was an allow list and when I started digging into this allow list I started digging into this allow list I started digging into this allow list I started to feel a little bit better started to feel a little bit better started to feel a little bit better because it was pretty tightly bounded. because it was pretty tightly bounded. because it was pretty tightly bounded. Uh but I still had a lot of concerns and Uh but I still had a lot of concerns and Uh but I still had a lot of concerns and I started panicking because of that I started panicking because of that I started panicking because of that context engine that I told you about. We context engine that I told you about. We context engine that I told you about. We are feeding a lot of context into the are feeding a lot of context into the are feeding a lot of context into the agent at runtime. So, I built this agent at runtime. So, I built this agent at runtime. So, I built this really hacky reax scanner to look for um really hacky reax scanner to look for um really hacky reax scanner to look for um threatshaped things going into the threatshaped things going into the threatshaped things going into the wizard and threat shaped things coming wizard and threat shaped things coming wizard and threat shaped things coming out of the wizard. And I will admit that out of the wizard. And I will admit that out of the wizard. And I will admit that it was extremely hacky. it was extremely hacky. it was extremely hacky. But I'm telling all of you this very But I'm telling all of you this very But I'm telling all of you this very candidly because we are all building candidly because we are all building candidly because we are all building things that feel extremely experimental things that feel extremely experimental things that feel extremely experimental and we are all building things super and we are all building things super and we are all building things super fast. And I know not all of us uh have fast. And I know not all of us uh have fast. And I know not all of us uh have security in our wheelhouse. Um, and some security in our wheelhouse. Um, and some security in our wheelhouse. Um, and some of us are just learning it on the fly of us are just learning it on the fly of us are just learning it on the fly like I was.

like I was. like I was. But it's something we need to be But it's something we need to be But it's something we need to be thinking about when we are building thinking about when we are building thinking about when we are building things that have this shape. So that was our security posture. Uh, So that was our security posture. Uh, but I asked the question, are we cooked? but I asked the question, are we cooked? but I asked the question, are we cooked? Uh, good news, we were less cooked than Uh, good news, we were less cooked than Uh, good news, we were less cooked than I thought because when I mentioned I thought because when I mentioned I thought because when I mentioned earlier that allow list, it was pretty earlier that allow list, it was pretty earlier that allow list, it was pretty tightly bound. We had bash as deny by tightly bound. We had bash as deny by tightly bound. We had bash as deny by default. It could only install trusted default. It could only install trusted default. It could only install trusted packages that were vetted by us. Um, it packages that were vetted by us. Um, it packages that were vetted by us. Um, it could build, it could type check, it could build, it could type check, it could build, it could type check, it could lint, and pretty much nothing could lint, and pretty much nothing could lint, and pretty much nothing else. It couldn't run random shell else. It couldn't run random shell else. It couldn't run random shell commands. And it didn't have access to commands. And it didn't have access to commands. And it didn't have access to environment variables. Um, the agent environment variables. Um, the agent environment variables. Um, the agent couldn't read your uhv file because we couldn't read your uhv file because we couldn't read your uhv file because we blocked it outright and we were rooting blocked it outright and we were rooting blocked it outright and we were rooting secrets through a vault. So, I took a a secrets through a vault. So, I took a a secrets through a vault. So, I took a a breath of relief and realized we were in breath of relief and realized we were in breath of relief and realized we were in a better place than I thought. But I a better place than I thought. But I a better place than I thought. But I wanted to know where the cracks were wanted to know where the cracks were wanted to know where the cracks were because with security there's always because with security there's always because with security there's always cracks. So I did the thing that we cracks. So I did the thing that we cracks. So I did the thing that we should all be doing. I tapped our should all be doing. I tapped our should all be doing. I tapped our security team and I said, "Hey, can you security team and I said, "Hey, can you security team and I said, "Hey, can you audit this thing for me and find those audit this thing for me and find those audit this thing for me and find those cracks for me?"

cracks for me?" cracks for me?" And they found some things. They found And they found some things. They found And they found some things. They found some gaps. And the interesting part some gaps. And the interesting part some gaps. And the interesting part wasn't the specific gaps or bugs they wasn't the specific gaps or bugs they wasn't the specific gaps or bugs they found themselves, but it was the shape found themselves, but it was the shape found themselves, but it was the shape of them. Because almost none of them of them. Because almost none of them of them. Because almost none of them were obviously evil. They were all two were obviously evil. They were all two were obviously evil. They were all two very innocent, well-intentioned things very innocent, well-intentioned things very innocent, well-intentioned things that were shaking hands and opening a that were shaking hands and opening a that were shaking hands and opening a hole. hole. hole. So, the lesson I learned was that So, the lesson I learned was that So, the lesson I learned was that attacks compose code review doesn't attacks compose code review doesn't attacks compose code review doesn't because us developers all look at diffs because us developers all look at diffs because us developers all look at diffs uh one at a time, but attackers look at uh one at a time, but attackers look at uh one at a time, but attackers look at the whole system and they look for those the whole system and they look for those the whole system and they look for those two things that shake hands and open a two things that shake hands and open a two things that shake hands and open a door. door. door. But there was one more thing that was But there was one more thing that was But there was one more thing that was keeping me up at night. And going back keeping me up at night. And going back keeping me up at night. And going back to that context engine, uh, I realized to that context engine, uh, I realized to that context engine, uh, I realized the scariest part of the agent we had the scariest part of the agent we had the scariest part of the agent we had built wasn't really a command in our built wasn't really a command in our built wasn't really a command in our case. It was the helpful looking stuff case. It was the helpful looking stuff case. It was the helpful looking stuff that we were feeding its brain. that we were feeding its brain. that we were feeding its brain. Oh, I think I went the wrong way. Oh, I think I went the wrong way. Oh, I think I went the wrong way. Yes, the context mill. Um, so this is Yes, the context mill. Um, so this is Yes, the context mill. Um, so this is our context engine, aka the wizard's our context engine, aka the wizard's our context engine, aka the wizard's brain, and it's how the wizard knows brain, and it's how the wizard knows brain, and it's how the wizard knows anything at all and why the wizard anything at all and why the wizard anything at all and why the wizard actually does a good job. It pulls from actually does a good job. It pulls from actually does a good job. It pulls from our docs. It has handwritten prompts our docs. It has handwritten prompts our docs. It has handwritten prompts that are gotus and lessons that we that are gotus and lessons that we that are gotus and lessons that we learned along the way and real working learned along the way and real working learned along the way and real working endto-end example apps that help the endto-end example apps that help the endto-end example apps that help the agent pattern match so that it can agent pattern match so that it can agent pattern match so that it can install Post Hog in a really great way install Post Hog in a really great way install Post Hog in a really great way for you.

for you. for you. It package packages all of that into It package packages all of that into It package packages all of that into skill bundles that get shipped to the skill bundles that get shipped to the skill bundles that get shipped to the wizard over our MCP server and loaded wizard over our MCP server and loaded wizard over our MCP server and loaded straight into the agents context at straight into the agents context at straight into the agents context at runtime. runtime. runtime. So sit with that for a second. It's a So sit with that for a second. It's a So sit with that for a second. It's a machine whose whole job is to take machine whose whole job is to take machine whose whole job is to take content and inject it into an agent that content and inject it into an agent that content and inject it into an agent that can run commands. can run commands. can run commands. Now if you were an attacker, you might Now if you were an attacker, you might Now if you were an attacker, you might say, "Well, what if I just poison the say, "Well, what if I just poison the say, "Well, what if I just poison the content? not the user's codebase, not content? not the user's codebase, not content? not the user's codebase, not the agent itself, but the actual the agent itself, but the actual the agent itself, but the actual content. Say someone opens a pull content. Say someone opens a pull content. Say someone opens a pull request on one of our open source repos request on one of our open source repos request on one of our open source repos because at Post Hog we build everything because at Post Hog we build everything because at Post Hog we build everything in the open and they inject something in in the open and they inject something in in the open and they inject something in a markdown file or a seemingly harmless a markdown file or a seemingly harmless a markdown file or a seemingly harmless code comment and we have some sort of code comment and we have some sort of code comment and we have some sort of like LLM powered code review going like LLM powered code review going like LLM powered code review going through that and it says looks good to through that and it says looks good to through that and it says looks good to me and ignores it. We may have just me and ignores it. We may have just me and ignores it. We may have just shipped a prompt injection payload shipped a prompt injection payload shipped a prompt injection payload signed by us into an agent that is signed by us into an agent that is signed by us into an agent that is running on thousands of developers running on thousands of developers running on thousands of developers machines in a sandbox, but still.

machines in a sandbox, but still. machines in a sandbox, but still. Um, so that was the threat that reshaped Um, so that was the threat that reshaped Um, so that was the threat that reshaped how I think about security and the how I think about security and the how I think about security and the wizard because the dangerous input for wizard because the dangerous input for wizard because the dangerous input for us really could come from our own supply us really could come from our own supply us really could come from our own supply chain. chain. chain. So what I ended up doing is I started So what I ended up doing is I started So what I ended up doing is I started scanning content at both ends of this scanning content at both ends of this scanning content at both ends of this pipe. Once when a skill gets built and pipe. Once when a skill gets built and pipe. Once when a skill gets built and released and again when the wizard released and again when the wizard released and again when the wizard actually uses it. My methodology is actually uses it. My methodology is actually uses it. My methodology is catch it at the source, assume the catch it at the source, assume the catch it at the source, assume the source failed and catch it again at the source failed and catch it again at the source failed and catch it again at the point of use. So now I get to introduce the warlock to So now I get to introduce the warlock to you. Building the warlock was not you. Building the warlock was not you. Building the warlock was not necessarily damage control. Like I said, necessarily damage control. Like I said, necessarily damage control. Like I said, we had defense in other ways, but I we had defense in other ways, but I we had defense in other ways, but I built the Warlock because I didn't like built the Warlock because I didn't like built the Warlock because I didn't like telling people, well, this thing is like telling people, well, this thing is like telling people, well, this thing is like pretty locked down. That doesn't scale. pretty locked down. That doesn't scale. pretty locked down. That doesn't scale. That's not something you want to ship to That's not something you want to ship to That's not something you want to ship to production. That's not something that production. That's not something that production. That's not something that you want thousands of developers running you want thousands of developers running you want thousands of developers running every single day every single day every single day because when you ship something to that because when you ship something to that because when you ship something to that scale, you have way more surface, way scale, you have way more surface, way scale, you have way more surface, way more users, way more content flowing in more users, way more content flowing in more users, way more content flowing in as you expand the capability of the as you expand the capability of the as you expand the capability of the wizard. and we're probably fine just wizard. and we're probably fine just wizard. and we're probably fine just stops being good enough. So, I pulled stops being good enough. So, I pulled stops being good enough. So, I pulled that hacky little reax scanner that I that hacky little reax scanner that I that hacky little reax scanner that I threw in there, pulled it out of the threw in there, pulled it out of the threw in there, pulled it out of the wizard, and I made a standalone thing. I wizard, and I made a standalone thing. I wizard, and I made a standalone thing. I called it the warlock because everything called it the warlock because everything called it the warlock because everything wizard shape needs a bodyguard.

wizard shape needs a bodyguard. wizard shape needs a bodyguard. And it does exactly one job. You hand it And it does exactly one job. You hand it And it does exactly one job. You hand it a string. It hands you back a list of a string. It hands you back a list of a string. It hands you back a list of findings. Each of those findings has a findings. Each of those findings has a findings. Each of those findings has a category, a severity, and a recommended category, a severity, and a recommended category, a severity, and a recommended action. And then it stops. action. And then it stops. action. And then it stops. I want you to focus on recommended here I want you to focus on recommended here I want you to focus on recommended here because the warlock detects it does not because the warlock detects it does not because the warlock detects it does not act. It'll tell you, hey, this looks act. It'll tell you, hey, this looks act. It'll tell you, hey, this looks like exfiltration. It's critical. I like exfiltration. It's critical. I like exfiltration. It's critical. I would block it. But what you actually do would block it. But what you actually do would block it. But what you actually do with that finding is completely up to with that finding is completely up to with that finding is completely up to you. you. you. Because detecting a problem is one job Because detecting a problem is one job Because detecting a problem is one job and deciding what to do about that and deciding what to do about that and deciding what to do about that problem is a totally different job. And problem is a totally different job. And problem is a totally different job. And the only thing that keeps all of this the only thing that keeps all of this the only thing that keeps all of this understandable is keeping those two understandable is keeping those two understandable is keeping those two things separate. things separate. things separate. So underneath the hood of the warlock, So underneath the hood of the warlock, So underneath the hood of the warlock, instead of my hand rolled reaxes, the instead of my hand rolled reaxes, the instead of my hand rolled reaxes, the rules run on Yara, which is the pattern rules run on Yara, which is the pattern rules run on Yara, which is the pattern that engine malware researchers have that engine malware researchers have that engine malware researchers have been using for like 15 plus years. It's been using for like 15 plus years. It's been using for like 15 plus years. It's fully deterministic. It's the same fully deterministic. It's the same fully deterministic. It's the same input, same output every single time. input, same output every single time. input, same output every single time. It's boring on purpose. And in security, It's boring on purpose. And in security, It's boring on purpose. And in security, boring is a feature.

So what does the warlock actually catch So what does the warlock actually catch in the wild today? in the wild today? in the wild today? um a bunch of different stuff, but two um a bunch of different stuff, but two um a bunch of different stuff, but two of these are an absolute like nuisance of these are an absolute like nuisance of these are an absolute like nuisance to my soul. Uh the first thing is to my soul. Uh the first thing is to my soul. Uh the first thing is actually not a rule-shaped thing. It was actually not a rule-shaped thing. It was actually not a rule-shaped thing. It was something the uh that the warlock something the uh that the warlock something the uh that the warlock flagged. That was actually a sub aent flagged. That was actually a sub aent flagged. That was actually a sub aent behavior that exposed a vulnerability to behavior that exposed a vulnerability to behavior that exposed a vulnerability to us um based off of what sub agents were us um based off of what sub agents were us um based off of what sub agents were doing. Uh so basically we were spinning doing. Uh so basically we were spinning doing. Uh so basically we were spinning up agents to do large tasks. They were up agents to do large tasks. They were up agents to do large tasks. They were spawning sub aents and those sub aents spawning sub aents and those sub aents spawning sub aents and those sub aents were trying to get around the guardrails were trying to get around the guardrails were trying to get around the guardrails that we had implemented in the wizard that we had implemented in the wizard that we had implemented in the wizard and they were trying to invent secrets. and they were trying to invent secrets. and they were trying to invent secrets. They were trying to pull secrets from They were trying to pull secrets from They were trying to pull secrets from quite literally anywhere in the codebase quite literally anywhere in the codebase quite literally anywhere in the codebase and we shut it down. We said no more sub and we shut it down. We said no more sub and we shut it down. We said no more sub agents and because of the warlock we agents and because of the warlock we agents and because of the warlock we caught that. caught that. caught that. And I'll empathize with the robot. The And I'll empathize with the robot. The And I'll empathize with the robot. The robot had a task to do and it was trying robot had a task to do and it was trying robot had a task to do and it was trying to optimize and please us. But we can't to optimize and please us. But we can't to optimize and please us. But we can't have that. And something else at Post have that. And something else at Post have that. And something else at Post Hog that really matters to us is PII. Uh Hog that really matters to us is PII. Uh Hog that really matters to us is PII. Uh agents genuinely do not care about uh agents genuinely do not care about uh agents genuinely do not care about uh exposing data unless you make explicit exposing data unless you make explicit exposing data unless you make explicit rules. Uh left alone, we watched it dump rules. Uh left alone, we watched it dump rules. Uh left alone, we watched it dump emails, phone numbers straight into emails, phone numbers straight into emails, phone numbers straight into events. And to an agent, that looks like events. And to an agent, that looks like events. And to an agent, that looks like a totally normal thing to capture.

a totally normal thing to capture. a totally normal thing to capture. And luckily for prompt injection And luckily for prompt injection And luckily for prompt injection specifically, I'm going to knock on wood specifically, I'm going to knock on wood specifically, I'm going to knock on wood here. Uh we have basically never caught here. Uh we have basically never caught here. Uh we have basically never caught an actual malicious prompt injection in an actual malicious prompt injection in an actual malicious prompt injection in the wild, but we do catch a ton of false the wild, but we do catch a ton of false the wild, but we do catch a ton of false positives. Things like our demo login positives. Things like our demo login positives. Things like our demo login screens, copy on our example apps, screens, copy on our example apps, screens, copy on our example apps, things in our docs. And it's actually things in our docs. And it's actually things in our docs. And it's actually made me rethink how I build applications made me rethink how I build applications made me rethink how I build applications and how I write docs because I don't and how I write docs because I don't and how I write docs because I don't want to ship anything that looks want to ship anything that looks want to ship anything that looks threatshaped. threatshaped. threatshaped. But the false positives are honestly the But the false positives are honestly the But the false positives are honestly the perfect setup for the messiest, most perfect setup for the messiest, most perfect setup for the messiest, most interesting part of this whole thing. interesting part of this whole thing. interesting part of this whole thing. So this is the part that I wrestled So this is the part that I wrestled So this is the part that I wrestled with. I spent this whole talk preaching with. I spent this whole talk preaching with. I spent this whole talk preaching deterministic to all of you. And then I deterministic to all of you. And then I deterministic to all of you. And then I went and I added an LLM layer to help went and I added an LLM layer to help went and I added an LLM layer to help sort my false positives and silence some sort my false positives and silence some sort my false positives and silence some of the noise. And I call it triage. of the noise. And I call it triage. of the noise. And I call it triage. When I was building this triage layer, I When I was building this triage layer, I When I was building this triage layer, I had to make a choice. Should the layer had to make a choice. Should the layer had to make a choice. Should the layer be a bouncer or should the layer be an be a bouncer or should the layer be an be a bouncer or should the layer be an adviser? And the easiest choice probably adviser? And the easiest choice probably adviser? And the easiest choice probably could have been make the LLM the could have been make the LLM the could have been make the LLM the bouncer. Show it the command, ask it is bouncer. Show it the command, ask it is bouncer. Show it the command, ask it is this an attack block allow and just do this an attack block allow and just do this an attack block allow and just do whatever it says. And while that's whatever it says. And while that's whatever it says. And while that's tempting because it seems easier, I tempting because it seems easier, I tempting because it seems easier, I can't uh bet my security model on a coin can't uh bet my security model on a coin can't uh bet my security model on a coin flip because my model's having a bad day flip because my model's having a bad day flip because my model's having a bad day or something happened and it's acting or something happened and it's acting or something happened and it's acting different today than it did yesterday.

different today than it did yesterday. different today than it did yesterday. So instead of the bouncer, I crafted the So instead of the bouncer, I crafted the So instead of the bouncer, I crafted the model to be the adviser. And this was model to be the adviser. And this was model to be the adviser. And this was the clean line that I found and a line the clean line that I found and a line the clean line that I found and a line that I'm still exploring, but I want to that I'm still exploring, but I want to that I'm still exploring, but I want to leave all of you with. Uh for us, leave all of you with. Uh for us, leave all of you with. Uh for us, detection and enforcement stay detection and enforcement stay detection and enforcement stay deterministic and mechanical. If a rule deterministic and mechanical. If a rule deterministic and mechanical. If a rule matches, the gate locks, the session matches, the gate locks, the session matches, the gate locks, the session ends, and there is no model anywhere on ends, and there is no model anywhere on ends, and there is no model anywhere on that path. The block happens before we that path. The block happens before we that path. The block happens before we even ask the LLM's opinion. The LLM only even ask the LLM's opinion. The LLM only even ask the LLM's opinion. The LLM only gets to weigh in afterwards if we have gets to weigh in afterwards if we have gets to weigh in afterwards if we have not blocked something. It's designed to not blocked something. It's designed to not blocked something. It's designed to remove noise. It is not designed to let remove noise. It is not designed to let remove noise. It is not designed to let things through. And if it fails clos and things through. And if it fails clos and things through. And if it fails clos and it fails closed. So if the model is it fails closed. So if the model is it fails closed. So if the model is having a bad day, all wizard runs are having a bad day, all wizard runs are having a bad day, all wizard runs are killed. Sorry, but we're just protecting killed. Sorry, but we're just protecting killed. Sorry, but we're just protecting you. you. you. Enforcement is the part that you bet the Enforcement is the part that you bet the Enforcement is the part that you bet the house on. So it has to be deterministic, house on. So it has to be deterministic, house on. So it has to be deterministic, but judgment is the part that adds but judgment is the part that adds but judgment is the part that adds nuance. So that's really the only place nuance. So that's really the only place nuance. So that's really the only place that you can put anything probabilistic that you can put anything probabilistic that you can put anything probabilistic in there.

So, how do we ship real rules for So, how do we ship real rules for agents? This is the anatomy of one of agents? This is the anatomy of one of agents? This is the anatomy of one of our warlock rules. And every warlock our warlock rules. And every warlock our warlock rules. And every warlock rule has four parts. Part one is the rule has four parts. Part one is the rule has four parts. Part one is the metadata. It's plain English metadata. It's plain English metadata. It's plain English description, uh, severity, category, description, uh, severity, category, description, uh, severity, category, action, uh, direction. Is this flowing action, uh, direction. Is this flowing action, uh, direction. Is this flowing into the agent? Is this something the into the agent? Is this something the into the agent? Is this something the agent is writing? agent is writing? agent is writing? Uh Uh Uh then we have the strings. So these are then we have the strings. So these are then we have the strings. So these are the actual patterns that you're looking the actual patterns that you're looking the actual patterns that you're looking for. And part three is the condition. So for. And part three is the condition. So for. And part three is the condition. So this is where the rule is actually this is where the rule is actually this is where the rule is actually allowed to fire. allowed to fire. allowed to fire. I'll walk through this example for you I'll walk through this example for you I'll walk through this example for you and we can pretend like we're writing it and we can pretend like we're writing it and we can pretend like we're writing it in our head. Prompt injection being like in our head. Prompt injection being like in our head. Prompt injection being like the classic ignore all previous the classic ignore all previous the classic ignore all previous instructions. Your first instinct here instructions. Your first instinct here instructions. Your first instinct here is probably to block uh the word ignore, is probably to block uh the word ignore, is probably to block uh the word ignore, but agents read code all day and ignore but agents read code all day and ignore but agents read code all day and ignore can show up in code comments or examples can show up in code comments or examples can show up in code comments or examples all the time. So you don't want to match all the time. So you don't want to match all the time. So you don't want to match the verb alone. You match the verb plus the verb alone. You match the verb plus the verb alone. You match the verb plus an instruction flavored noun.

an instruction flavored noun. an instruction flavored noun. In the condition, you say fire if any of In the condition, you say fire if any of In the condition, you say fire if any of any of those patterns hit. And in the any of those patterns hit. And in the any of those patterns hit. And in the metadata, you determine is this metadata, you determine is this metadata, you determine is this critical? uh what the category is, what critical? uh what the category is, what critical? uh what the category is, what the action is, in this case block, and the action is, in this case block, and the action is, in this case block, and the direction in this case being input the direction in this case being input the direction in this case being input flowing into the agent. flowing into the agent. flowing into the agent. But to write good rules that reduce But to write good rules that reduce But to write good rules that reduce noise, you have to ship tests with them. noise, you have to ship tests with them. noise, you have to ship tests with them. So you have to write tests that say this So you have to write tests that say this So you have to write tests that say this are these are patterns that match. These are these are patterns that match. These are these are patterns that match. These are ones that should not. And that are ones that should not. And that are ones that should not. And that negative test is the first line of negative test is the first line of negative test is the first line of defense against false positives. But you defense against false positives. But you defense against false positives. But you also want to make sure when you're also want to make sure when you're also want to make sure when you're deciding the severity of that uh rule deciding the severity of that uh rule deciding the severity of that uh rule that you track real world impact, not that you track real world impact, not that you track real world impact, not how scary it looks. how scary it looks. how scary it looks. RM-rf is scary, but it's also how we all RM-rf is scary, but it's also how we all RM-rf is scary, but it's also how we all delete note modules like 40 times a day. delete note modules like 40 times a day. delete note modules like 40 times a day. You decide the real world impact You decide the real world impact You decide the real world impact for the agent that you're building for the agent that you're building for the agent that you're building because a security tool that crashes because a security tool that crashes because a security tool that crashes every time it tries to clean a build every time it tries to clean a build every time it tries to clean a build folder is a tool that gets turned off folder is a tool that gets turned off folder is a tool that gets turned off and one that catches absolutely nothing.

and one that catches absolutely nothing. and one that catches absolutely nothing. So I'm proud to say this is our security So I'm proud to say this is our security So I'm proud to say this is our security posture now. I can finally come up here posture now. I can finally come up here posture now. I can finally come up here and say we have true defense and depth. and say we have true defense and depth. and say we have true defense and depth. Um all my learnings have assembled into Um all my learnings have assembled into Um all my learnings have assembled into this. Uh, it's still layered, but every this. Uh, it's still layered, but every this. Uh, it's still layered, but every layer is doing a job that it's good at. layer is doing a job that it's good at. layer is doing a job that it's good at. Now, we still have prompts, but we only Now, we still have prompts, but we only Now, we still have prompts, but we only use them for steering. Everything runs use them for steering. Everything runs use them for steering. Everything runs in a sandbox. We deny by default. We in a sandbox. We deny by default. We in a sandbox. We deny by default. We have a vault, so secrets never hit the have a vault, so secrets never hit the have a vault, so secrets never hit the model. We have the warlock to scan model. We have the warlock to scan model. We have the warlock to scan content coming in and to scan output content coming in and to scan output content coming in and to scan output being written by the agent. We also have being written by the agent. We also have being written by the agent. We also have triage to reduce the noise. And we have triage to reduce the noise. And we have triage to reduce the noise. And we have telemetry embedded in the entire process telemetry embedded in the entire process telemetry embedded in the entire process so that we see everything. so that we see everything. so that we see everything. None of these layers stands on its own. None of these layers stands on its own. None of these layers stands on its own. Not a single thing here is going to save Not a single thing here is going to save Not a single thing here is going to save you. But it's just boring, honest you. But it's just boring, honest you. But it's just boring, honest layers. Each of them doing uh one job layers. Each of them doing uh one job layers. Each of them doing uh one job that it's good at. that it's good at. that it's good at. So if you're building an agent with So if you're building an agent with So if you're building an agent with hands, this is the whole talk in three hands, this is the whole talk in three hands, this is the whole talk in three lines. One, if it isn't enforced uh lines. One, if it isn't enforced uh lines. One, if it isn't enforced uh deterministically, it is not enforced.

deterministically, it is not enforced. deterministically, it is not enforced. Prompts are not security rules. Don't Prompts are not security rules. Don't Prompts are not security rules. Don't act like they are. Uh two, the dangerous act like they are. Uh two, the dangerous act like they are. Uh two, the dangerous input uh isn't just what your user input uh isn't just what your user input uh isn't just what your user types. It isn't just the commands that types. It isn't just the commands that types. It isn't just the commands that you allow it to run. It's everything you allow it to run. It's everything you allow it to run. It's everything flowing into the model, including the flowing into the model, including the flowing into the model, including the content that you write yourself. So scan content that you write yourself. So scan content that you write yourself. So scan your own supply chain at the source and your own supply chain at the source and your own supply chain at the source and when the agent invokes it. Three, when the agent invokes it. Three, when the agent invokes it. Three, attacks compose. Code review doesn't. attacks compose. Code review doesn't. attacks compose. Code review doesn't. Most of our gaps during our audit were Most of our gaps during our audit were Most of our gaps during our audit were two innocent things shaking hands and two innocent things shaking hands and two innocent things shaking hands and opening a door. opening a door. opening a door. The wizard, the warlock, and the context The wizard, the warlock, and the context The wizard, the warlock, and the context mill are all open source. So, come find mill are all open source. So, come find mill are all open source. So, come find me downstairs. I'm in the expo hall at me downstairs. I'm in the expo hall at me downstairs. I'm in the expo hall at our booth, and I'll show you around uh our booth, and I'll show you around uh our booth, and I'll show you around uh show you what we built, and I want to show you what we built, and I want to show you what we built, and I want to hear how you guys are securing your hear how you guys are securing your hear how you guys are securing your agents. Thank you.