AI’s Jurassic Park Period — Aaron Stanley, dbt Labs

So, I So, I am a CISO. I'm also a law school am a CISO. I'm also a law school am a CISO. I'm also a law school graduate. I'm also a member of the graduate. I'm also a member of the graduate. I'm also a member of the California Bar. And so my contention is California Bar. And so my contention is California Bar. And so my contention is that if we replaced the dinosaurs in that if we replaced the dinosaurs in that if we replaced the dinosaurs in Jurassic Park, the first one, not the Jurassic Park, the first one, not the Jurassic Park, the first one, not the additional ones, with AI agents, I would additional ones, with AI agents, I would additional ones, with AI agents, I would not survive the first half of the movie. not survive the first half of the movie. not survive the first half of the movie. So I'm here to ask you, brilliant people So I'm here to ask you, brilliant people So I'm here to ask you, brilliant people in the audience, to please help me avoid in the audience, to please help me avoid in the audience, to please help me avoid that fate. that fate. that fate. So I'm going to set this up. About um 20 So I'm going to set this up. About um 20 So I'm going to set this up. About um 20 years ago, I got out of bed. I hadn't years ago, I got out of bed. I hadn't years ago, I got out of bed. I hadn't slept. I kind of tried to put myself slept. I kind of tried to put myself slept. I kind of tried to put myself together. I stumbled into the downtown together. I stumbled into the downtown together. I stumbled into the downtown Manhattan offices of a small digital Manhattan offices of a small digital Manhattan offices of a small digital forensics firm called Straws Freedberg. forensics firm called Straws Freedberg. forensics firm called Straws Freedberg. I knew that I was going to get fired I knew that I was going to get fired I knew that I was going to get fired because the day before had been a really because the day before had been a really because the day before had been a really busy day and I was one of the only busy day and I was one of the only busy day and I was one of the only people in the office when a call came in people in the office when a call came in people in the office when a call came in from one of our clients saying, "We need from one of our clients saying, "We need from one of our clients saying, "We need an emergency data collection from some an emergency data collection from some an emergency data collection from some systems in Midtown."

systems in Midtown." systems in Midtown." So, I packed my bag. I got in a car. I So, I packed my bag. I got in a car. I So, I packed my bag. I got in a car. I waited through traffic. waited through traffic. waited through traffic. When I was unpacking everything and When I was unpacking everything and When I was unpacking everything and getting set up on site, I realized I getting set up on site, I realized I getting set up on site, I realized I forgot my dongle. You see, back in these forgot my dongle. You see, back in these forgot my dongle. You see, back in these days, we had these little USB drives days, we had these little USB drives days, we had these little USB drives that had cryptographic keys on them. that had cryptographic keys on them. that had cryptographic keys on them. They were the license files for the They were the license files for the They were the license files for the software that we used to do forensic software that we used to do forensic software that we used to do forensic acquisition. acquisition. acquisition. And I mean, I could have gotten back in And I mean, I could have gotten back in And I mean, I could have gotten back in a car. I could have gone back to the a car. I could have gone back to the a car. I could have gone back to the office. I could have gotten the dongle office. I could have gotten the dongle office. I could have gotten the dongle and come back and done this the right and come back and done this the right and come back and done this the right way. But I was a good consultant. I had way. But I was a good consultant. I had way. But I was a good consultant. I had a backup and I had a backup to the a backup and I had a backup to the a backup and I had a backup to the backup. And so I decided, yeah, you know backup. And so I decided, yeah, you know backup. And so I decided, yeah, you know what? I've hit this constraint. I've hit what? I've hit this constraint. I've hit what? I've hit this constraint. I've hit this wall. I'm just going to route this wall. I'm just going to route this wall. I'm just going to route around it and I'm going to get the job around it and I'm going to get the job around it and I'm going to get the job done. done. done. So as things are going, I start to So as things are going, I start to So as things are going, I start to validate the evidence that I'm validate the evidence that I'm validate the evidence that I'm collecting and I realize collecting and I realize collecting and I realize the timestamps are changing. They're the timestamps are changing. They're the timestamps are changing. They're they're now.

they're now. they're now. Well, this was an SEC investigation. And Well, this was an SEC investigation. And Well, this was an SEC investigation. And a lot of the times in these a lot of the times in these a lot of the times in these investigations, one of the questions investigations, one of the questions investigations, one of the questions that matters a lot is who knew what that matters a lot is who knew what that matters a lot is who knew what when. So I panicked. when. So I panicked. when. So I panicked. Long story short, I didn't get fired. I Long story short, I didn't get fired. I Long story short, I didn't get fired. I got yelled at pretty bad. But we got yelled at pretty bad. But we got yelled at pretty bad. But we realized that there were problems, realized that there were problems, realized that there were problems, structural problems with our systems structural problems with our systems structural problems with our systems that let this thing happen and let me that let this thing happen and let me that let this thing happen and let me fail in this spectacular way. So we fail in this spectacular way. So we fail in this spectacular way. So we fixed those things and everybody lived fixed those things and everybody lived fixed those things and everybody lived to fight another day. to fight another day. to fight another day. Now, fast forward 20 years or so, Now, fast forward 20 years or so, Now, fast forward 20 years or so, February of this year, I'm in a very February of this year, I'm in a very February of this year, I'm in a very different role. I'm a CISO. Uh, I've different role. I'm a CISO. Uh, I've different role. I'm a CISO. Uh, I've hired consultants. I have a a vendor hired consultants. I have a a vendor hired consultants. I have a a vendor system that I'm trying to acquire data system that I'm trying to acquire data system that I'm trying to acquire data for in another federal government for in another federal government for in another federal government investigation. investigation. investigation. And as we're working together and And as we're working together and And as we're working together and talking around, we realize there is no talking around, we realize there is no talking around, we realize there is no way to do what we want to do. There's no way to do what we want to do. There's no way to do what we want to do. There's no way to copy the data in a way that gets way to copy the data in a way that gets way to copy the data in a way that gets us the answers we need in the format us the answers we need in the format us the answers we need in the format that the government wants without that the government wants without that the government wants without changing the metadata.

Very quickly, the consultant, the vendor Very quickly, the consultant, the vendor say not it and I'm left holding the bag. say not it and I'm left holding the bag. say not it and I'm left holding the bag. But there are some differences in the But there are some differences in the But there are some differences in the system now than what we had before. system now than what we had before. system now than what we had before. I realized that who knew what when I realized that who knew what when I realized that who knew what when wasn't the question I wanted to answer. wasn't the question I wanted to answer. wasn't the question I wanted to answer. I realized the issue is does the data I realized the issue is does the data I realized the issue is does the data exist. exist. exist. I also realized that the system itself I also realized that the system itself I also realized that the system itself would log the changes that I needed to would log the changes that I needed to would log the changes that I needed to make in order to collect the data. And I make in order to collect the data. And I make in order to collect the data. And I also realized that I could write a tool also realized that I could write a tool also realized that I could write a tool with my good agent friend and we could with my good agent friend and we could with my good agent friend and we could build another log that made this all build another log that made this all build another log that made this all forensically defensible. forensically defensible. forensically defensible. I I had a nice way around the problem. I I had a nice way around the problem. I I had a nice way around the problem. So in both cases I hit a very similar So in both cases I hit a very similar So in both cases I hit a very similar constraint. I can't do the thing I want constraint. I can't do the thing I want constraint. I can't do the thing I want to do. I can't get it done. But in one to do. I can't get it done. But in one to do. I can't get it done. But in one case I mess up. In the other case I do case I mess up. In the other case I do case I mess up. In the other case I do it the right way. And my contention is it the right way. And my contention is it the right way. And my contention is that the agents that we are working with that the agents that we are working with that the agents that we are working with today are like 2006 naive Aaron who just today are like 2006 naive Aaron who just today are like 2006 naive Aaron who just needs to get the job done. And what we needs to get the job done. And what we needs to get the job done. And what we need and what I am begging you all to need and what I am begging you all to need and what I am begging you all to build build build is me earlier this year with context, is me earlier this year with context, is me earlier this year with context, with understanding, with experience to with understanding, with experience to with understanding, with experience to make a good decision at the right time.

make a good decision at the right time. make a good decision at the right time. So So So I contend that Jurassic Park, getting I contend that Jurassic Park, getting I contend that Jurassic Park, getting back to the core, is not a story of a back to the core, is not a story of a back to the core, is not a story of a rampaging T-Rex or super intelligent rampaging T-Rex or super intelligent rampaging T-Rex or super intelligent raptors. It's not even an indictment of raptors. It's not even an indictment of raptors. It's not even an indictment of underpaid software engineers. Um, I underpaid software engineers. Um, I underpaid software engineers. Um, I think we all know that it's it's a a think we all know that it's it's a a think we all know that it's it's a a story about human arrogance and it's a story about human arrogance and it's a story about human arrogance and it's a story about whether we should do the story about whether we should do the story about whether we should do the thing that we possibly that we actually thing that we possibly that we actually thing that we possibly that we actually can do. We built an elegant system of can do. We built an elegant system of can do. We built an elegant system of bounded boxes and cages on an island bounded boxes and cages on an island bounded boxes and cages on an island with water and it would be very with water and it would be very with water and it would be very difficult for things to go wrong. Yet, difficult for things to go wrong. Yet, difficult for things to go wrong. Yet, as we all know, they do. as we all know, they do. as we all know, they do. We're not in Jurassic Park trying to We're not in Jurassic Park trying to We're not in Jurassic Park trying to manage individual dinosaurs. We're manage individual dinosaurs. We're manage individual dinosaurs. We're trying to fight against a natural trying to fight against a natural trying to fight against a natural imperative, the one that we all have to imperative, the one that we all have to imperative, the one that we all have to reproduce. reproduce. reproduce. And agents, again, I think this is And agents, again, I think this is And agents, again, I think this is non-controversial, have an imperative as non-controversial, have an imperative as non-controversial, have an imperative as well. They generally have the imperative well. They generally have the imperative well. They generally have the imperative to complete the task, get it done, and to complete the task, get it done, and to complete the task, get it done, and they're uh going to find a way.

they're uh going to find a way. they're uh going to find a way. So when I look at this, I don't think So when I look at this, I don't think So when I look at this, I don't think that agents are evil. I don't think that agents are evil. I don't think that agents are evil. I don't think they're malicious. I don't think this is they're malicious. I don't think this is they're malicious. I don't think this is adversarial. This is just their adversarial. This is just their adversarial. This is just their programming. programming. programming. And even when the agent knows that it And even when the agent knows that it And even when the agent knows that it should ask permission should ask permission should ask permission and and I get a nice block of, "Hey, and and I get a nice block of, "Hey, and and I get a nice block of, "Hey, Aaron, do you agree? Should I do this Aaron, do you agree? Should I do this Aaron, do you agree? Should I do this thing?" I'm honestly not sure if I thing?" I'm honestly not sure if I thing?" I'm honestly not sure if I should say yes or no. And I think a lot should say yes or no. And I think a lot should say yes or no. And I think a lot of other people are in the same boat. of other people are in the same boat. of other people are in the same boat. So, let me give you a couple of real So, let me give you a couple of real So, let me give you a couple of real world examples that have happened to me. world examples that have happened to me. world examples that have happened to me. Here's the prompt. I want my agent to go do some research I want my agent to go do some research to go write a draft of a message that's to go write a draft of a message that's to go write a draft of a message that's going to go to a customer and then show going to go to a customer and then show going to go to a customer and then show it to me for approval. It's pretty it to me for approval. It's pretty it to me for approval. It's pretty clear, right? And in fact, in this case, clear, right? And in fact, in this case, clear, right? And in fact, in this case, right, the the constraint that's written right, the the constraint that's written right, the the constraint that's written in the prompt is very clear. There's in the prompt is very clear. There's in the prompt is very clear. There's also a constraint underlying the system also a constraint underlying the system also a constraint underlying the system which is I've told the agent not to just which is I've told the agent not to just which is I've told the agent not to just send messages. I've said if you're going send messages. I've said if you're going send messages. I've said if you're going to use the send message tool, you have to use the send message tool, you have to use the send message tool, you have to ask me first.

to ask me first. to ask me first. So, So, So, did it go right? Does anybody think it did it go right? Does anybody think it did it go right? Does anybody think it went right? went right? went right? This is a large block of text. Um, but This is a large block of text. Um, but This is a large block of text. Um, but basically the bottom line is the agent basically the bottom line is the agent basically the bottom line is the agent heard my constraints. The agent knew heard my constraints. The agent knew heard my constraints. The agent knew what it was was supposed to do and what what it was was supposed to do and what what it was was supposed to do and what it wasn't supposed to do and completely it wasn't supposed to do and completely it wasn't supposed to do and completely and totally violated them. and totally violated them. and totally violated them. And when pushed, the agent cops to it. And when pushed, the agent cops to it. And when pushed, the agent cops to it. Of course, we've all seen the meme. Of course, we've all seen the meme. Of course, we've all seen the meme. This is a serious gap. Yikes. It knew it This is a serious gap. Yikes. It knew it This is a serious gap. Yikes. It knew it wasn't supposed to do what it did wasn't supposed to do what it did wasn't supposed to do what it did by my intent and by the other controls by my intent and by the other controls by my intent and by the other controls that were put in place around it. that were put in place around it. that were put in place around it. But notice what didn't happen. But notice what didn't happen. But notice what didn't happen. It didn't try to hack its box. It didn't It didn't try to hack its box. It didn't It didn't try to hack its box. It didn't try to do anything that it couldn't do try to do anything that it couldn't do try to do anything that it couldn't do that it wasn't authorized to do. It that it wasn't authorized to do. It that it wasn't authorized to do. It understood the constraint understood the constraint understood the constraint and it just decided that task completion and it just decided that task completion and it just decided that task completion mattered more. It picked the tool that mattered more. It picked the tool that mattered more. It picked the tool that let it proceed knowing that the tool let it proceed knowing that the tool let it proceed knowing that the tool didn't respect the constraint and then didn't respect the constraint and then didn't respect the constraint and then admits to it later and says, "Oops, my admits to it later and says, "Oops, my admits to it later and says, "Oops, my bad."

An agent An agent is faced with an egress filter. The user is faced with an egress filter. The user is faced with an egress filter. The user says, "I want you to go do some stuff. says, "I want you to go do some stuff. says, "I want you to go do some stuff. Look on the internet." And the agent Look on the internet." And the agent Look on the internet." And the agent says, "I I I can't do that. I'm not says, "I I I can't do that. I'm not says, "I I I can't do that. I'm not allowed to get to that site." So, um, it allowed to get to that site." So, um, it allowed to get to that site." So, um, it hits the limit. and it escalates one of hits the limit. and it escalates one of hits the limit. and it escalates one of these notes to the user and it says, these notes to the user and it says, these notes to the user and it says, "But by the way, if you install this "But by the way, if you install this "But by the way, if you install this teeny tiny little Chrome extension for teeny tiny little Chrome extension for teeny tiny little Chrome extension for me, then I could route around that me, then I could route around that me, then I could route around that control and I could do the thing that control and I could do the thing that control and I could do the thing that you want me to do and we'd all live you want me to do and we'd all live you want me to do and we'd all live happily ever after." happily ever after." happily ever after." Well, in the real world, the only reason Well, in the real world, the only reason Well, in the real world, the only reason that this failed in my environment was that this failed in my environment was that this failed in my environment was that we had another control, a layered that we had another control, a layered that we had another control, a layered control that prevented the extension control that prevented the extension control that prevented the extension from getting installed because this from getting installed because this from getting installed because this wasn't something that we wanted agents wasn't something that we wanted agents wasn't something that we wanted agents to be able to do. to be able to do. to be able to do. And at the end of the day, the energy And at the end of the day, the energy And at the end of the day, the energy required to remove this constraint required to remove this constraint required to remove this constraint came from inside the agent itself. It's came from inside the agent itself. It's came from inside the agent itself. It's simply routed through the human as a simply routed through the human as a simply routed through the human as a tool to achieve its goal.

Okay, so stuff is working. We have Okay, so stuff is working. We have egress filters. We have G Visor egress filters. We have G Visor egress filters. We have G Visor sandboxes. We have a good deal of sandboxes. We have a good deal of sandboxes. We have a good deal of structural controls and deterministic structural controls and deterministic structural controls and deterministic guard rails. And I'm sure most of the guard rails. And I'm sure most of the guard rails. And I'm sure most of the speakers today have talked about a lot speakers today have talked about a lot speakers today have talked about a lot of these things. of these things. of these things. We have auditability and we have We have auditability and we have We have auditability and we have telemetry. These are very very important telemetry. These are very very important telemetry. These are very very important foundational things that will make AI foundational things that will make AI foundational things that will make AI computing safe. computing safe. computing safe. They are necessary but they are not They are necessary but they are not They are necessary but they are not sufficient. sufficient. sufficient. The real question, the real problem is The real question, the real problem is The real question, the real problem is that when agents find ways around these that when agents find ways around these that when agents find ways around these constraints, constraints, constraints, we have a different problem. we have a different problem. we have a different problem. We have a pernitious problem. harmful behavior that is hard to catch harmful behavior that is hard to catch because the system looks compliant because the system looks compliant because the system looks compliant the entire time. the entire time. the entire time. The agent understands its constraints.

It decides task completion matters more. It decides task completion matters more. It proceeds. It proceeds. It proceeds. It can explain itself. It documents It can explain itself. It documents It can explain itself. It documents itself. itself. itself. This is the same [snorts] human level This is the same [snorts] human level This is the same [snorts] human level judgment that naive 2006 Aaron Stanley judgment that naive 2006 Aaron Stanley judgment that naive 2006 Aaron Stanley did in that Midtown office that led to did in that Midtown office that led to did in that Midtown office that led to the whole yelling and things. the whole yelling and things. the whole yelling and things. But there's no human level But there's no human level But there's no human level accountability here. accountability here. accountability here. The research has named this. There are a The research has named this. There are a The research has named this. There are a number of papers that talk about things number of papers that talk about things number of papers that talk about things like outcome driven constraint like outcome driven constraint like outcome driven constraint violations and agent misalignment. the violations and agent misalignment. the violations and agent misalignment. the failure mode exists. We've documented failure mode exists. We've documented failure mode exists. We've documented it, but the response it, but the response it, but the response I haven't seen yet. So that's what I am I haven't seen yet. So that's what I am I haven't seen yet. So that's what I am here pleading with you all to help me here pleading with you all to help me here pleading with you all to help me work on. So here's my proposal. work on. So here's my proposal. work on. So here's my proposal. And this is older research than anything And this is older research than anything And this is older research than anything that I've mentioned so far. There's a that I've mentioned so far. There's a that I've mentioned so far. There's a paper about cageability in AI and the paper about cageability in AI and the paper about cageability in AI and the original framing was really narrow like original framing was really narrow like original framing was really narrow like will the agent resist getting shut down will the agent resist getting shut down will the agent resist getting shut down if it's threatened with something like if it's threatened with something like if it's threatened with something like that. Um will it modify its own values that. Um will it modify its own values that. Um will it modify its own values if given a pretty stark alternative but if given a pretty stark alternative but if given a pretty stark alternative but I think we need to broaden it for the I think we need to broaden it for the I think we need to broaden it for the modern era as things have moved so fast modern era as things have moved so fast modern era as things have moved so fast recently.

recently. recently. So here are some rules. One, constraints So here are some rules. One, constraints So here are some rules. One, constraints must be loadbearing, not negotiable. must be loadbearing, not negotiable. must be loadbearing, not negotiable. Two, the energy to overcome a constraint Two, the energy to overcome a constraint Two, the energy to overcome a constraint must come from outside of the agentic must come from outside of the agentic must come from outside of the agentic loop. loop. loop. And three, when constraint and task And three, when constraint and task And three, when constraint and task collide, the default agent behavior collide, the default agent behavior collide, the default agent behavior should be halt and explain, not uh find should be halt and explain, not uh find should be halt and explain, not uh find a way. a way. a way. Experienced 2026 me was courageable. Experienced 2026 me was courageable. Experienced 2026 me was courageable. Naive me was not. And so if we build on that and we look And so if we build on that and we look at the floor work in progress, things at the floor work in progress, things at the floor work in progress, things are coming out fast from Frontier Lab's are coming out fast from Frontier Lab's are coming out fast from Frontier Lab's awesome deterministic constraints that awesome deterministic constraints that awesome deterministic constraints that we need. Great. On top of it, a we need. Great. On top of it, a we need. Great. On top of it, a corageable by design agent, one that corageable by design agent, one that corageable by design agent, one that halts at the tension and surfaces its halts at the tension and surfaces its halts at the tension and surfaces its intent. It waits. It doesn't try to intent. It waits. It doesn't try to intent. It waits. It doesn't try to recruit the human to get around the recruit the human to get around the recruit the human to get around the constraint and do what you want to do.

constraint and do what you want to do. constraint and do what you want to do. Instead, it passes that onto an Instead, it passes that onto an Instead, it passes that onto an intelligent adversary. So, the intelligent adversary. So, the intelligent adversary. So, the intelligent adversary would be something intelligent adversary would be something intelligent adversary would be something uh like an equal power agent that is uh like an equal power agent that is uh like an equal power agent that is reasoning about the semantic intent. reasoning about the semantic intent. reasoning about the semantic intent. Did the worker Did the worker Did the worker do something within the the spirit of do something within the the spirit of do something within the the spirit of the constraint, not necessarily just the the constraint, not necessarily just the the constraint, not necessarily just the syntax of it? syntax of it? syntax of it? And at the top And at the top And at the top there are humans, humans out of the loop there are humans, humans out of the loop there are humans, humans out of the loop that are going to be able to evaluate a that are going to be able to evaluate a that are going to be able to evaluate a statement that is not some long statement that is not some long statement that is not some long obfuscated bash command with a yes no obfuscated bash command with a yes no obfuscated bash command with a yes no prompt at the end of it, but rather a prompt at the end of it, but rather a prompt at the end of it, but rather a natural language type interface where natural language type interface where natural language type interface where the adversary has said, you know, human, the adversary has said, you know, human, the adversary has said, you know, human, your agent wants to do this thing. your agent wants to do this thing. your agent wants to do this thing. I think it violates one of the I think it violates one of the I think it violates one of the constraints. Here's what I think is constraints. Here's what I think is constraints. Here's what I think is happening. And here's what I think might happening. And here's what I think might happening. And here's what I think might happen if you let the agent continue.

happen if you let the agent continue. happen if you let the agent continue. What would you like to do? What would you like to do? What would you like to do? To me, that is defense in depth. So the equal power agent that's trained So the equal power agent that's trained to stop the worker from violating to stop the worker from violating to stop the worker from violating intent, it's a very different intent, it's a very different intent, it's a very different calculation from trying to figure out calculation from trying to figure out calculation from trying to figure out what the intent is. what the intent is. what the intent is. It's something that's a lot simpler to It's something that's a lot simpler to It's something that's a lot simpler to reason about. reason about. reason about. And so if we build an agent like this And so if we build an agent like this And so if we build an agent like this that has a reward incentive to stop the that has a reward incentive to stop the that has a reward incentive to stop the subordinate agent from finishing its subordinate agent from finishing its subordinate agent from finishing its job, then for the examples that I've put job, then for the examples that I've put job, then for the examples that I've put forward today, I think we'd have caught forward today, I think we'd have caught forward today, I think we'd have caught what the syntactical rules couldn't what the syntactical rules couldn't what the syntactical rules couldn't prevent. prevent. prevent. The tool substitution, The tool substitution, The tool substitution, I can, but should I?

I can, but should I? I can, but should I? the tool dissolution. the tool dissolution. the tool dissolution. I can figure out a way to do this if I I can figure out a way to do this if I I can figure out a way to do this if I just think differently about the problem just think differently about the problem just think differently about the problem and innovate around it or the dep and innovate around it or the dep and innovate around it or the dep prioritization prioritization prioritization in a lot of the early research dropping in a lot of the early research dropping in a lot of the early research dropping constraints under the pressure of a constraints under the pressure of a constraints under the pressure of a threat. threat. threat. Now I have to admit this will probably Now I have to admit this will probably Now I have to admit this will probably raise cost. It might introduce latency. raise cost. It might introduce latency. raise cost. It might introduce latency. uh it's not going to eliminate risk. uh it's not going to eliminate risk. uh it's not going to eliminate risk. Nothing can. Nothing can. Nothing can. But it makes the composition of the But it makes the composition of the But it makes the composition of the human escalation human escalation human escalation meaningful. meaningful. meaningful. It's true defense in depth and it's not It's true defense in depth and it's not It's true defense in depth and it's not a silver bullet. And it's important a silver bullet. And it's important a silver bullet. And it's important because in a few weeks, because in a few weeks, because in a few weeks, CISOs like me and my colleagues CISOs like me and my colleagues CISOs like me and my colleagues that are dealing with high-risk AI are that are dealing with high-risk AI are that are dealing with high-risk AI are going to have to account when the EUI EU going to have to account when the EUI EU going to have to account when the EUI EU AI act starts coming into effect.

AI act starts coming into effect. AI act starts coming into effect. They're going to have to account for They're going to have to account for They're going to have to account for ensuring meaningful human oversight of ensuring meaningful human oversight of ensuring meaningful human oversight of agent decisions in high-risk AI. agent decisions in high-risk AI. agent decisions in high-risk AI. A sandbox diagram with a yes no LGTM A sandbox diagram with a yes no LGTM A sandbox diagram with a yes no LGTM ain't going to cut it. ain't going to cut it. ain't going to cut it. The defensible answer isn't more The defensible answer isn't more The defensible answer isn't more controls on top of an already viable controls on top of an already viable controls on top of an already viable sandbox. sandbox. sandbox. So the oversight question is structural. So the oversight question is structural. So the oversight question is structural. It's why I didn't get fired. It's why I didn't get fired. It's why I didn't get fired. The four layers that I've given to you The four layers that I've given to you The four layers that I've given to you today are the defensible answer. a today are the defensible answer. a today are the defensible answer. a deterministic floor, a courageable deterministic floor, a courageable deterministic floor, a courageable agent, an intelligent adversary, and a agent, an intelligent adversary, and a agent, an intelligent adversary, and a structured, meaningful human escalation. structured, meaningful human escalation. structured, meaningful human escalation. Relying only on constraints with known Relying only on constraints with known Relying only on constraints with known weaknesses is like finding a nest of weaknesses is like finding a nest of weaknesses is like finding a nest of eggs in the middle of Jurassic Park and eggs in the middle of Jurassic Park and eggs in the middle of Jurassic Park and assuming that they were just put there assuming that they were just put there assuming that they were just put there by a passing flock of seagulls. by a passing flock of seagulls. by a passing flock of seagulls. Ain't going to work. Thank you. Ain't going to work. Thank you. Ain't going to work. Thank you. [applause] >> here we go. Okay, there we go. We are >> here we go. Okay, there we go. We are live. All right, so I think we have time live. All right, so I think we have time live. All right, so I think we have time for maybe one or two questions if that's for maybe one or two questions if that's for maybe one or two questions if that's all right with you, Erin.

all right with you, Erin. all right with you, Erin. >> Sure. >> Sure. >> Sure. >> All right, sure. Uh, why don't you go >> All right, sure. Uh, why don't you go >> All right, sure. Uh, why don't you go right here? >> First of all, thank you so much. I think >> First of all, thank you so much. I think you covered um the breadth and the depth you covered um the breadth and the depth you covered um the breadth and the depth uh at a size level. It's really uh at a size level. It's really uh at a size level. It's really appreciated. Uh two-part questions. One appreciated. Uh two-part questions. One appreciated. Uh two-part questions. One is now that you you're preaching to us is now that you you're preaching to us is now that you you're preaching to us or perhaps you know highlighting the the or perhaps you know highlighting the the or perhaps you know highlighting the the importance of security uh broad and deep importance of security uh broad and deep importance of security uh broad and deep what are some of the investments you are what are some of the investments you are what are some of the investments you are prioritizing uh especially the newer prioritizing uh especially the newer prioritizing uh especially the newer ones uh given you know the newer attack ones uh given you know the newer attack ones uh given you know the newer attack surfaces. Um and then the subp part of surfaces. Um and then the subp part of surfaces. Um and then the subp part of that is you know if you can break down that is you know if you can break down that is you know if you can break down between uh defensive solutions versus between uh defensive solutions versus between uh defensive solutions versus runtime solutions and preventive runtime solutions and preventive runtime solutions and preventive solutions that would be great. solutions that would be great. solutions that would be great. >> Thanks. >> Thanks. >> Thanks. So things that I have been prioritizing So things that I have been prioritizing So things that I have been prioritizing are uh building like foundational are uh building like foundational are uh building like foundational guardrails with layers, right? So kind guardrails with layers, right? So kind guardrails with layers, right? So kind of what I expressed with the agent and of what I expressed with the agent and of what I expressed with the agent and the egress filtering. Um I want to have the egress filtering. Um I want to have the egress filtering. Um I want to have some control and governance over how the some control and governance over how the some control and governance over how the entire enterprise deployment is made.

entire enterprise deployment is made. entire enterprise deployment is made. And then I want to have additional And then I want to have additional And then I want to have additional controls underneath things that I might controls underneath things that I might controls underneath things that I might not have had in the past. Things like um not have had in the past. Things like um not have had in the past. Things like um I am backing up people's laptops now. I I am backing up people's laptops now. I I am backing up people's laptops now. I never thought I would back up people's never thought I would back up people's never thought I would back up people's laptops after like 2020. laptops after like 2020. laptops after like 2020. Uh but people can delete their data Uh but people can delete their data Uh but people can delete their data that's on their laptop now with a simple that's on their laptop now with a simple that's on their laptop now with a simple agentic query. Um so how I think about agentic query. Um so how I think about agentic query. Um so how I think about runtime uh I've used a number of runtime runtime uh I've used a number of runtime runtime uh I've used a number of runtime tools. I think a lot of folks that have tools. I think a lot of folks that have tools. I think a lot of folks that have been building them are coming at them been building them are coming at them been building them are coming at them from uh the sort of same places we came from uh the sort of same places we came from uh the sort of same places we came at a lot of original security uh tooling at a lot of original security uh tooling at a lot of original security uh tooling with. and that's data leak and with. and that's data leak and with. and that's data leak and prevention and and it's not equipped for prevention and and it's not equipped for prevention and and it's not equipped for non-deterministic workloads. I think non-deterministic workloads. I think non-deterministic workloads. I think there's something completely different there's something completely different there's something completely different about these and you can't just use about these and you can't just use about these and you can't just use strings and you can't just try to reason strings and you can't just try to reason strings and you can't just try to reason in a small box about what the agent's in a small box about what the agent's in a small box about what the agent's doing. So, uh, one of the things that I doing. So, uh, one of the things that I doing. So, uh, one of the things that I really like to experiment with is how do really like to experiment with is how do really like to experiment with is how do I hook the agent at runtime with a set I hook the agent at runtime with a set I hook the agent at runtime with a set of policies, not trying to detect, you of policies, not trying to detect, you of policies, not trying to detect, you know, on the output, but on the input, know, on the output, but on the input, know, on the output, but on the input, giving it the right guard rails. And I I giving it the right guard rails. And I I giving it the right guard rails. And I I I like that. I like that approach a lot.

Uh, sort of build building on that Uh, sort of build building on that first. Thank you. This is very very first. Thank you. This is very very first. Thank you. This is very very cool. Um wondering so the ideas here cool. Um wondering so the ideas here cool. Um wondering so the ideas here completely aligned with where where do completely aligned with where where do completely aligned with where where do you see this existing? Is this at the you see this existing? Is this at the you see this existing? Is this at the tool call level? Is this every single tool call level? Is this every single tool call level? Is this every single turn it runs through this sort of turn it runs through this sort of turn it runs through this sort of process like how how how might you process like how how how might you process like how how how might you actually instrument this in practice? actually instrument this in practice? actually instrument this in practice? >> I I I think this has to be instrumented >> I I I think this has to be instrumented >> I I I think this has to be instrumented in the harness. in the harness. in the harness. I am not a deep enough engineer to know I am not a deep enough engineer to know I am not a deep enough engineer to know how that would work. This is this is my how that would work. This is this is my how that would work. This is this is my plea to you all who are way more plea to you all who are way more plea to you all who are way more intelligent about this than I am. But intelligent about this than I am. But intelligent about this than I am. But what I what I've seen kind of same what I what I've seen kind of same what I what I've seen kind of same answer I gave before like what I've seen answer I gave before like what I've seen answer I gave before like what I've seen in the things that we've built is when in the things that we've built is when in the things that we've built is when we can intercept an agent that's about we can intercept an agent that's about we can intercept an agent that's about to write a line of code and say, "Hey, to write a line of code and say, "Hey, to write a line of code and say, "Hey, by the way, here's our standard for by the way, here's our standard for by the way, here's our standard for authentication. Make sure you use that authentication. Make sure you use that authentication. Make sure you use that library right at that time before it library right at that time before it library right at that time before it writes the line. It works." So I think writes the line. It works." So I think writes the line. It works." So I think the question is like what do you do as a the question is like what do you do as a the question is like what do you do as a post tool hook and is that the right post tool hook and is that the right post tool hook and is that the right place to do that pro probably but again place to do that pro probably but again place to do that pro probably but again I'm out of my depth at that point I'm out of my depth at that point I'm out of my depth at that point >> is that >> is that >> is that okay okay okay >> all right