How to Know When an AI Agent Can Work Unsupervised — Evan Kormos, Coinbase

Hi everyone. Hi everyone. My name is Evan Cormos. I've been My name is Evan Cormos. I've been My name is Evan Cormos. I've been delivering, designing, building software delivering, designing, building software delivering, designing, building software for 20 years. The last 15 in enterprise for 20 years. The last 15 in enterprise for 20 years. The last 15 in enterprise financial services. financial services. financial services. I am a player coach at Coinbase. I am a player coach at Coinbase. I am a player coach at Coinbase. And we're here today to talk about And we're here today to talk about And we're here today to talk about trusting agents in policy-driven trusting agents in policy-driven trusting agents in policy-driven workflows. So, let's start with the policy and it's So, let's start with the policy and it's pretty boring. pretty boring. pretty boring. Um, Um, Um, I'm going to go over here for my notes. I'm going to go over here for my notes. I'm going to go over here for my notes. So, our starting point center centers So, our starting point center centers So, our starting point center centers around the policy and around the policy and around the policy and this sets the rules for how work gets this sets the rules for how work gets this sets the rules for how work gets done within the organization and done within the organization and done within the organization and including especially risk and including especially risk and including especially risk and compliance. Think of policy as the original Think of policy as the original institutional intent institutional intent institutional intent or the original system prompt. or the original system prompt. or the original system prompt. So, ideally we have this written down. So, ideally we have this written down. So, ideally we have this written down. Um, it's a little bit boring, so stick Um, it's a little bit boring, so stick Um, it's a little bit boring, so stick with me. with me. with me. Um, using language like must, may, and Um, using language like must, may, and Um, using language like must, may, and should.

should. should. And the again, these help define the And the again, these help define the And the again, these help define the rules and decisions about how the system rules and decisions about how the system rules and decisions about how the system operates operates operates in the organization. in the organization. in the organization. Separate from policy, we have the actual Separate from policy, we have the actual Separate from policy, we have the actual running system. So, the first thing we're going to do is So, the first thing we're going to do is separate these two separate these two separate these two uh uh uh these two both the build state and the these two both the build state and the these two both the build state and the run state in two different moments in run state in two different moments in run state in two different moments in time. time. time. So, typically we're either defining or So, typically we're either defining or So, typically we're either defining or chaining changing our policy chaining changing our policy chaining changing our policy but this the system is constantly but this the system is constantly but this the system is constantly running and it's constantly moving and running and it's constantly moving and running and it's constantly moving and changing. So, we could then sort of imagine a So, we could then sort of imagine a compilation event that Oops, sorry. I'm compilation event that Oops, sorry. I'm compilation event that Oops, sorry. I'm skipping ahead there. skipping ahead there. skipping ahead there. It's constantly changing. It's constantly changing. It's constantly changing. So, it's evolving quickly with these it's evolving quickly with these product-driven changes. So, everything product-driven changes. So, everything product-driven changes. So, everything is expected to still be roughly bound by is expected to still be roughly bound by is expected to still be roughly bound by the policy, but ultimately um the policy, but ultimately um the policy, but ultimately um we separate this from the run state and we separate this from the run state and we separate this from the run state and the build state.

So, inside the system, So, inside the system, what does it look like? We have a what does it look like? We have a what does it look like? We have a familiar model if we have any enterprise familiar model if we have any enterprise familiar model if we have any enterprise uh folks in the in the room here. uh folks in the in the room here. uh folks in the in the room here. Um people, process, and technology. Um people, process, and technology. Um people, process, and technology. But, where really do agents fit into But, where really do agents fit into But, where really do agents fit into this model? Um so, I see them as an operational Um so, I see them as an operational model cutting across all three. model cutting across all three. model cutting across all three. They work with people, execute parts of They work with people, execute parts of They work with people, execute parts of the process the process the process themselves, and they use and are made up themselves, and they use and are made up themselves, and they use and are made up of technology. So, for our talk today, we're going to So, for our talk today, we're going to be focusing here where agents meet be focusing here where agents meet be focusing here where agents meet process. process. process. What work we give them, what boundaries What work we give them, what boundaries What work we give them, what boundaries we set, we set, we set, and how we build trust in their outputs. So, let's dive into your workflow. So, let's dive into your workflow. And it's high-level and seems basic, but And it's high-level and seems basic, but And it's high-level and seems basic, but if you haven't, there are good reasons if you haven't, there are good reasons if you haven't, there are good reasons to map this out.

to map this out. to map this out. So, maybe a change has to be approved So, maybe a change has to be approved So, maybe a change has to be approved before it goes live, or a customer goes before it goes live, or a customer goes before it goes live, or a customer goes through has to go through verification through has to go through verification through has to go through verification before their account is opened. before their account is opened. before their account is opened. We expect work to flow in a specific and We expect work to flow in a specific and We expect work to flow in a specific and gated direction. So, I believe it makes it agents make it So, I believe it makes it agents make it worth revisiting what we can do in worth revisiting what we can do in worth revisiting what we can do in parallel. So, previously some work used to cost So, previously some work used to cost too much or be too complex to coordinate too much or be too complex to coordinate too much or be too complex to coordinate in this way. in this way. in this way. But now we can run independent reviews But now we can run independent reviews But now we can run independent reviews alongside each other, bring the results alongside each other, bring the results alongside each other, bring the results together, but keep the required checks together, but keep the required checks together, but keep the required checks in place. So this is a directed acyclic graph or a So this is a directed acyclic graph or a DAG. DAG. DAG. It is a way to represent this workflow It is a way to represent this workflow It is a way to represent this workflow or process, the steps, the dependencies or process, the steps, the dependencies or process, the steps, the dependencies for a flow that moves forward without for a flow that moves forward without for a flow that moves forward without looping back on itself. Apache Airflow, if you're familiar, is Apache Airflow, if you're familiar, is an example of open source software that an example of open source software that an example of open source software that represents workflows, represents workflows, represents workflows, their dependencies as DAGs in Python their dependencies as DAGs in Python their dependencies as DAGs in Python code.

So So this Python code, this these DAGs can be this Python code, this these DAGs can be this Python code, this these DAGs can be used to define and manage a number of used to define and manage a number of used to define and manage a number of policy driven processes like, for policy driven processes like, for policy driven processes like, for example, commercial lending. We could also use a DAG to represent the We could also use a DAG to represent the workflow to list a token on the exchange workflow to list a token on the exchange workflow to list a token on the exchange in a specific region, if you're into in a specific region, if you're into in a specific region, if you're into that. Or we could use a DAG to represent a Or we could use a DAG to represent a regulated flow of delivering software, regulated flow of delivering software, regulated flow of delivering software, which is a hot topic. which is a hot topic. which is a hot topic. And one we'll spend a little bit of time And one we'll spend a little bit of time And one we'll spend a little bit of time on, I think that on, I think that on, I think that we've seen a lot of adoption within we've seen a lot of adoption within we've seen a lot of adoption within agents, within software delivery, and agents, within software delivery, and agents, within software delivery, and ultimately we're looking to leverage ultimately we're looking to leverage ultimately we're looking to leverage that same that same that same efficiency and scale in other areas of efficiency and scale in other areas of efficiency and scale in other areas of the business and close that the the the business and close that the the the business and close that the the adoption gap. So let's stick with the So let's stick with the software delivery example for a little software delivery example for a little software delivery example for a little bit. What happens inside one of these nodes What happens inside one of these nodes of the process? And of the process? And of the process? And I'll give you a hint, it's a loop.

>> I'm oversimplified, but we have a basic >> I'm oversimplified, but we have a basic version of what's happening within the version of what's happening within the version of what's happening within the Let's Let's say this is a code review Let's Let's say this is a code review Let's Let's say this is a code review process, right? process, right? process, right? So, So, So, we're going to essentially we're going to essentially we're going to essentially do a number of steps and essentially do a number of steps and essentially do a number of steps and essentially once we realize the work we recognize once we realize the work we recognize once we realize the work we recognize that the work is good enough, we're that the work is good enough, we're that the work is good enough, we're going to allow it to pass on to the next going to allow it to pass on to the next going to allow it to pass on to the next step. And that's essentially the step. And that's essentially the step. And that's essentially the function of the code review node. The work doesn't always uh end up okay. The work doesn't always uh end up okay. So, sometimes we have to take the So, sometimes we have to take the So, sometimes we have to take the feedback from the code review and feedback from the code review and feedback from the code review and iterate on it um until the work is a iterate on it um until the work is a iterate on it um until the work is a good enough to pass. So, this is a representation of a So, this is a representation of a directed cyclic graph or DCG. So, as far as open source software, uh So, as far as open source software, uh something like LangChain LangGraph would something like LangChain LangGraph would something like LangChain LangGraph would be what I would be what I would be what I would um um um say is helpful to manage the state of say is helpful to manage the state of say is helpful to manage the state of DCGs.

Um and this is an oversimpli- an Um and this is an oversimpli- an oversimplification. These graphs can be oversimplification. These graphs can be oversimplification. These graphs can be very complex. So, zooming back out, So, zooming back out, the inner flow um of the node, we have the inner flow um of the node, we have the inner flow um of the node, we have people and agents working together. people and agents working together. people and agents working together. On the outer flow, we have a policy On the outer flow, we have a policy On the outer flow, we have a policy boundary. But not everything goes to plan. But not everything goes to plan. Um so, we have to handle exception Um so, we have to handle exception Um so, we have to handle exception flows. We don't want the work to get flows. We don't want the work to get flows. We don't want the work to get stuff stuck, and we don't want to design stuff stuck, and we don't want to design stuff stuck, and we don't want to design for every edge case, either. for every edge case, either. for every edge case, either. Um the exception flows will monitor and Um the exception flows will monitor and Um the exception flows will monitor and see kind of what happens if you're see kind of what happens if you're see kind of what happens if you're shipping code that gets canceled or shipping code that gets canceled or shipping code that gets canceled or doesn't meet security standards, that's doesn't meet security standards, that's doesn't meet security standards, that's something you would should want to know something you would should want to know something you would should want to know about. Volume shifts across the flow once we Volume shifts across the flow once we start to add agents into the mix and start to add agents into the mix and start to add agents into the mix and we've seen this with software delivery we've seen this with software delivery we've seen this with software delivery and we'll see it in other processes as and we'll see it in other processes as and we'll see it in other processes as we bring agents into those processes as we bring agents into those processes as we bring agents into those processes as well where now all of a sudden if you're well where now all of a sudden if you're well where now all of a sudden if you're reviewing code faster reviewing code faster reviewing code faster you need to design it quicker or you you need to design it quicker or you you need to design it quicker or you need to do the security scan that's need to do the security scan that's need to do the security scan that's faster. So we see the volume shift faster. So we see the volume shift faster. So we see the volume shift across the flow.

And this is where it's really important And this is where it's really important that we have visibility into the process that we have visibility into the process that we have visibility into the process overall from a a metric standpoint overall from a a metric standpoint overall from a a metric standpoint because as we start to supercharge and because as we start to supercharge and because as we start to supercharge and and and and parallelize and add more efficiency to parallelize and add more efficiency to parallelize and add more efficiency to the process we start to develop more and the process we start to develop more and the process we start to develop more and more bottlenecks and so the operational more bottlenecks and so the operational more bottlenecks and so the operational visibility of the process is extremely visibility of the process is extremely visibility of the process is extremely important much more important than the important much more important than the important much more important than the authoring and the building of your authoring and the building of your authoring and the building of your workflow tool. workflow tool. workflow tool. Um so we want to visualize the process. So we want to add parallel capacity by So we want to add parallel capacity by still keeping the controls and so this still keeping the controls and so this still keeping the controls and so this is where we'll add more additional is where we'll add more additional is where we'll add more additional capacity. Okay. So we're doing some work we're Okay. So we're doing some work we're checking it we're using the feedback checking it we're using the feedback checking it we're using the feedback again to refine it until the work is again to refine it until the work is again to refine it until the work is good enough to move on. good enough to move on. good enough to move on. But why is that hard? Oh it seems like a But why is that hard? Oh it seems like a But why is that hard? Oh it seems like a pretty easy thing to do. pretty easy thing to do. pretty easy thing to do. But as I mentioned before these graphs But as I mentioned before these graphs But as I mentioned before these graphs get actually pretty complex get actually pretty complex get actually pretty complex and we'll dive a little bit more into and we'll dive a little bit more into and we'll dive a little bit more into why this is difficult.

And if you're familiar one challenge And if you're familiar one challenge with AI engineering is the variability with AI engineering is the variability with AI engineering is the variability of inputs and outputs and of inputs and outputs and of inputs and outputs and I believe that higher quality inputs will result in better outputs at a lower will result in better outputs at a lower compute cost. compute cost. compute cost. But what are quality inputs? >> So, >> So, we would generally speak to it as we would generally speak to it as we would generally speak to it as context context context model maybe. We can nitpick that, but model maybe. We can nitpick that, but model maybe. We can nitpick that, but this isn't our area of focus today. this isn't our area of focus today. this isn't our area of focus today. We're going to focus more on outputs. We're going to focus more on outputs. We're going to focus more on outputs. But, we can't really talk about quality But, we can't really talk about quality But, we can't really talk about quality of outputs without the quality of of outputs without the quality of of outputs without the quality of without talking about the quality of without talking about the quality of without talking about the quality of inputs. So, ideally we've established a trusted So, ideally we've established a trusted context for our agent. context for our agent. context for our agent. Um and there's some references on that, Um and there's some references on that, Um and there's some references on that, but one thing we can do in this model is but one thing we can do in this model is but one thing we can do in this model is inject the DAG state into our agent and inject the DAG state into our agent and inject the DAG state into our agent and let it know kind of where it fits into let it know kind of where it fits into let it know kind of where it fits into the bigger picture.

Okay, so Okay, so now we've added even long time horizon now we've added even long time horizon now we've added even long time horizon goals and that's emphasizes the need to goals and that's emphasizes the need to goals and that's emphasizes the need to optimize the cost and efficiency of this optimize the cost and efficiency of this optimize the cost and efficiency of this loop. So, the check is where we decide whether So, the check is where we decide whether the work is good enough to move forward. the work is good enough to move forward. the work is good enough to move forward. We need to define what good looks like. We need to define what good looks like. We need to define what good looks like. That's the hard part. That's the hard part. That's the hard part. And we need to have a reliable way to And we need to have a reliable way to And we need to have a reliable way to evaluate it. Otherwise, more passes evaluate it. Otherwise, more passes evaluate it. Otherwise, more passes don't necessarily don't necessarily don't necessarily uh mean better results. The loop is uh mean better results. The loop is uh mean better results. The loop is really only as trustworthy as the check. really only as trustworthy as the check. really only as trustworthy as the check. So, if we want to talk about quality, So, if we want to talk about quality, So, if we want to talk about quality, let's talk about the check and let's let's talk about the check and let's let's talk about the check and let's talk about quality. Okay, um I think building Okay, um I think building with AI is an opportunity to improve with AI is an opportunity to improve with AI is an opportunity to improve product quality overall. I like some of product quality overall. I like some of product quality overall. I like some of the conversation about durable software. the conversation about durable software. the conversation about durable software. Um Um Um So, So, So, I'm a believer in quality I'm a believer in quality I'm a believer in quality and in traditional software and in traditional software and in traditional software um we we built confidence mechanisms in um we we built confidence mechanisms in um we we built confidence mechanisms in place place place uh to give us confidence that the uh to give us confidence that the uh to give us confidence that the software is of high quality.

software is of high quality. software is of high quality. And that's done with a variety of uh And that's done with a variety of uh And that's done with a variety of uh mechanisms, but primarily systematically mechanisms, but primarily systematically mechanisms, but primarily systematically with test cases, unit tests, with test cases, unit tests, with test cases, unit tests, um checking these as a part of our um checking these as a part of our um checking these as a part of our delivery flow to ensure that when we delivery flow to ensure that when we delivery flow to ensure that when we make code changes, mistakes aren't made make code changes, mistakes aren't made make code changes, mistakes aren't made that break other things, right? So, this that break other things, right? So, this that break other things, right? So, this is a confidence mechanism that we've put is a confidence mechanism that we've put is a confidence mechanism that we've put in place with traditional software. The agents are probabilistic, and so the The agents are probabilistic, and so the same request can produce same request can produce same request can produce different answers. different answers. different answers. More than one answer may be acceptable. And we still need evidence, but exact And we still need evidence, but exact output output output uh comparisons alone really aren't good uh comparisons alone really aren't good uh comparisons alone really aren't good enough. enough. enough. Uh we need to build a representative Uh we need to build a representative Uh we need to build a representative data sets and evaluate them data sets and evaluate them data sets and evaluate them um um um and define what good looks like. and define what good looks like. and define what good looks like. We need to run uh We need to run uh We need to run uh ev- these evaluations in repeated uh ev- these evaluations in repeated uh ev- these evaluations in repeated uh series, and this is a statistical series, and this is a statistical series, and this is a statistical analysis, right? Um where we actually analysis, right? Um where we actually analysis, right? Um where we actually look at the outputs look at the outputs look at the outputs uh and in volume and not just as a uh and in volume and not just as a uh and in volume and not just as a direct comparison.

So, in combining with evals is sort of So, in combining with evals is sort of your run time logs of what happened, or your run time logs of what happened, or your run time logs of what happened, or sorry, traces, uh wh- which are sort of sorry, traces, uh wh- which are sort of sorry, traces, uh wh- which are sort of your run time logs of what happens. your run time logs of what happens. your run time logs of what happens. Um we've built this confidence mechanism Um we've built this confidence mechanism Um we've built this confidence mechanism to complement um traditional software. And ultimately, this is what we're going And ultimately, this is what we're going to be talking about a little bit more in to be talking about a little bit more in to be talking about a little bit more in in terms of driving confidence in the in terms of driving confidence in the in terms of driving confidence in the output, not only putting this mechanism output, not only putting this mechanism output, not only putting this mechanism in place, but how do we begin to turn it in place, but how do we begin to turn it in place, but how do we begin to turn it on? So, So, where does the agent belong? Um back to where does the agent belong? Um back to where does the agent belong? Um back to the lending example, like we can do KYC, the lending example, like we can do KYC, the lending example, like we can do KYC, financials in parallel and sequence like financials in parallel and sequence like financials in parallel and sequence like credit memo, risk review, documentation, credit memo, risk review, documentation, credit memo, risk review, documentation, decision, all these things in our our decision, all these things in our our decision, all these things in our our DAG. Um looking at each node, we want to DAG. Um looking at each node, we want to DAG. Um looking at each node, we want to ask some certain questions like how ask some certain questions like how ask some certain questions like how valuable would it be to have an agent valuable would it be to have an agent valuable would it be to have an agent drive this node versus a human drive drive this node versus a human drive drive this node versus a human drive this node? Um and we can sort of look at this node? Um and we can sort of look at this node? Um and we can sort of look at some of these basic questions like can some of these basic questions like can some of these basic questions like can we check it against the policy easily?

we check it against the policy easily? we check it against the policy easily? Can it run in a bounded loop? Can we Can it run in a bounded loop? Can we Can it run in a bounded loop? Can we trace it? Can we fall back to a human? trace it? Can we fall back to a human? trace it? Can we fall back to a human? Um if we can answer yes to these type of Um if we can answer yes to these type of Um if we can answer yes to these type of questions, questions, questions, and we we can kind of have a good idea and we we can kind of have a good idea and we we can kind of have a good idea going in that this particular node may going in that this particular node may going in that this particular node may be better suited for automation. Um but be better suited for automation. Um but be better suited for automation. Um but throughout the DAG, we should look to throughout the DAG, we should look to throughout the DAG, we should look to drive agent assistance. So, what we're drive agent assistance. So, what we're drive agent assistance. So, what we're really talking about is not whether or really talking about is not whether or really talking about is not whether or not AI is involved, but whether or not not AI is involved, but whether or not not AI is involved, but whether or not AI we expect to take over at some point. On the decision, you would probably On the decision, you would probably agree that we probably need that to be agree that we probably need that to be agree that we probably need that to be human-driven. This is essentially the human-driven. This is essentially the human-driven. This is essentially the check against the policy. Um so, this is check against the policy. Um so, this is check against the policy. Um so, this is going to be a little bit more difficult, going to be a little bit more difficult, going to be a little bit more difficult, I think, to hill climb. But what do we mean by hill climb? But what do we mean by hill climb? All right, how do we turn this on? All right, how do we turn this on? All right, how do we turn this on? So, day one, um we start supervised and day one, um we start supervised and as we have agents and humans working as we have agents and humans working as we have agents and humans working together as a part of the process, we together as a part of the process, we together as a part of the process, we can can can um manually um verify the output to um manually um verify the output to um manually um verify the output to ensure that it is high high quality.

ensure that it is high high quality. ensure that it is high high quality. Um this is where we refine our data sets Um this is where we refine our data sets Um this is where we refine our data sets and ensure that we have the right and ensure that we have the right and ensure that we have the right confidence mechanisms in place. We've confidence mechanisms in place. We've confidence mechanisms in place. We've defined what good looks like with the defined what good looks like with the defined what good looks like with the domain subject matter experts and that domain subject matter experts and that domain subject matter experts and that they're beginning to verify it. When measured confidence reaches a When measured confidence reaches a policy threshold, we can move towards policy threshold, we can move towards policy threshold, we can move towards sampling. And what that means is humans sampling. And what that means is humans sampling. And what that means is humans are looking at a slice of the output and are looking at a slice of the output and are looking at a slice of the output and not necessarily all of it. Um we begin not necessarily all of it. Um we begin not necessarily all of it. Um we begin systematic more systematic evaluation systematic more systematic evaluation systematic more systematic evaluation through traces through traces through traces Um in order to ensure that what we're Um in order to ensure that what we're Um in order to ensure that what we're what the humans aren't seeing is still what the humans aren't seeing is still what the humans aren't seeing is still meeting the quality bar. And then finally, once we clear the And then finally, once we clear the confidence threshold, humans handle more confidence threshold, humans handle more confidence threshold, humans handle more of the exceptions of the process, but of the exceptions of the process, but of the exceptions of the process, but still acting as participants in refining still acting as participants in refining still acting as participants in refining the data sets and ensuring the quality the data sets and ensuring the quality the data sets and ensuring the quality outputs. Okay, so a few takeaways from this.

Okay, so a few takeaways from this. Um Um Um DAG and DCG, DAG and DCG, DAG and DCG, basic graph theory, but ultimately, how basic graph theory, but ultimately, how basic graph theory, but ultimately, how do we build these processes that exist do we build these processes that exist do we build these processes that exist today and where people are working them today and where people are working them today and where people are working them and bring agents into the mix. And the and bring agents into the mix. And the and bring agents into the mix. And the recipe really is to keep it recipe really is to keep it recipe really is to keep it deterministic on the outside and make it deterministic on the outside and make it deterministic on the outside and make it adaptive on the inside. The next is quality through evaluation. The next is quality through evaluation. And this is really establishing a And this is really establishing a And this is really establishing a complementary confidence mechanism. complementary confidence mechanism. complementary confidence mechanism. So, it's not a feeling about a model. We So, it's not a feeling about a model. We So, it's not a feeling about a model. We can actually provide research and can actually provide research and can actually provide research and statistics about the quality of our statistics about the quality of our statistics about the quality of our agents. So, this builds confidence and So, this builds confidence and inherently trusts output. Quality is a team sport. Quality is a team sport. AI engineers own the pipeline AI engineers own the pipeline AI engineers own the pipeline to taking product signals to taking product signals to taking product signals from all directions from all directions from all directions and deliver applications that are well and deliver applications that are well and deliver applications that are well built and utilize ML operations behind built and utilize ML operations behind built and utilize ML operations behind the scenes.

Finally, agents accelerate Finally, agents accelerate across the operating model. I've included some references on the I've included some references on the final slide of things that I've spoken final slide of things that I've spoken final slide of things that I've spoken about today. about today. about today. This concludes my presentation. This concludes my presentation. This concludes my presentation. Thank you very much. Thank you very much. Thank you very much. >> [applause]