From coding to Knowledge work agents — Karan Vaidya, Composio

Hello everyone. I'm Karan Hello everyone. I'm Karan Vedya, co-founder and Vedya, co-founder and Vedya, co-founder and CTO of CTO of CTO of Composio. Most Composio. Most Composio. Most agent tool calls agent tool calls today still today still occur in only occur in only occur in only one industry. Without one industry. Without one industry. Without options, this is options, this is options, this is software development software development software development . All . All . All other types of work lag other types of work lag other types of work lag far behind. far behind. If models are getting If models are getting better, why are we still better, why are we still better, why are we still limited to limited to limited to agent-based coding? That's the agent-based coding? That's the agent-based coding? That's the trillion trillion trillion dollar question I'm going to dollar question I'm going to dollar question I'm going to answer here. Three years answer here. Three years answer here. Three years ago, ago, ago, coding agents were coding agents were coding agents were just just just autocomplete. autocomplete. autocomplete. Today, software development Today, software development Today, software development has become completely has become completely has become completely autonomous. We've autonomous. We've autonomous. We've gone from gone from gone from pressing "tab" pressing "tab" pressing "tab" to just letting to just letting to just letting Claude do the work. It's Claude do the work. It's Claude do the work. It's just magic. Why did just magic. Why did just magic. Why did this this this happen so quickly in programming? happen so quickly in programming? Most people will think it's about the Most people will think it's about the models. Yes, models. Yes, models. Yes, the models have really gotten the models have really gotten the models have really gotten much better over the much better over the much better over the last two or three years. last two or three years. last two or three years. As well as tools such as As well as tools such as As well as tools such as Claude Code or Claude Code or Claude Code or Codex Cursors. But Codex Cursors. But Codex Cursors. But they wouldn't have been able to do it on their own they wouldn't have been able to do it on their own they wouldn't have been able to do it on their own . It . It . It only worked because the only worked because the only worked because the entire entire entire infrastructure and infrastructure and infrastructure and systems around the code systems around the code systems around the code were literally were literally were literally built for agents.

built for agents. The code had the support The code had the support the agents needed. the agents needed. the agents needed. You have a repository. You have a repository. You have a repository. Commit history, Commit history, Commit history, tests, CI/CD, reviews, tests, CI/CD, reviews, tests, CI/CD, reviews, linters, rollback if linters, rollback if linters, rollback if something goes wrong. This is what something goes wrong. This is what something goes wrong. This is what makes you makes you makes you trust agents—the trust agents—the systems around the code. systems around the code. Now we direct Now we direct these same these same these same incredible agents incredible agents incredible agents to everything else. to everything else. to everything else. Support, finance, Support, finance, Support, finance, sales. But the agents sales. But the agents sales. But the agents who did a great job who did a great job who did a great job at coding are acting at coding are acting at coding are acting blindly here. Because blindly here. Because blindly here. Because the infrastructure the infrastructure the infrastructure around coding around coding around coding doesn't even exist in doesn't even exist in doesn't even exist in other industries. So how do we other industries. So how do we other industries. So how do we bridge the gap between bridge the gap between programming agents and programming agents and agents for agents for agents for intelligent intelligent intelligent work? We believe work? We believe work? We believe that it's about six that it's about six that it's about six basic primitives: basic primitives: basic primitives: coding had them coding had them coding had them all, but all, but all, but intellectual intellectual intellectual work had none, and that's what work had none, and that's what work had none, and that's what we need to we need to we need to build. The first is build. The first is build. The first is centralization. centralization.

The agents in the coding The agents in the coding worked so well worked so well worked so well in part because they in part because they in part because they were very close to the were very close to the were very close to the source of truth. They source of truth. They source of truth. They knew what, why and how. knew what, why and how. You give them a You give them a repository, repository, repository, infrastructure as infrastructure as infrastructure as code, close the loop— code, close the loop— and let the model and let the model and let the model run. The agent starts run. The agent starts run. The agent starts working with everything it working with everything it working with everything it needs in one needs in one needs in one place—in the codebase place—in the codebase . This is exactly what is . This is exactly what is . This is exactly what is missing missing missing in intellectual in intellectual in intellectual work today. For example, work today. For example, work today. For example, one deal is one deal is one deal is scattered across scattered across scattered across five different five different five different platforms. Records in platforms. Records in platforms. Records in Salesforce, documents in Salesforce, documents in Salesforce, documents in Notion, emails in Gmail, Notion, emails in Gmail, Notion, emails in Gmail, conversations in Slack, and conversations in Slack, and conversations in Slack, and support history in support history in support history in Zendesk. There is no single Zendesk. There is no single Zendesk. There is no single source of truth, no source of truth, no source of truth, no single place where single place where single place where all information can be gathered all information can be gathered all information can be gathered . Everything is . Everything is . Everything is separate, and each separate, and each separate, and each application has its application has its application has its own login. own login. own login. Before an agent for Before an agent for Before an agent for intelligent intelligent intelligent work can work can work can start anything, it has to start anything, it has to start anything, it has to gather all the threads and gather all the threads and tie them together on its own. And this is still the tie them together on its own. And this is still the starting point from which the starting point from which the starting point from which the agent began in agent began in agent began in coding. He had it coding. He had it coding. He had it from the beginning.

from the beginning. from the beginning. So how can you So how can you So how can you expect an agent expect an agent expect an agent for intelligent for intelligent for intelligent work to work to work to perform at the perform at the perform at the same level as an same level as an same level as an agent for agent for agent for programming? So the programming? So the programming? So the first thing we first thing we first thing we create is a create is a create is a missing center. missing center. missing center. The only place where The only place where The only place where all your all your all your applications, all your applications, all your applications, all your connections, all your connections, all your connections, all your logins are collected. So the agent doesn't logins are collected. So the agent doesn't logins are collected. So the agent doesn't have to do the have to do the have to do the hard work of hard work of hard work of putting it all putting it all putting it all together. They together. They together. They find everything in find everything in find everything in one place. And they one place. And they one place. And they get the same get the same get the same base that the base that the base that the agent programmer started with — a agent programmer started with — a repository, repository, repository, information from all the information from all the information from all the stacks in one place stacks in one place . This is the foundation . This is the foundation . This is the foundation you start with, and you start with, and you start with, and you can grant you can grant you can grant your agent the your agent the your agent the appropriate appropriate appropriate access rights. The next thing access rights. The next thing access rights. The next thing an agent needs is a an agent needs is a an agent needs is a sense of history. sense of history. sense of history. The ability to look back The ability to look back The ability to look back into the past. In the code you into the past. In the code you into the past. In the code you get it for get it for get it for free. It free. It free. It keeps a record of keeps a record of keeps a record of every thing entered, every thing entered, every thing entered, every every every change made. So the agent change made. So the agent change made. So the agent can always can always can always look back and look back and look back and see how a see how a see how a certain change was made, certain change was made, certain change was made, why something worked why something worked why something worked and why it didn't. Think and why it didn't. Think and why it didn't. Think about what you are about what you are about what you are actually asking actually asking actually asking your agent to do.

your agent to do. your agent to do. We have had to We have had to We have had to undo a change in undo a change in undo a change in the past due to the past due to the past due to some glitch, but it some glitch, but it some glitch, but it was quite difficult was quite difficult was quite difficult to do. Can you to do. Can you to do. Can you look at this and look at this and look at this and turn it back? He turn it back? He turn it back? He simply reads the story, simply reads the story, simply reads the story, rewinds it, and rewinds it, and rewinds it, and processes it. A story is processes it. A story is processes it. A story is not only needed for an not only needed for an not only needed for an agent. You also agent. You also agent. You also need it to need it to need it to keep track of what the keep track of what the keep track of what the agent does. You agent does. You agent does. You can see what the can see what the can see what the agent is doing, where it's agent is doing, where it's agent is doing, where it's making making making mistakes and where it's doing mistakes and where it's doing mistakes and where it's doing well, and instead of well, and instead of well, and instead of taking the taking the taking the agent's word for it, you agent's word for it, you agent's word for it, you can just go into can just go into can just go into those specific applications those specific applications those specific applications and see what it and see what it and see what it did. Now did. Now did. Now ask the same ask the same ask the same questions about questions about questions about intellectual intellectual intellectual work. What led work. What led work. What led to CRM being in to CRM being in to CRM being in this state now? How this state now? How this state now? How did my colleague create that did my colleague create that did my colleague create that amazing letter that amazing letter that amazing letter that led to the led to the led to the deal? What is the actual deal? What is the actual deal? What is the actual process for escalating a process for escalating a process for escalating a support issue support issue support issue or even closing it or even closing it ? The answers are scattered ? The answers are scattered ? The answers are scattered across hundreds of apps, and across hundreds of apps, and across hundreds of apps, and none of them none of them none of them keep a history. So keep a history. So keep a history. So the agent has no memory. the agent has no memory. the agent has no memory. Every time he starts Every time he starts Every time he starts almost from scratch almost from scratch almost from scratch . No . No . No idea what has been idea what has been idea what has been tried before, what tried before, what tried before, what worked and what didn't. And worked and what didn't. And worked and what didn't. And you also have nothing to you also have nothing to you also have nothing to look at. When an look at. When an look at. When an agent completes a agent completes a agent completes a job and says everything job and says everything job and says everything went well, you do went well, you do went well, you do n't know if n't know if n't know if it really did. There is no way it really did. There is no way it really did. There is no way to know if this is to know if this is to know if this is correct or not. And that's exactly what's correct or not. And that's exactly what's missing—a record of the missing—a record of the work done.

work done. Now, since everything Now, since everything ultimately goes ultimately goes ultimately goes through one place—that's through one place—that's through one place—that's centralization—we centralization—we centralization—we can build a can build a recording layer on top of that. Any action recording layer on top of that. Any action performed by the agent performed by the agent performed by the agent can be logged in can be logged in can be logged in all other applications. all other applications. all other applications. Everything he Everything he Everything he touched, everything he touched, everything he touched, everything he missed, what missed, what missed, what worked and what didn't. worked and what didn't. worked and what didn't. Thanks to this, Thanks to this, firstly, the agent gains firstly, the agent gains firstly, the agent gains memory. He can memory. He can memory. He can review how review how review how similar tasks similar tasks similar tasks have been done before, have been done before, have been done before, what was successful, and what was successful, and what was successful, and recreate it again. recreate it again. He doesn't start from scratch every time. Secondly, you Secondly, you gain trust. You gain trust. You gain trust. You can finally can finally can finally see exactly what the see exactly what the see exactly what the agent is doing. So agent is doing. So agent is doing. So instead of instead of instead of hoping he hoping he hoping he does everything does everything does everything right, you can right, you can right, you can just check on just check on just check on him and stop him in time him and stop him in time him and stop him in time if he if he if he does something wrong. And does something wrong. And does something wrong. And as you as you as you see him see him see him do things do things do things right more often, you right more often, you right more often, you will develop trust and will develop trust and will develop trust and assign him assign him assign him more tasks. more tasks. The next thing an The next thing an agent needs is context. agent needs is context. agent needs is context. If you think about it, If you think about it, If you think about it, there are actually two there are actually two there are actually two types of context.

types of context. types of context. The first is the form of The first is the form of The first is the form of the platform, its the platform, its the platform, its architecture. How things architecture. How things architecture. How things interact with interact with interact with each other, how they are each other, how they are each other, how they are connected, connected, connected, data flows. It's like a map data flows. It's like a map data flows. It's like a map that an experienced that an experienced that an experienced engineer keeps in their engineer keeps in their engineer keeps in their head, and it head, and it head, and it takes a junior engineer takes a junior engineer takes a junior engineer about 3 months to about 3 months to about 3 months to master it. The second master it. The second is style. This is not what is style. This is not what is style. This is not what is objectively is objectively is objectively right, but rather what right, but rather what quality work looks like in your quality work looks like in your company. That is, how you company. That is, how you company. That is, how you do things. Things do things. Things do things. Things like linters, like linters, like linters, type checking, etc. type checking, etc. type checking, etc. Maybe you Maybe you Maybe you 're using a 're using a 're using a TypeScript decorator that no TypeScript decorator that no TypeScript decorator that no one else is one else is one else is using. This is not using. This is not using. This is not exactly spelled out in exactly spelled out in exactly spelled out in the instructions. This is the instructions. This is the instructions. This is more likely to be in more likely to be in more likely to be in your codebase. your codebase. your codebase. It's all in your It's all in your It's all in your codebase, so an codebase, so an codebase, so an agent can just agent can just agent can just go and look and go and look and go and look and understand understand understand the specs, what the specs, what the specs, what you like, you like, you like, linters, linters, linters, formatters, etc. formatters, etc. Now let's move on to Now let's move on to Now let's move on to intellectual intellectual intellectual work, everything is the same here work, everything is the same here work, everything is the same here . Let's say you're . Let's say you're . Let's say you're writing a document for a writing a document for a writing a document for a client. To even client. To even client. To even get started, I need to get started, I need to get started, I need to open the database open the database open the database to get their to get their usage data, usage data, check Posthog to see how check Posthog to see how check Posthog to see how they actually they actually they actually used used used the product, and Salesforce the product, and Salesforce the product, and Salesforce to look at the to look at the to look at the deal details. Only deal details. Only deal details. Only then can I start then can I start then can I start writing the first line of writing the first line of writing the first line of the document. The answer the document. The answer the document. The answer was not isolated was not isolated was not isolated to just one of these to just one of these to just one of these tools. I can tools. I can tools. I can write this write this write this document because I am document because I am document because I am gathering threads from all gathering threads from all gathering threads from all these tools into these tools into these tools into one context in my one context in my one context in my head. So, head. So, head. So, by combining history and by combining history and by combining history and context, you context, you context, you map the work of the map the work of the map the work of the entire organization. And entire organization. And entire organization. And this part is this part is this part is not yet available to the not yet available to the not yet available to the agent directly. So, agent directly. So, agent directly. So, since we've

since we've since we've implemented implemented implemented centralization and centralization and centralization and logging, the record logging, the record logging, the record we just we just we just created—the one that gives the created—the one that gives the created—the one that gives the agent memory and agent memory and agent memory and allows us to audit allows us to audit allows us to audit its actions—also its actions—also its actions—also does something else interesting does something else interesting . If you log . If you log . If you log enough of what enough of what enough of what each agent does, each agent does, each agent does, you start to see you start to see you start to see patterns. You patterns. You patterns. You start to see how the start to see how the start to see how the organization works. And organization works. And organization works. And you start you start you start to build skills that to build skills that to build skills that are a kind of are a kind of are a kind of distillation of how the distillation of how the organization worked. Which organization worked. Which approaches work, which do approaches work, which do approaches work, which do n't, what led to n't, what led to n't, what led to failures in the past, failures in the past, failures in the past, etc. This record is etc. This record is etc. This record is no longer just a story of what no longer just a story of what no longer just a story of what happened. This is a happened. This is a happened. This is a picture of how picture of how picture of how your company operates. your company operates. your company operates. And it actually works And it actually works And it actually works on three different on three different on three different levels. How the tool levels. How the tool levels. How the tool works in general, what is works in general, what is works in general, what is applicable to each applicable to each applicable to each person, how the company person, how the company person, how the company does things, and how you does things, and how you does things, and how you prefer to prefer to prefer to do them, what do them, what do them, what means "good" to you. And this is means "good" to you. And this is means "good" to you. And this is exactly the context that the exactly the context that the agent lacked for agent lacked for intellectual intellectual intellectual work. How the work work. How the work work. How the work is is is actually done, actually done, actually done, a kind of real-world a kind of real-world a kind of real-world guide, as well as guide, as well as guide, as well as the preferences of the company the preferences of the company the preferences of the company or individual or individual or individual user. And now the user. And now the user. And now the agent can agent can agent can ask this and ask this and ask this and stop guessing stop guessing stop guessing how the company works.

how the company works. Another reason why Another reason why coding agents coding agents coding agents work so work so work so well is because well is because well is because they test themselves they test themselves , the work tests , the work tests , the work tests itself. itself. itself. Verification. Verification. Verification. Once the agent writes the code Once the agent writes the code , a series of , a series of , a series of checks follow. Unit checks follow. Unit checks follow. Unit tests can detect tests can detect tests can detect small errors. small errors. small errors. Integration tests Integration tests Integration tests detect those that detect those that detect those that affect affect affect components across three components across three components across three levels. The type system levels. The type system levels. The type system won't even work or won't even work or won't even work or start if start if start if something goes wrong. something goes wrong. something goes wrong. The compiler won't even The compiler won't even The compiler won't even start building. On top start building. On top start building. On top are are are softer checks: softer checks: softer checks: linters, linters, linters, formatters, formatters, documentation checks, documentation checks, review skills, review skills, review skills, etc. They ensure etc. They ensure that the code meets that the code meets that the code meets your your your team's standards. None of this team's standards. None of this team's standards. None of this requires your requires your requires your participation. The agent participation. The agent participation. The agent independently closes independently closes independently closes the loop, monitoring the loop, monitoring the loop, monitoring compliance with compliance with compliance with standards and standards and standards and code performance. code performance. Now think about it, at one Now think about it, at one time I time I time I directed my " directed my " open claw" at a open claw" at a open claw" at a recruitment mailing list. recruitment mailing list. recruitment mailing list. Mass mailing of Mass mailing of Mass mailing of emails emails emails to candidates. She to candidates. She to candidates. She started. She started. She started. She sent a bunch of sent a bunch of sent a bunch of letters. Some of you letters. Some of you letters. Some of you may may may have also received a letter from have also received a letter from have also received a letter from my "open my "open my "open claw". He did claw". He did claw". He did exactly what I exactly what I exactly what I told him to do. That told him to do. That told him to do. That was also a disaster. The was also a disaster. The was also a disaster. The ones that end up on ones that end up on ones that end up on Twitter with my name at the Twitter with my name at the Twitter with my name at the top. Yes, I think you top. Yes, I think you top. Yes, I think you can see " can see " you're gone, Karan Vedya" you're gone, Karan Vedya" . I wasn't thrilled . I wasn't thrilled . I wasn't thrilled when it happened. And here's the when it happened. And here's the when it happened. And here's the thing: every thing: every thing: every test from the test from the test from the previous slide previous slide previous slide would have would have would have passed. The letters were passed. The letters were passed. The letters were valid, the addresses were valid, the addresses were real. They real. They real. They actually reached out to

actually reached out to actually reached out to real people who real people who real people who were posting were posting were posting ads. There ads. There ads. There was no test in the world to was no test in the world to was no test in the world to truly truly truly question what was question what was question what was important. Was it worth important. Was it worth doing it at all? That's where the doing it at all? That's where the gap lies. In code, gap lies. In code, gap lies. In code, such tests show such tests show such tests show what is correct and what is not. what is correct and what is not. what is correct and what is not. The internet immediately The internet immediately The internet immediately told me I was told me I was told me I was wrong. So, we wrong. So, we wrong. So, we create checks create checks create checks that are missing. that are missing. that are missing. The problem in the above The problem in the above The problem in the above thread was not thread was not thread was not that the mailing was that the mailing was that the mailing was incorrect. And the incorrect. And the incorrect. And the fact that she left before fact that she left before fact that she left before I even I even I even found out about it. So the solution is found out about it. So the solution is found out about it. So the solution is simple. Identify simple. Identify simple. Identify the problem before the problem before the problem before it becomes real. it becomes real. it becomes real. So, we have two So, we have two So, we have two ways to do this. ways to do this. ways to do this. First: before First: before First: before sending, the agent sending, the agent sending, the agent checks drafts of checks drafts of checks drafts of letters that I letters that I letters that I sent earlier. Does sent earlier. Does sent earlier. Does it fit my it fit my it fit my style, does it match the style, does it match the style, does it match the quality I quality I quality I value? Second: before value? Second: before value? Second: before any any any destructive actions in the destructive actions in the destructive actions in the real world, we real world, we real world, we provide agents with " provide agents with " sandboxes" that sandboxes" that sandboxes" that simulate real-world simulate real-world simulate real-world tools where they tools where they tools where they can perform can perform can perform actions. So instead of the actions. So instead of the actions. So instead of the impact zone impact zone impact zone touching the real touching the real touching the real world, it will be world, it will be world, it will be confined to confined to confined to the sandbox, and I will be able the sandbox, and I will be able the sandbox, and I will be able to test it before the to test it before the to test it before the agent agent agent actually does anything.

actually does anything. actually does anything. Combine these two Combine these two Combine these two approaches and you approaches and you approaches and you get something that has get something that has get something that has never been seen in never been seen in never been seen in intelligent intelligent intelligent work: a way for an work: a way for an work: a way for an agent to validate agent to validate agent to validate their work before their work before their work before it even becomes it even becomes it even becomes real. It real. It real. It can finally close its can finally close its can finally close its cycle on its own, cycle on its own, cycle on its own, instead of instead of instead of stopping and waiting stopping and waiting stopping and waiting for you. This way for you. This way for you. This way you can trust you can trust you can trust his actions without his actions without his actions without getting the barrage of getting the barrage of getting the barrage of tweets I tweets I tweets I post. The next thing post. The next thing post. The next thing an agent needs an agent needs is control. is control. is control. Building trust is about Building trust is about Building trust is about controlling what an controlling what an controlling what an agent can do, agent can do, agent can do, setting the setting the setting the right boundaries right boundaries right boundaries around them. In around them. In around them. In programming, this is programming, this is programming, this is mostly solved and mostly solved and mostly solved and has many levels. has many levels. has many levels. An agent can do whatever it An agent can do whatever it wants in its wants in its wants in its branch, but it cannot branch, but it cannot branch, but it cannot merge it with the main branch merge it with the main branch . A human reviewer . A human reviewer . A human reviewer stands between it and the stands between it and the stands between it and the merge with the main merge with the main merge with the main branch. Critical branch. Critical branch. Critical files have files have files have code owners. So code owners. So code owners. So when an agent when an agent when an agent touches one of touches one of touches one of them, the them, the them, the right people are involved. We right people are involved. We right people are involved. We use use use agents to release agents to release agents to release previous versions, previous versions, previous versions, never letting never letting never letting them touch the them touch the them touch the production environment, production environment, production environment, so we control them so we control them so we control them there. This management is there. This management is not one barrier, but not one barrier, but not one barrier, but many, each of which many, each of which many, each of which has its own size has its own size has its own size depending on the depending on the depending on the risk zone it risk zone it risk zone it creates. This does not creates. This does not creates. This does not slow down the slow down the slow down the agent's work on safe agent's work on safe agent's work on safe paths, but only paths, but only paths, but only prevents prevents prevents damage to the working damage to the working damage to the working environment. And the environment. And the environment. And the tighter these tighter these tighter these restrictions are, the more restrictions are, the more restrictions are, the more you can trust the you can trust the you can trust the agent and allow agent and allow agent and allow them to act freely. You've them to act freely. You've them to act freely. You've probably seen this probably seen this probably seen this case. The Director of case. The Director of case. The Director of Coordination at Coordination at Coordination at Meta Superintelligence Lab Meta Superintelligence Lab Meta Superintelligence Lab connected the agent to connected the agent to connected the agent to her email, and it her email, and it her email, and it began destroying her began destroying her began destroying her mailbox, deleting mailbox, deleting mailbox, deleting a bunch of emails. She a bunch of emails. She a bunch of emails. She ordered him ordered him ordered him to stop. But he

to stop. But he to stop. But he continued. In the end, continued. In the end, continued. In the end, she had to run to the she had to run to the she had to run to the physical physical physical computer to computer to computer to stop it. But by stop it. But by stop it. But by that time, 200 letters had already that time, 200 letters had already that time, 200 letters had already effectively disappeared. effectively disappeared. She She had previously indicated in the request had previously indicated in the request had previously indicated in the request to confirm such to confirm such to confirm such actions, but it was only a actions, but it was only a actions, but it was only a request that request that request that was likely superseded by was likely superseded by was likely superseded by other data. And if other data. And if other data. And if someone whose job someone whose job someone whose job is solely to is solely to is solely to coordinate AI can't coordinate AI can't get an agent right, get an agent right, then probably none of then probably none of then probably none of us can. And that's the us can. And that's the us can. And that's the real reason real reason real reason why these agents are so why these agents are so why these agents are so hard to trust. Not hard to trust. Not hard to trust. Not because they are worse than because they are worse than because they are worse than agents who write agents who write agents who write code, but because there code, but because there code, but because there are no restrictions around them are no restrictions around them are no restrictions around them . In . In . In the code, barriers were the code, barriers were the code, barriers were built into the system built into the system built into the system during development. There during development. There are also certain are also certain elements here and there in intellectual work. elements here and there in intellectual work. elements here and there in intellectual work. For example, Gmail has For example, Gmail has For example, Gmail has access scopes, access scopes, access scopes, Salesforce has Salesforce has Salesforce has permission levels, but it's all permission levels, but it's all permission levels, but it's all so scattered so scattered so scattered that it's hard to have that it's hard to have that it's hard to have real control, and real control, and real control, and usually people usually people usually people try to do try to do try to do it through requests. And it through requests. And it through requests. And the requests are unreliable. the requests are unreliable. The agent will find these The agent will find these loopholes, the data will be loopholes, the data will be loopholes, the data will be pushed out, and at some pushed out, and at some pushed out, and at some point one of these point one of these point one of these protective barriers will protective barriers will protective barriers will break down and you break down and you break down and you will find yourself in the same will find yourself in the same will find yourself in the same situation where situation where situation where 200 of your important 200 of your important 200 of your important emails are gone. So what emails are gone. So what emails are gone. So what could actually could actually could actually stop it? Not a better stop it? Not a better stop it? Not a better instruction, but a wall instruction, but a wall instruction, but a wall that the agent cannot that the agent cannot that the agent cannot cross, even cross, even cross, even if he forgets about its if he forgets about its if he forgets about its existence. So we existence. So we existence. So we build these walls in build these walls in build these walls in two levels. The first two levels. The first two levels. The first level is level is level is deterministic deterministic deterministic control over control over control over what the agent can what the agent can what the agent can reach, what it

reach, what it reach, what it has access to. For example has access to. For example , a hiring agent , a hiring agent , a hiring agent can probably only can probably only can probably only read emails. A read emails. A read emails. A support agent can support agent can support agent can create a draft of an create a draft of an create a draft of an email but not email but not email but not send it. This send it. This send it. This boundary exists beyond boundary exists beyond boundary exists beyond these agents. these agents. these agents. The agent cannot The agent cannot The agent cannot argue with it, argue with it, argue with it, forget about it, or forget about it, or forget about it, or displace it. displace it. displace it. The instruction didn't The instruction didn't The instruction didn't work because it work because it work because it was in the was in the was in the agent's memory, in the agent's memory, in the agent's memory, in the request itself. And this is no. But request itself. And this is no. But request itself. And this is no. But restricting restricting restricting access alone wouldn't save access alone wouldn't save access alone wouldn't save her, since she her, since she her, since she was creating a was creating a was creating a mail agent. So mail agent. So mail agent. So he definitely he definitely he definitely needed access needed access needed access to that Gmail. Another thing to that Gmail. Another thing to that Gmail. Another thing we do is we do is we do is provide policies provide policies provide policies that can be used to that can be used to that can be used to define in natural define in natural define in natural language what an agent can language what an agent can language what an agent can do even with do even with do even with this this this access. For example: access. For example: access. For example: never delete never delete never delete more than 10 emails without more than 10 emails without more than 10 emails without my permission. Never my permission. Never my permission. Never send emails send emails send emails outside of a specific domain. outside of a specific domain. outside of a specific domain. Rules that control behavior even Rules that control behavior even Rules that control behavior even with such access with such access . So, . So, combining these two things combining these two things : one layer : one layer : one layer controls what the agent controls what the agent controls what the agent can read, and the other controls can read, and the other controls its behavior in terms of what its behavior in terms of what its behavior in terms of what it can it can it can do with the do with the do with the access it has. Together, this access it has. Together, this access it has. Together, this creates real creates real creates real control for the control for the control for the agent. We are not asking agent. We are not asking agent. We are not asking the agent to behave the agent to behave the agent to behave appropriately, we are appropriately, we are appropriately, we are limiting what they limiting what they limiting what they can do. The last can do. The last can do. The last pillar is the pillar is the pillar is the reversibility of actions. And this is reversibility of actions. And this is reversibility of actions. And this is one of those, this is what one of those, this is what one of those, this is what we come to we come to we come to when something goes wrong.

when something goes wrong. when something goes wrong. Can I Can I Can I cancel this? In code, it's cancel this? In code, it's cancel this? In code, it's almost always possible. almost always possible. almost always possible. Every change Every change Every change is recorded. Everything is recorded. Everything is recorded. Everything can be turned can be turned can be turned back. You can back. You can back. You can undo the last undo the last undo the last commit or find commit or find commit or find the one the one the one that broke production through bisection and that broke production through bisection and that broke production through bisection and roll back. Well, I'm not roll back. Well, I'm not roll back. Well, I'm not saying that's good. I won't saying that's good. I won't saying that's good. I won't pretend that's the pretend that's the pretend that's the case. If something case. If something case. If something gets into production gets into production gets into production and breaks, it's and breaks, it's and breaks, it's always bad, but not always bad, but not always bad, but not forever. You forever. You forever. You can still turn it all can still turn it all can still turn it all back. And that's what gives you back. And that's what gives you back. And that's what gives you the confidence to the confidence to the confidence to let your let your let your agents work and agents work and agents work and create magic, because create magic, because create magic, because even if they even if they even if they break something, you have a way break something, you have a way break something, you have a way back. There back. There is no "cancel" button for intellectual work. is no "cancel" button for intellectual work. Consider, for example, Consider, for example, Consider, for example, working with mail. Those 200 working with mail. Those 200 working with mail. Those 200 letters disappeared. They letters disappeared. They letters disappeared. They simply evaporated. simply evaporated. simply evaporated. By the way, this is a typical By the way, this is a typical By the way, this is a typical situation. A situation. A situation. A catastrophic catastrophic catastrophic event is a event is a event is a sent letter sent letter sent letter that cannot that cannot that cannot be recalled. A transfer be recalled. A transfer be recalled. A transfer that has already been made. that has already been made. that has already been made. So you can't get So you can't get So you can't get that money back. A that money back. A that money back. A deleted entry deleted entry deleted entry disappears forever. disappears forever. disappears forever. Most actions in Most actions in Most actions in intelligent intelligent intelligent work don't actually work don't actually work don't actually have an have an have an undo button. And that undo button. And that undo button. And that changes the whole equation.

changes the whole equation. changes the whole equation. This changes the scale of This changes the scale of This changes the scale of possible losses. With a possible losses. With a possible losses. With a code, you can code, you can code, you can trust the agent trust the agent trust the agent after the fact. after the fact. after the fact. Let him Let him Let him do the work, do the work, do the work, check the result, check the result, check the result, cancel if he cancel if he cancel if he made a mistake. There is made a mistake. There is made a mistake. There is no way back here. The no way back here. The no way back here. The only thing only thing only thing left for you is left for you is left for you is to trust the agent before to trust the agent before to trust the agent before he starts he starts he starts acting. That's why these acting. That's why these acting. That's why these agents seem more agents seem more agents seem more dangerous than dangerous than coding agents ever were. It's not that coding agents ever were. It's not that they are often they are often they are often wrong. And the fact that wrong. And the fact that wrong. And the fact that there is a mistake there is there is a mistake there is there is a mistake there is forever. So you either forever. So you either forever. So you either trust completely trust completely trust completely right away or right away or right away or don't allow it to act at all. I don't allow it to act at all. I 'll be honest, 'll be honest, reversibility is the hardest thing to reversibility is the hardest thing to reversibility is the hardest thing to recreate in recreate in recreate in intellectual intellectual intellectual work. True work. True work. True undo, as in code, is undo, as in code, is undo, as in code, is probably not possible for probably not possible for probably not possible for all all intelligence scenarios. But we have intelligence scenarios. But we have certain scenarios where certain scenarios where certain scenarios where cancellation works, and cancellation works, and cancellation works, and we use them. we use them. we use them. Let's say you added a Let's say you added a Let's say you added a label. You can label. You can label. You can delete it later. delete it later. But for actions that But for actions that cannot be undone, cannot be undone, cannot be undone, such as completely such as completely such as completely deleting emails from deleting emails from deleting emails from your mailbox, we your mailbox, we your mailbox, we again provide a again provide a again provide a "sandbox" where the agent "sandbox" where the agent "sandbox" where the agent first performs the action, first performs the action, first performs the action, you verify it, and you verify it, and you verify it, and only then does it only then does it only then does it enter the production enter the production enter the production environment. None of environment. None of environment. None of this applies to the this applies to the this applies to the real world. This is the real world. This is the real world. This is the whole change in whole change in whole change in approach. In code, you can approach. In code, you can approach. In code, you can undo an error undo an error undo an error after it after it after it has occurred. Here you has occurred. Here you has occurred. Here you fix it before fix it before fix it before it it it happens. Different time happens. Different time , same result.

, same result. , same result. A mistake that won't A mistake that won't A mistake that won't stick. stick. stick. Think about it Think about it Think about it again. again. again. We We We provided an provided an provided an "undo" button for actions that we could undo. "undo" button for actions that we could undo. "undo" button for actions that we could undo. Those that couldn't—the agent Those that couldn't—the agent Those that couldn't—the agent first sent to the " first sent to the " sandbox," and sandbox," and sandbox," and the user received the user received the user received a notification: your 1,200 a notification: your 1,200 a notification: your 1,200 emails will be deleted. emails will be deleted. emails will be deleted. Would you like to continue? Would you like to continue? It's not ready yet. But It's not ready yet. But among the billions of actions among the billions of actions among the billions of actions we process, we process, we process, we learn on the fly we learn on the fly we learn on the fly which ones can be which ones can be which ones can be rolled back and which ones can't, and rolled back and which ones can't, and rolled back and which ones can't, and prepare a " prepare a " sandbox" accordingly. If you sandbox" accordingly. If you remember remember one thing today, let it be one thing today, let it be one thing today, let it be this. For two this. For two this. For two years, the bottleneck years, the bottleneck years, the bottleneck was the model. Therefore, everyone was the model. Therefore, everyone was the model. Therefore, everyone competed to competed to competed to create an increasingly create an increasingly create an increasingly better model. Now better model. Now better model. Now the models have become the models have become the models have become so so so advanced that advanced that advanced that software development software development software development can can can be completely be completely be completely autonomous. But now autonomous. But now everything else has become the bottleneck. This same everything else has become the bottleneck. This same model that writes your model that writes your model that writes your code can also code can also code can also handle hiring, handle hiring, handle hiring, sales, and other sales, and other sales, and other intellectual intellectual intellectual work. But now work. But now work. But now she is working blindly. she is working blindly. she is working blindly. No history, No history, No history, no context, no context, no context, no no no verification methods, no verification methods, no verification methods, no safeguards or safeguards or safeguards or undo actions. So, the undo actions. So, the undo actions. So, the bottleneck bottleneck bottleneck has shifted. Now this is has shifted. Now this is has shifted. Now this is infrastructure that no infrastructure that no infrastructure that no one has built before one has built before , and that's what we're , and that's what we're , and that's what we're building at Compose.io.

building at Compose.io. Yes. We provide Yes. We provide over a billion over a billion tool calls in total . There are . There are . There are 300 300 300 million million million tool calls every month. And if tool calls every month. And if tool calls every month. And if you’re building an agent, you’re building an agent, you’re building an agent, just point just point just point it to Compose.io and it to Compose.io and it to Compose.io and watch the watch the watch the miracle of miracle of miracle of intellectual intellectual intellectual labor unfold. And if you labor unfold. And if you labor unfold. And if you want to create the want to create the want to create the future of the future of the future of the AI ​​agent platform, AI ​​agent platform, AI ​​agent platform, please contact please contact please contact me. We are definitely me. We are definitely me. We are definitely hiring hiring , and there is still a lot of , and there is still a lot of , and there is still a lot of work ahead. The models work ahead. The models work ahead. The models will continue to will continue to will continue to get better. The get better. The get better. The bottleneck bottleneck bottleneck will not be the models. It will not be the models. It will not be the models. It will be what surrounds them. will be what surrounds them. will be what surrounds them. Thank you.