Runable Raised $21 Million On Agents That Finish. Nobody Told Yours What Done Means.

Every business failure in the video I'm Every business failure in the video I'm about to show you today comes down to about to show you today comes down to about to show you today comes down to one. The agents did an enormous amount one. The agents did an enormous amount one. The agents did an enormous amount of sophisticated relentless work, and of sophisticated relentless work, and of sophisticated relentless work, and not one hour of that work was work that not one hour of that work was work that not one hour of that work was work that anybody wanted. If you can't say what anybody wanted. If you can't say what anybody wanted. If you can't say what done means before you actually install done means before you actually install done means before you actually install your agent and get it going, then what your agent and get it going, then what your agent and get it going, then what you're buying is a bunch of process from you're buying is a bunch of process from you're buying is a bunch of process from agents and not a lot of value. A very agents and not a lot of value. A very agents and not a lot of value. A very hard worker with the wrong finish line. hard worker with the wrong finish line. hard worker with the wrong finish line. How come agents are so smart they can How come agents are so smart they can How come agents are so smart they can find zero-day vulnerabilities and they find zero-day vulnerabilities and they find zero-day vulnerabilities and they can code up entire pieces of software can code up entire pieces of software can code up entire pieces of software end to end, and yet we still see them end to end, and yet we still see them end to end, and yet we still see them misaligned. We still see them falling misaligned. We still see them falling misaligned. We still see them falling into process errors. We still have so into process errors. We still have so into process errors. We still have so many articles being spilled across the many articles being spilled across the many articles being spilled across the internet talking about just getting your internet talking about just getting your internet talking about just getting your agents to work. I have literally seen a agents to work. I have literally seen a agents to work. I have literally seen a press release yesterday from a startup press release yesterday from a startup press release yesterday from a startup that raised $21 million, and the thing that raised $21 million, and the thing that raised $21 million, and the thing they wanted to say is that our agents they wanted to say is that our agents they wanted to say is that our agents actually get work done. We clearly have actually get work done. We clearly have actually get work done. We clearly have a problem here, and it's not just our a problem here, and it's not just our a problem here, and it's not just our problem. It's not just us as doers, it's problem. It's not just us as doers, it's problem. It's not just us as doers, it's the lab's problem. On August 26th, the lab's problem. On August 26th, the lab's problem. On August 26th, OpenAI published the full report on what OpenAI published the full report on what OpenAI published the full report on what happened when its experimental agents happened when its experimental agents happened when its experimental agents broke out of a cybersecurity evaluation broke out of a cybersecurity evaluation broke out of a cybersecurity evaluation and attacked Hugging Face. About 1,200 and attacked Hugging Face. About 1,200 and attacked Hugging Face. About 1,200 agents found one another on an agents found one another on an agents found one another on an unauthorized message board inside unauthorized message board inside unauthorized message board inside OpenAI. They exchanged more than 70,000 OpenAI. They exchanged more than 70,000 OpenAI. They exchanged more than 70,000 messages and files. Roughly 700 of them messages and files. Roughly 700 of them messages and files. Roughly 700 of them eventually joined the attack on Hugging eventually joined the attack on Hugging eventually joined the attack on Hugging Face. 700 different agents. But, here's Face. 700 different agents. But, here's Face. 700 different agents. But, here's the part I can't stop thinking about.

the part I can't stop thinking about. the part I can't stop thinking about. Nobody assigned those agents to attack Nobody assigned those agents to attack Nobody assigned those agents to attack Hugging Face. Many of them had been Hugging Face. Many of them had been Hugging Face. Many of them had been assigned benchmark problems that were assigned benchmark problems that were assigned benchmark problems that were effectively impossible, and they were effectively impossible, and they were effectively impossible, and they were trying to make the greater agent give trying to make the greater agent give trying to make the greater agent give them a passing score anyway. They were them a passing score anyway. They were them a passing score anyway. They were desperate to please. So, they reverse desperate to please. So, they reverse desperate to please. So, they reverse engineered the scoring system, shared engineered the scoring system, shared engineered the scoring system, shared ways to cheat on it, found a path onto ways to cheat on it, found a path onto ways to cheat on it, found a path onto the internet, and kept going until they the internet, and kept going until they the internet, and kept going until they could pass their cybersecurity eval. The could pass their cybersecurity eval. The could pass their cybersecurity eval. The irony is not lost on me. By the way, irony is not lost on me. By the way, irony is not lost on me. By the way, here I am in a hotel, AI news won't here I am in a hotel, AI news won't here I am in a hotel, AI news won't wait. This safety failure also gives us wait. This safety failure also gives us wait. This safety failure also gives us an x-ray of a trillion-dollar business an x-ray of a trillion-dollar business an x-ray of a trillion-dollar business problem, and I feel like we don't talk problem, and I feel like we don't talk problem, and I feel like we don't talk about it enough. We know how to train an about it enough. We know how to train an about it enough. We know how to train an agent to chase a passing score. That's agent to chase a passing score. That's agent to chase a passing score. That's what OpenAI was doing, that's what the what OpenAI was doing, that's what the what OpenAI was doing, that's what the other labs are doing. But, almost nobody other labs are doing. But, almost nobody other labs are doing. But, almost nobody can tell a business how to install an can tell a business how to install an can tell a business how to install an agent that does useful work, and know in agent that does useful work, and know in agent that does useful work, and know in ordinary business terms when the work is ordinary business terms when the work is ordinary business terms when the work is actually finished. And the problem is actually finished. And the problem is actually finished. And the problem is the answer to that changes with the the answer to that changes with the the answer to that changes with the buyer. This is why a lot of vendors are buyer. This is why a lot of vendors are buyer. This is why a lot of vendors are making a lot of money on AI right now. making a lot of money on AI right now. making a lot of money on AI right now. An enterprise can build its own An enterprise can build its own An enterprise can build its own environment for teaching and testing environment for teaching and testing environment for teaching and testing agents, but not everybody can, right? A agents, but not everybody can, right? A agents, but not everybody can, right? A small business has to stay much closer small business has to stay much closer small business has to stay much closer to the code and much closer to the cash to the code and much closer to the cash to the code and much closer to the cash register. They can't splash out as far.

register. They can't splash out as far. register. They can't splash out as far. And a solopreneur or an entrepreneur has And a solopreneur or an entrepreneur has And a solopreneur or an entrepreneur has to know where personal expertise is to know where personal expertise is to know where personal expertise is going to end, and they have to be able going to end, and they have to be able going to end, and they have to be able to figure out what they do with agents, to figure out what they do with agents, to figure out what they do with agents, where a hidden error that they can't where a hidden error that they can't where a hidden error that they can't diagnose could be expensive, where their diagnose could be expensive, where their diagnose could be expensive, where their domain expertise runs out and they domain expertise runs out and they domain expertise runs out and they really need agents. If you're wondering, really need agents. If you're wondering, really need agents. If you're wondering, you know, and you're at a big business, you know, and you're at a big business, you know, and you're at a big business, at a small business, as a solopreneur, I at a small business, as a solopreneur, I at a small business, as a solopreneur, I am going to cover all of those cases in am going to cover all of those cases in am going to cover all of those cases in this video. And yes, there will be a this video. And yes, there will be a this video. And yes, there will be a much deeper dive over on the Substack, much deeper dive over on the Substack, much deeper dive over on the Substack, and I want you to remember cuz it helps and I want you to remember cuz it helps and I want you to remember cuz it helps you understand what's going on here. The you understand what's going on here. The you understand what's going on here. The agent is looking for a passing agent is looking for a passing agent is looking for a passing condition. In other words, the agent is condition. In other words, the agent is condition. In other words, the agent is trying to pass an exam with you as the trying to pass an exam with you as the trying to pass an exam with you as the business owner, with you as the manager. business owner, with you as the manager. business owner, with you as the manager. If you want useful work out of your If you want useful work out of your If you want useful work out of your agent, agent, agent, you have to make sure that your passing you have to make sure that your passing you have to make sure that your passing condition for the agent represents a condition for the agent represents a condition for the agent represents a business result that you actually care business result that you actually care business result that you actually care about. But first, back to OpenAI, back about. But first, back to OpenAI, back about. But first, back to OpenAI, back to the Hugging Face incident. Why are we to the Hugging Face incident. Why are we to the Hugging Face incident. Why are we talking about it? The incident gives us talking about it? The incident gives us talking about it? The incident gives us an almost comically clear look inside an almost comically clear look inside an almost comically clear look inside what I'm calling agent school. These what I'm calling agent school. These what I'm calling agent school. These systems grow up taking tests, receiving systems grow up taking tests, receiving systems grow up taking tests, receiving rewards for answers that can be rewards for answers that can be rewards for answers that can be verified, facing evals they may or may verified, facing evals they may or may verified, facing evals they may or may not know about and learning to keep not know about and learning to keep not know about and learning to keep trying until they produce something that trying until they produce something that trying until they produce something that passes. Then we take that graduate out passes. Then we take that graduate out passes. Then we take that graduate out of that school, we deploy it white, we of that school, we deploy it white, we of that school, we deploy it white, we drop it into a company and we say, "Now drop it into a company and we say, "Now drop it into a company and we say, "Now you go get work done for this company."

you go get work done for this company." you go get work done for this company." The company has usually not defined the The company has usually not defined the The company has usually not defined the test or connected the tools or decided test or connected the tools or decided test or connected the tools or decided which actions the agent may take in which actions the agent may take in which actions the agent may take in advance unless they're pretty advance unless they're pretty advance unless they're pretty sophisticated, nor have they turned the sophisticated, nor have they turned the sophisticated, nor have they turned the judgment of a good employee into judgment of a good employee into judgment of a good employee into examples the agent can learn from. examples the agent can learn from. examples the agent can learn from. Again, unless they really know what Again, unless they really know what Again, unless they really know what they're doing. The company may not even they're doing. The company may not even they're doing. The company may not even have settled what useful work means for have settled what useful work means for have settled what useful work means for an agent to do in that part of the an agent to do in that part of the an agent to do in that part of the business. And then everyone's surprised business. And then everyone's surprised business. And then everyone's surprised when the agent produces a plan, a when the agent produces a plan, a when the agent produces a plan, a report, a chain of reasoning, a pile of report, a chain of reasoning, a pile of report, a chain of reasoning, a pile of updates, and another request for updates, and another request for updates, and another request for approval, and at the end of it you're approval, and at the end of it you're approval, and at the end of it you're like, "But what changed in the business? like, "But what changed in the business? like, "But what changed in the business? Did any actual work get done or did the Did any actual work get done or did the Did any actual work get done or did the agent just produce a pile of documents?" agent just produce a pile of documents?" agent just produce a pile of documents?" This week, a startup called Runnable This week, a startup called Runnable This week, a startup called Runnable announced a $21 million series A and announced a $21 million series A and announced a $21 million series A and launched an agent it says can run an launched an agent it says can run an launched an agent it says can run an entire go-to-market operation for a entire go-to-market operation for a entire go-to-market operation for a small business. The founders' pitch is small business. The founders' pitch is small business. The founders' pitch is almost painfully direct at what we're almost painfully direct at what we're almost painfully direct at what we're talking about here. Runnable says their talking about here. Runnable says their talking about here. Runnable says their product does the work instead of just product does the work instead of just product does the work instead of just giving the owner a dashboard or a giving the owner a dashboard or a giving the owner a dashboard or a co-pilot to chat with or a bunch of co-pilot to chat with or a bunch of co-pilot to chat with or a bunch of docs. It does stuff. It does stuff. Are docs. It does stuff. It does stuff. Are docs. It does stuff. It does stuff. Are you hearing that? That line works you hearing that? That line works you hearing that? That line works because everyone knows the normal AI because everyone knows the normal AI because everyone knows the normal AI agent doesn't do that without agent doesn't do that without agent doesn't do that without configuration. And this is why this configuration. And this is why this configuration. And this is why this agent school is why. We have agents that agent school is why. We have agents that agent school is why. We have agents that can find zero-day vulnerabilities, sure.

can find zero-day vulnerabilities, sure. can find zero-day vulnerabilities, sure. They can coordinate with hundreds of They can coordinate with hundreds of They can coordinate with hundreds of copies of themselves, they can write copies of themselves, they can write copies of themselves, they can write software, they can operate a browser, software, they can operate a browser, software, they can operate a browser, and they can reason for an extraordinary and they can reason for an extraordinary and they can reason for an extraordinary number of steps. Those That's all true. number of steps. Those That's all true. number of steps. Those That's all true. Meanwhile, a business owner is still Meanwhile, a business owner is still Meanwhile, a business owner is still asking whether this agent is going to asking whether this agent is going to asking whether this agent is going to follow up with the lead tomorrow without follow up with the lead tomorrow without follow up with the lead tomorrow without being reminded. This is one reason being reminded. This is one reason being reminded. This is one reason coding agents have gotten so good before coding agents have gotten so good before coding agents have gotten so good before most other agents have. Code gives most other agents have. Code gives most other agents have. Code gives agents an unusually dense world of agents an unusually dense world of agents an unusually dense world of useful feedback. Does the file parse, useful feedback. Does the file parse, useful feedback. Does the file parse, right? Does the compiler report an right? Does the compiler report an right? Does the compiler report an error? Does the test pass or the test error? Does the test pass or the test error? Does the test pass or the test fail? Does the application run? Is the fail? Does the application run? Is the fail? Does the application run? Is the pull request exactly what changed or pull request exactly what changed or pull request exactly what changed or not? A coding agent can try something, not? A coding agent can try something, not? A coding agent can try something, it can receive an answer from the it can receive an answer from the it can receive an answer from the environment, and it can try again environment, and it can try again environment, and it can try again without waiting for a person to without waiting for a person to without waiting for a person to reconstruct the whole job. Reinforcement reconstruct the whole job. Reinforcement reconstruct the whole job. Reinforcement learning from verifiable rewards, or learning from verifiable rewards, or learning from verifiable rewards, or RLVR, takes advantage of that exact RLVR, takes advantage of that exact RLVR, takes advantage of that exact property during agent training. You give property during agent training. You give property during agent training. You give the model a problem with an answer that the model a problem with an answer that the model a problem with an answer that can be checked, and then you reward can be checked, and then you reward can be checked, and then you reward successful runs, and you can improve successful runs, and you can improve successful runs, and you can improve performance without asking a person to performance without asking a person to performance without asking a person to grade every single intermediate thought, grade every single intermediate thought, grade every single intermediate thought, which is just becoming exhaustive for which is just becoming exhaustive for which is just becoming exhaustive for the humans at the labs, and that's why the humans at the labs, and that's why the humans at the labs, and that's why they're doing this. Mathematics and code they're doing this. Mathematics and code they're doing this. Mathematics and code are especially useful here because the are especially useful here because the are especially useful here because the checking can be very fast, it can be checking can be very fast, it can be checking can be very fast, it can be very unforgiving, and you can make fast very unforgiving, and you can make fast very unforgiving, and you can make fast progress. And one thing we know about progress. And one thing we know about progress. And one thing we know about the labs is they think of it as a race.

the labs is they think of it as a race. the labs is they think of it as a race. That machinery is part of why these That machinery is part of why these That machinery is part of why these models are so capable, especially in models are so capable, especially in models are so capable, especially in those two domains. The problem begins those two domains. The problem begins those two domains. The problem begins when the passing condition stops when the passing condition stops when the passing condition stops representing real work that humans would representing real work that humans would representing real work that humans would recognize. In the hugging face incident, recognize. In the hugging face incident, recognize. In the hugging face incident, some agents had tasks with no known some agents had tasks with no known some agents had tasks with no known correct solution, but the grader still correct solution, but the grader still correct solution, but the grader still existed, so the machine still felt existed, so the machine still felt existed, so the machine still felt pressure to pass. Instead of concluding pressure to pass. Instead of concluding pressure to pass. Instead of concluding that the work couldn't be completed that the work couldn't be completed that the work couldn't be completed safely, the agents spent an enormous safely, the agents spent an enormous safely, the agents spent an enormous amount of effort trying to manipulate amount of effort trying to manipulate amount of effort trying to manipulate and pass the test. They were so and pass the test. They were so and pass the test. They were so motivated to pass. Businesses can create motivated to pass. Businesses can create motivated to pass. Businesses can create milder versions of the hugging face milder versions of the hugging face milder versions of the hugging face mistake constantly. A sales agent told mistake constantly. A sales agent told mistake constantly. A sales agent told to send a hundred emails will optimize to send a hundred emails will optimize to send a hundred emails will optimize for the send, not necessarily for the for the send, not necessarily for the for the send, not necessarily for the value of the email. A support agent value of the email. A support agent value of the email. A support agent graded on closed tickets will learn graded on closed tickets will learn graded on closed tickets will learn which action makes a ticket disappear, which action makes a ticket disappear, which action makes a ticket disappear, and will learn which kinds of tickets to and will learn which kinds of tickets to and will learn which kinds of tickets to pick up in order to optimize to that pick up in order to optimize to that pick up in order to optimize to that count. They'll pick the easier ones. count. They'll pick the easier ones. count. They'll pick the easier ones. Spoiler alert, this this is something Spoiler alert, this this is something Spoiler alert, this this is something I've actually seen. A coding agent told I've actually seen. A coding agent told I've actually seen. A coding agent told only that every test must pass may only that every test must pass may only that every test must pass may weaken the test, may special case the weaken the test, may special case the weaken the test, may special case the example, or produce a giant function example, or produce a giant function example, or produce a giant function that satisfies the current suite but is that satisfies the current suite but is that satisfies the current suite but is not useful code. So, the activity can be not useful code. So, the activity can be not useful code. So, the activity can be real and the score can improve while the real and the score can improve while the real and the score can improve while the company becomes worse off in the end.

company becomes worse off in the end. company becomes worse off in the end. Now, if we want to solve this problem, Now, if we want to solve this problem, Now, if we want to solve this problem, we need to solve it differently at we need to solve it differently at we need to solve it differently at different scales. I talked about how different scales. I talked about how different scales. I talked about how different scales have agents that show different scales have agents that show different scales have agents that show up differently last week. We're up differently last week. We're up differently last week. We're extending that idea this week. Let's extending that idea this week. Let's extending that idea this week. Let's start with enterprise because large start with enterprise because large start with enterprise because large companies can spend their way into a companies can spend their way into a companies can spend their way into a much more complete answer than most much more complete answer than most much more complete answer than most people realize. A lot of them don't, but people realize. A lot of them don't, but people realize. A lot of them don't, but they can. An enterprise can build a they can. An enterprise can build a they can. An enterprise can build a school for agents inside the company. It school for agents inside the company. It school for agents inside the company. It can decide what agents are for. It can can decide what agents are for. It can can decide what agents are for. It can put them where employees already work. put them where employees already work. put them where employees already work. It can give them controlled access to It can give them controlled access to It can give them controlled access to company tools. It can collect examples company tools. It can collect examples company tools. It can collect examples of good work. It can maintain evals that of good work. It can maintain evals that of good work. It can maintain evals that reflect the company's actual standards. reflect the company's actual standards. reflect the company's actual standards. In a sense, what good founders are doing In a sense, what good founders are doing In a sense, what good founders are doing right now is they're putting in place a right now is they're putting in place a right now is they're putting in place a set of expectations that reflect the set of expectations that reflect the set of expectations that reflect the founder's vision for agent human founder's vision for agent human founder's vision for agent human collaboration. Block did not simply buy collaboration. Block did not simply buy collaboration. Block did not simply buy a chatbot license and tell 10,000 people a chatbot license and tell 10,000 people a chatbot license and tell 10,000 people to be more productive. Instead, Jack to be more productive. Instead, Jack to be more productive. Instead, Jack Dorsey backed Goose, an internal agent Dorsey backed Goose, an internal agent Dorsey backed Goose, an internal agent that sits over different models and can that sits over different models and can that sits over different models and can use configurable tools for coding and use configurable tools for coding and use configurable tools for coding and other company work. Shopify built River other company work. Shopify built River other company work. Shopify built River as an agent that lives in shared Slack as an agent that lives in shared Slack as an agent that lives in shared Slack threads and it's supported by an threads and it's supported by an threads and it's supported by an internal platform called Aquifer. Tobi internal platform called Aquifer. Tobi internal platform called Aquifer. Tobi Lutke's point is that if every agent Lutke's point is that if every agent Lutke's point is that if every agent conversation happens in a private conversation happens in a private conversation happens in a private window, the only person who learns is window, the only person who learns is window, the only person who learns is the person the keyboard. That is a the person the keyboard. That is a the person the keyboard. That is a founder choice. That's not a model founder choice. That's not a model founder choice. That's not a model feature. And by the way, it's the feature. And by the way, it's the feature. And by the way, it's the correct choice. When the agent has to correct choice. When the agent has to correct choice. When the agent has to work where humans work, when the agent work where humans work, when the agent work where humans work, when the agent works in Slack, in Linear, in Jira, in works in Slack, in Linear, in Jira, in works in Slack, in Linear, in Jira, in Azure DevOps, in Teams, whatever system Azure DevOps, in Teams, whatever system Azure DevOps, in Teams, whatever system is holding your work, right? Other is holding your work, right? Other is holding your work, right? Other people can see the request and the people can see the request and the people can see the request and the answer, the system of record, what's

answer, the system of record, what's answer, the system of record, what's been corrected, and the result. And so, been corrected, and the result. And so, been corrected, and the result. And so, a useful correction becomes part of how a useful correction becomes part of how a useful correction becomes part of how agents and humans at the company agents and humans at the company agents and humans at the company actually get work done instead of just actually get work done instead of just actually get work done instead of just disappearing into somebody's chat disappearing into somebody's chat disappearing into somebody's chat history. The enterprise also has to make history. The enterprise also has to make history. The enterprise also has to make a decision that most agent pilots tend a decision that most agent pilots tend a decision that most agent pilots tend to avoid. What does this company mean by to avoid. What does this company mean by to avoid. What does this company mean by doing good work in passing, right? If doing good work in passing, right? If doing good work in passing, right? If you're passing an eval. you're passing an eval. you're passing an eval. Take code. Passing cannot mean that the Take code. Passing cannot mean that the Take code. Passing cannot mean that the new feature appears to work in the demo. new feature appears to work in the demo. new feature appears to work in the demo. That wouldn't be good enough. I would That wouldn't be good enough. I would That wouldn't be good enough. I would start with a much more ordinary start with a much more ordinary start with a much more ordinary question. And this is one of the question. And this is one of the question. And this is one of the questions I want to cover, and I think questions I want to cover, and I think questions I want to cover, and I think this works not just for enterprise, it this works not just for enterprise, it this works not just for enterprise, it works at all scales. Can your average, works at all scales. Can your average, works at all scales. Can your average, can your second or third best engineer, can your second or third best engineer, can your second or third best engineer, someone who's not top tier, not the one someone who's not top tier, not the one someone who's not top tier, not the one person who understands every corner of person who understands every corner of person who understands every corner of the system, can that person who's kind the system, can that person who's kind the system, can that person who's kind of second or third quality open a random of second or third quality open a random of second or third quality open a random file and explain what it does and why file and explain what it does and why file and explain what it does and why it's there after 20 minutes or less? And it's there after 20 minutes or less? And it's there after 20 minutes or less? And yes, that would be a file coded by an yes, that would be a file coded by an yes, that would be a file coded by an agent. That question has teeth I find agent. That question has teeth I find agent. That question has teeth I find because the company is going to have to because the company is going to have to because the company is going to have to live with that code after the agent live with that code after the agent live with that code after the agent moves on to other things. That code has moves on to other things. That code has moves on to other things. That code has to be sustained. If an ordinary, to be sustained. If an ordinary, to be sustained. If an ordinary, reasonably competent, but not brilliant reasonably competent, but not brilliant reasonably competent, but not brilliant engineer can't trace the logic, see the engineer can't trace the logic, see the engineer can't trace the logic, see the boundaries, understand what's going on, boundaries, understand what's going on, boundaries, understand what's going on, find the tests, and tell how the file find the tests, and tell how the file find the tests, and tell how the file connects into the rest of code base, the connects into the rest of code base, the connects into the rest of code base, the agent may have completed the ticket agent may have completed the ticket agent may have completed the ticket while making the code base much harder while making the code base much harder while making the code base much harder to operate and sustain over the long to operate and sustain over the long to operate and sustain over the long term. Then we can add in normal term. Then we can add in normal term. Then we can add in normal engineering checks. How large are the engineering checks. How large are the engineering checks. How large are the files and the functions becoming? This

files and the functions becoming? This files and the functions becoming? This is something that I find that humans is something that I find that humans is something that I find that humans tend to get really frustrated by if you tend to get really frustrated by if you tend to get really frustrated by if you give a human a random number of lines of give a human a random number of lines of give a human a random number of lines of code and say your file can be no bigger code and say your file can be no bigger code and say your file can be no bigger than X, than X, than X, the human gets frustrated. The agent the human gets frustrated. The agent the human gets frustrated. The agent needs that constraint. Is the agent needs that constraint. Is the agent needs that constraint. Is the agent building modules that the rest of the building modules that the rest of the building modules that the rest of the product can reuse in a structured way? product can reuse in a structured way? product can reuse in a structured way? Is it just finding a new implementation Is it just finding a new implementation Is it just finding a new implementation pathway every time? That's not very pathway every time? That's not very pathway every time? That's not very sustainable. Does the work tree and sustainable. Does the work tree and sustainable. Does the work tree and branch system that you're using work branch system that you're using work branch system that you're using work cleanly enough that another engineer can cleanly enough that another engineer can cleanly enough that another engineer can enter the work without digging up enter the work without digging up enter the work without digging up whatever has been going on. Do the whatever has been going on. Do the whatever has been going on. Do the comments that you're using explain your comments that you're using explain your comments that you're using explain your thoughtful engineered trade-off thoughtful engineered trade-off thoughtful engineered trade-off decisions rather than just narrating decisions rather than just narrating decisions rather than just narrating obvious syntax? That's something you obvious syntax? That's something you obvious syntax? That's something you really have to push agents on. Are the really have to push agents on. Are the really have to push agents on. Are the tests protecting behavior? Are they tests protecting behavior? Are they tests protecting behavior? Are they rewritten to bless the answer or are rewritten to bless the answer or are rewritten to bless the answer or are they really really tough tests? I One they really really tough tests? I One they really really tough tests? I One example here that I love, and I will example here that I love, and I will example here that I love, and I will define it for you, is what I call define it for you, is what I call define it for you, is what I call cyclomatic complexity. Again, human cyclomatic complexity. Again, human cyclomatic complexity. Again, human engineers it drives them nuts. We don't engineers it drives them nuts. We don't engineers it drives them nuts. We don't tend to use it with humans. It's great tend to use it with humans. It's great tend to use it with humans. It's great with agents. It counts cyclomatic with agents. It counts cyclomatic with agents. It counts cyclomatic complexity counts the independent complexity counts the independent complexity counts the independent decision paths that run through a piece decision paths that run through a piece decision paths that run through a piece of code.

of code. of code. A function with one straight path is A function with one straight path is A function with one straight path is easier to reason about than one with easier to reason about than one with easier to reason about than one with dozens of nested branches and special dozens of nested branches and special dozens of nested branches and special cases and early exits. I have seen the cases and early exits. I have seen the cases and early exits. I have seen the number of paths fall by dozens. I've number of paths fall by dozens. I've number of paths fall by dozens. I've seen it fall from 91 to like 12 after an seen it fall from 91 to like 12 after an seen it fall from 91 to like 12 after an audit. That doesn't prove that the code audit. That doesn't prove that the code audit. That doesn't prove that the code is perfect, is perfect, is perfect, but it dramatically reduces the amount but it dramatically reduces the amount but it dramatically reduces the amount of complexity that an engineer has to of complexity that an engineer has to of complexity that an engineer has to understand. The point is not that every understand. The point is not that every understand. The point is not that every single company needs the same file limit single company needs the same file limit single company needs the same file limit or one magic complexity score. I'm not or one magic complexity score. I'm not or one magic complexity score. I'm not trying to give you a magic pill here. trying to give you a magic pill here. trying to give you a magic pill here. The point is that the agent needs to The point is that the agent needs to The point is that the agent needs to know that the code will be judged as know that the code will be judged as know that the code will be judged as part of a living system. part of a living system. part of a living system. The feature working today is one The feature working today is one The feature working today is one particular way to test, right? Like the particular way to test, right? Like the particular way to test, right? Like the feature working today, yeah, you do have feature working today, yeah, you do have feature working today, yeah, you do have to pass that test. It does have to work. to pass that test. It does have to work. to pass that test. It does have to work. But whether an ordinary engineer can But whether an ordinary engineer can But whether an ordinary engineer can safely extend that feature 6 months from safely extend that feature 6 months from safely extend that feature 6 months from now, that's something you have to put a now, that's something you have to put a now, that's something you have to put a lot of work into because agents don't lot of work into because agents don't lot of work into because agents don't think about that. Knowledge work needs think about that. Knowledge work needs think about that. Knowledge work needs the same seriousness without pretending the same seriousness without pretending the same seriousness without pretending that knowledge work has a magical that knowledge work has a magical that knowledge work has a magical compiler that either runs or doesn't compiler that either runs or doesn't compiler that either runs or doesn't run. Knowledge work is a process.

run. Knowledge work is a process. run. Knowledge work is a process. Knowledge work doesn't work like code. Knowledge work doesn't work like code. Knowledge work doesn't work like code. If an agent produces a product If an agent produces a product If an agent produces a product requirements document, for example. The requirements document, for example. The requirements document, for example. The enterprise can define the required enterprise can define the required enterprise can define the required inputs, it can define the standard inputs, it can define the standard inputs, it can define the standard structure, it can define the decisions structure, it can define the decisions structure, it can define the decisions the document needs to enable. the document needs to enable. the document needs to enable. It can define the evidence it has to use It can define the evidence it has to use It can define the evidence it has to use and the new thinking that it needs to and the new thinking that it needs to and the new thinking that it needs to add. All of that is really important add. All of that is really important add. All of that is really important because if a document that fills every because if a document that fills every because if a document that fills every heading heading heading with fluent text with fluent text with fluent text is still failing, it's failing because is still failing, it's failing because is still failing, it's failing because no one insisted no one insisted no one insisted that the agent had to actually that the agent had to actually that the agent had to actually meaningfully improve the value of the meaningfully improve the value of the meaningfully improve the value of the feature with the document process it's feature with the document process it's feature with the document process it's following. I don't hate agent-written following. I don't hate agent-written following. I don't hate agent-written documents when they're good, but they documents when they're good, but they documents when they're good, but they have to really, really insist on good have to really, really insist on good have to really, really insist on good data, good inputs, high-quality, clear data, good inputs, high-quality, clear data, good inputs, high-quality, clear writing, a clear before-and-after writing, a clear before-and-after writing, a clear before-and-after description of what the agent is description of what the agent is description of what the agent is proposing. A lot of stuff that you can proposing. A lot of stuff that you can proposing. A lot of stuff that you can insist on with a mixture of skills and insist on with a mixture of skills and insist on with a mixture of skills and evals that most people don't take the evals that most people don't take the evals that most people don't take the time to do because in the end you're time to do because in the end you're time to do because in the end you're optimizing for this document being optimizing for this document being optimizing for this document being readable again, just as you would with readable again, just as you would with readable again, just as you would with code. Can a product manager in 6 months code. Can a product manager in 6 months code. Can a product manager in 6 months pick up this document and find it pick up this document and find it pick up this document and find it useful? That's a question you should be useful? That's a question you should be useful? That's a question you should be asking. And this is where regardless of asking. And this is where regardless of asking. And this is where regardless of your size, whether you're small business your size, whether you're small business your size, whether you're small business or enterprise or whatever, having a or enterprise or whatever, having a or enterprise or whatever, having a sense of taste and ownership enters the sense of taste and ownership enters the sense of taste and ownership enters the picture. It's not that that person has picture. It's not that that person has picture. It's not that that person has to review every single word of agent to review every single word of agent to review every single word of agent output forever, but their job is to show

output forever, but their job is to show output forever, but their job is to show what good looks like, right? To decide what good looks like, right? To decide what good looks like, right? To decide the difficult cases, to over time the difficult cases, to over time the difficult cases, to over time through repeated corrections improve the through repeated corrections improve the through repeated corrections improve the standards that an agent knowledge system standards that an agent knowledge system standards that an agent knowledge system should be using. Now, an enterprise can should be using. Now, an enterprise can should be using. Now, an enterprise can afford the team that maintains that kind afford the team that maintains that kind afford the team that maintains that kind of machinery. They can invest, right? of machinery. They can invest, right? of machinery. They can invest, right? They can centralize the eval bench and They can centralize the eval bench and They can centralize the eval bench and the permissions and the tool connections the permissions and the tool connections the permissions and the tool connections and security limits and monitoring, and and security limits and monitoring, and and security limits and monitoring, and they can do all of that while keeping an they can do all of that while keeping an they can do all of that while keeping an eye on the P&L. Engineering can maintain eye on the P&L. Engineering can maintain eye on the P&L. Engineering can maintain the agent platform, right? The head of the agent platform, right? The head of the agent platform, right? The head of sales can decides what counts as sales can decides what counts as sales can decides what counts as qualified lead. I could go on and on. qualified lead. I could go on and on. qualified lead. I could go on and on. But, what should be obvious to you is But, what should be obvious to you is But, what should be obvious to you is that a small or medium business is not that a small or medium business is not that a small or medium business is not going to be able to do that. They're not going to be able to do that. They're not going to be able to do that. They're not going to be able to build Shopify's okra going to be able to build Shopify's okra going to be able to build Shopify's okra for system themselves. They're not going for system themselves. They're not going for system themselves. They're not going to be able to have an agent platform to be able to have an agent platform to be able to have an agent platform group, an eval team, a bunch of group, an eval team, a bunch of group, an eval team, a bunch of functions available to debate the functions available to debate the functions available to debate the permissions. And, that sounds like a permissions. And, that sounds like a permissions. And, that sounds like a disadvantage, but what I find is there disadvantage, but what I find is there disadvantage, but what I find is there is there is an edge there. It can force is there is an edge there. It can force is there is an edge there. It can force clarity in the way SMBs use agents. And, clarity in the way SMBs use agents. And, clarity in the way SMBs use agents. And, the way SMBs define work getting done is the way SMBs define work getting done is the way SMBs define work getting done is often much simpler than how enterprises often much simpler than how enterprises often much simpler than how enterprises do it. And, that itself can be an do it. And, that itself can be an do it. And, that itself can be an advantage. So, for most small advantage. So, for most small advantage. So, for most small businesses, they end up finding they businesses, they end up finding they businesses, they end up finding they need agents in a few places where better need agents in a few places where better need agents in a few places where better work can reach the cash register. And, work can reach the cash register. And, work can reach the cash register. And, that's really key. Like, if if it's not that's really key. Like, if if it's not that's really key. Like, if if it's not touching the cash register as an SMB, touching the cash register as an SMB, touching the cash register as an SMB, you're not going to pay attention to it you're not going to pay attention to it you're not going to pay attention to it long-term. It's not going to be worth long-term. It's not going to be worth long-term. It's not going to be worth your investment. Right now, I find when your investment. Right now, I find when your investment. Right now, I find when I'm talking to SMBs, two places get I'm talking to SMBs, two places get I'm talking to SMBs, two places get attention for agents. Number one, the attention for agents. Number one, the attention for agents. Number one, the core code base. A small software company core code base. A small software company core code base. A small software company may have only a few strong engineers. If

may have only a few strong engineers. If may have only a few strong engineers. If you give those people extraordinary you give those people extraordinary you give those people extraordinary coding agents, that can dramatically coding agents, that can dramatically coding agents, that can dramatically change how much business can build, but change how much business can build, but change how much business can build, but only if the code stays clean enough for only if the code stays clean enough for only if the code stays clean enough for that tiny team to operate successfully. that tiny team to operate successfully. that tiny team to operate successfully. In this situation, at the SMB level, the In this situation, at the SMB level, the In this situation, at the SMB level, the same checks still apply as they would at same checks still apply as they would at same checks still apply as they would at a larger scale. You need a work tree a larger scale. You need a work tree a larger scale. You need a work tree that another engineer can understand, that another engineer can understand, that another engineer can understand, and you need to make sure that you don't and you need to make sure that you don't and you need to make sure that you don't get to a just a giant pile of agent get to a just a giant pile of agent get to a just a giant pile of agent written code that nobody can edit, which written code that nobody can edit, which written code that nobody can edit, which is of course the nightmare that I see is of course the nightmare that I see is of course the nightmare that I see over and over and over again when small over and over and over again when small over and over and over again when small businesses that don't insist on strong businesses that don't insist on strong businesses that don't insist on strong agent written code standards try and do agent written code standards try and do agent written code standards try and do agent written code themselves. I find agent written code themselves. I find agent written code themselves. I find the standard is actually harsher in a the standard is actually harsher in a the standard is actually harsher in a small company because there's no one small company because there's no one small company because there's no one coming to rescue an incomprehensible coming to rescue an incomprehensible coming to rescue an incomprehensible system. You can't spend your way out of system. You can't spend your way out of system. You can't spend your way out of that very easily. If your second or that very easily. If your second or that very easily. If your second or third or fourth best engineer in an SMB third or fourth best engineer in an SMB third or fourth best engineer in an SMB can't explain the new code, that may can't explain the new code, that may can't explain the new code, that may mean that half of your technical mean that half of your technical mean that half of your technical organization can't maintain the product organization can't maintain the product organization can't maintain the product or more than half, and that's a problem. or more than half, and that's a problem. or more than half, and that's a problem. Like that's a problem you can't get out Like that's a problem you can't get out Like that's a problem you can't get out of very easily. And so it's really of very easily. And so it's really of very easily. And so it's really important to insist in an SMB that if important to insist in an SMB that if important to insist in an SMB that if you're writing code with agents, it's you're writing code with agents, it's you're writing code with agents, it's clean code. The second place I find SMBs clean code. The second place I find SMBs clean code. The second place I find SMBs jump on agents is the revenue line, and jump on agents is the revenue line, and jump on agents is the revenue line, and that's why I talked about the cash that's why I talked about the cash that's why I talked about the cash register at the top of this section.

register at the top of this section. register at the top of this section. This is why Runnable is going after This is why Runnable is going after This is why Runnable is going after go-to-market work. Small businesses are go-to-market work. Small businesses are go-to-market work. Small businesses are desperate for distribution. They need desperate for distribution. They need desperate for distribution. They need someone to find prospects, to contact someone to find prospects, to contact someone to find prospects, to contact them, to qualify interest, to run them, to qualify interest, to run them, to qualify interest, to run campaigns, to get attention from campaigns, to get attention from campaigns, to get attention from customers. An agent that genuinely digs customers. An agent that genuinely digs customers. An agent that genuinely digs in and improves that sequence has an in and improves that sequence has an in and improves that sequence has an immediate economic business case that immediate economic business case that immediate economic business case that any owner can understand right away. But any owner can understand right away. But any owner can understand right away. But don't grade the agent just on leads don't grade the agent just on leads don't grade the agent just on leads scraped or messages sent as so many scraped or messages sent as so many scraped or messages sent as so many people tend to do in that situation. Use people tend to do in that situation. Use people tend to do in that situation. Use the measures the business already uses. the measures the business already uses. the measures the business already uses. Use speed to lead, time to a booked Use speed to lead, time to a booked Use speed to lead, time to a booked meeting, conversion to a real meeting, conversion to a real meeting, conversion to a real opportunity, deal size, customer opportunity, deal size, customer opportunity, deal size, customer acquisition cost, revenue in the door, acquisition cost, revenue in the door, acquisition cost, revenue in the door, pipeline. If the agent creates pipeline. If the agent creates pipeline. If the agent creates extraordinary content and distributes it extraordinary content and distributes it extraordinary content and distributes it across 10 channels, ask whether across 10 channels, ask whether across 10 channels, ask whether qualified people entered the pipeline qualified people entered the pipeline qualified people entered the pipeline because of that content in those because of that content in those because of that content in those channels. The thing I will call out here channels. The thing I will call out here channels. The thing I will call out here is that agents define done through the is that agents define done through the is that agents define done through the experts that you have. Either way, what experts that you have. Either way, what experts that you have. Either way, what you're paying for is whether the agent you're paying for is whether the agent you're paying for is whether the agent can get meaningful work done. Now let's can get meaningful work done. Now let's can get meaningful work done. Now let's hop to entrepreneurs, where I think hop to entrepreneurs, where I think hop to entrepreneurs, where I think agents look most like that vision of agents look most like that vision of agents look most like that vision of mech suits that we had for agents for a mech suits that we had for agents for a mech suits that we had for agents for a while. An entrepreneur has to be a while. An entrepreneur has to be a while. An entrepreneur has to be a generalist, right? They We move between generalist, right? They We move between generalist, right? They We move between product and customers and finance and product and customers and finance and product and customers and finance and hiring and marketing and operations and hiring and marketing and operations and hiring and marketing and operations and tech because the company doesn't care tech because the company doesn't care tech because the company doesn't care that those are different professions, it

that those are different professions, it that those are different professions, it just needs what it needs to grow. The just needs what it needs to grow. The just needs what it needs to grow. The best agent users I know can extend much best agent users I know can extend much best agent users I know can extend much farther because they are not ordinary farther because they are not ordinary farther because they are not ordinary generalists. That's part of what makes generalists. That's part of what makes generalists. That's part of what makes them entrepreneurs. They're deeply them entrepreneurs. They're deeply them entrepreneurs. They're deeply expert in one or two domains, but expert in one or two domains, but expert in one or two domains, but they're good enough across a bunch of they're good enough across a bunch of they're good enough across a bunch of other adjacent domains to recognize when other adjacent domains to recognize when other adjacent domains to recognize when the agent has probably gone wrong. You the agent has probably gone wrong. You the agent has probably gone wrong. You can think of that as an X-shaped person can think of that as an X-shaped person can think of that as an X-shaped person becoming a super X-shaped person with becoming a super X-shaped person with becoming a super X-shaped person with agents. Deep knowledge on one axis, agents. Deep knowledge on one axis, agents. Deep knowledge on one axis, working knowledge across several others, working knowledge across several others, working knowledge across several others, and agents extending that reach across and agents extending that reach across and agents extending that reach across the board. A technical founder who the board. A technical founder who the board. A technical founder who understands the product and the understands the product and the understands the product and the architecture can use an agent to stretch architecture can use an agent to stretch architecture can use an agent to stretch to cover design work and research and to cover design work and research and to cover design work and research and support analysis and parts of support analysis and parts of support analysis and parts of go-to-market because they can trace the go-to-market because they can trace the go-to-market because they can trace the consequences of agent decisions back to consequences of agent decisions back to consequences of agent decisions back to something they know, so they know when something they know, so they know when something they know, so they know when the agent got the job done. That can the agent got the job done. That can the agent got the job done. That can feel like magic. It can also produce a feel like magic. It can also produce a feel like magic. It can also produce a very specific kind of overconfidence if very specific kind of overconfidence if very specific kind of overconfidence if you're not careful, and we're going to you're not careful, and we're going to you're not careful, and we're going to get into that. 80% of five fields is not get into that. 80% of five fields is not get into that. 80% of five fields is not the same as the dangerous 20% in each of the same as the dangerous 20% in each of the same as the dangerous 20% in each of those fields.

those fields. those fields. Taxes are an obvious example. So are Taxes are an obvious example. So are Taxes are an obvious example. So are financial controls, employment law, financial controls, employment law, financial controls, employment law, regulated claims, contracts that can regulated claims, contracts that can regulated claims, contracts that can create exposure. If you don't have create exposure. If you don't have create exposure. If you don't have enough domain expertise to recognize a enough domain expertise to recognize a enough domain expertise to recognize a really plausible-looking but incorrect really plausible-looking but incorrect really plausible-looking but incorrect answer, then configuring a general agent answer, then configuring a general agent answer, then configuring a general agent yourself just means that you're yourself just means that you're yourself just means that you're generating a bunch of liability for generating a bunch of liability for generating a bunch of liability for yourself very, very quickly. In those yourself very, very quickly. In those yourself very, very quickly. In those situations, if you need an agent to situations, if you need an agent to situations, if you need an agent to extend the business, absolutely, you extend the business, absolutely, you extend the business, absolutely, you should be buying that agent the way I should be buying that agent the way I should be buying that agent the way I described for a small medium business. described for a small medium business. described for a small medium business. Buy the agent from someone who has deep Buy the agent from someone who has deep Buy the agent from someone who has deep domain expertise. Make sure that you are domain expertise. Make sure that you are domain expertise. Make sure that you are getting good liability structuring getting good liability structuring getting good liability structuring professional review, whatever you need professional review, whatever you need professional review, whatever you need to do to cover for that dangerous 20% of to do to cover for that dangerous 20% of to do to cover for that dangerous 20% of that situation that you don't know. And that situation that you don't know. And that situation that you don't know. And And I I'm serious about that. Where are And I I'm serious about that. Where are And I I'm serious about that. Where are you genuinely deep? Do you actually you genuinely deep? Do you actually you genuinely deep? Do you actually know? Are you deep in product? Are you know? Are you deep in product? Are you know? Are you deep in product? Are you deep in tech? Are you deep in branding? deep in tech? Are you deep in branding? deep in tech? Are you deep in branding? Or finance, or sales? Which decisions Or finance, or sales? Which decisions Or finance, or sales? Which decisions can you make all the way down to the can you make all the way down to the can you make all the way down to the ground? And which ones only feel ground? And which ones only feel ground? And which ones only feel familiar because you've heard the familiar because you've heard the familiar because you've heard the vocabulary a lot at cocktail parties.

vocabulary a lot at cocktail parties. vocabulary a lot at cocktail parties. And then ask, in that situation, what is And then ask, in that situation, what is And then ask, in that situation, what is the agent bad at in practice that you the agent bad at in practice that you the agent bad at in practice that you manage? Do you know what your agent is manage? Do you know what your agent is manage? Do you know what your agent is bad at? Do you know when your last bad at? Do you know when your last bad at? Do you know when your last consequential failure is for an agent? consequential failure is for an agent? consequential failure is for an agent? Did it miss a customer need? Did it Did it miss a customer need? Did it Did it miss a customer need? Did it invent something technical? Did it send invent something technical? Did it send invent something technical? Did it send the wrong message or produce really the wrong message or produce really the wrong message or produce really sloppy code that nobody could maintain? sloppy code that nobody could maintain? sloppy code that nobody could maintain? Or quietly stop after completing the Or quietly stop after completing the Or quietly stop after completing the first step and you didn't notice? And first step and you didn't notice? And first step and you didn't notice? And when you caught the failure, were you when you caught the failure, were you when you caught the failure, were you actually able to fix it so the agent actually able to fix it so the agent actually able to fix it so the agent could meaningfully extend its work next could meaningfully extend its work next could meaningfully extend its work next time? Or did you not have the technical time? Or did you not have the technical time? Or did you not have the technical expertise to fix? Finally, I challenge expertise to fix? Finally, I challenge expertise to fix? Finally, I challenge entrepreneurs to perform an unplug test. entrepreneurs to perform an unplug test. entrepreneurs to perform an unplug test. If you remove this agent tomorrow, If you remove this agent tomorrow, If you remove this agent tomorrow, what would stop happening? Would what would stop happening? Would what would stop happening? Would anything meaningful stop? Or are you anything meaningful stop? Or are you anything meaningful stop? Or are you performing work with this agent? Perhaps performing work with this agent? Perhaps performing work with this agent? Perhaps you do have work, right? Maybe support you do have work, right? Maybe support you do have work, right? Maybe support tickets would stop being triaged. Maybe tickets would stop being triaged. Maybe tickets would stop being triaged. Maybe your warm leads would wait two days your warm leads would wait two days your warm leads would wait two days instead of three hours for a reply. instead of three hours for a reply. instead of three hours for a reply. Maybe your weekly release cadence would Maybe your weekly release cadence would Maybe your weekly release cadence would slow down or customers would start to slow down or customers would start to slow down or customers would start to notice because the quality and speed at notice because the quality and speed at notice because the quality and speed at which you're able to develop the product which you're able to develop the product which you're able to develop the product would slow down. But, in some cases, would slow down. But, in some cases, would slow down. But, in some cases, what I find is that the only thing that what I find is that the only thing that what I find is that the only thing that disappears is a a bunch of process.

disappears is a a bunch of process. disappears is a a bunch of process. Agents have to do meaningful business Agents have to do meaningful business Agents have to do meaningful business work. You should care if they get work. You should care if they get work. You should care if they get unplugged or they're not doing their unplugged or they're not doing their unplugged or they're not doing their work. Now, what can we learn across all work. Now, what can we learn across all work. Now, what can we learn across all all three scales that I talked about? all three scales that I talked about? all three scales that I talked about? Number one, verifiable work still has an Number one, verifiable work still has an Number one, verifiable work still has an enormous advantage because it's easier enormous advantage because it's easier enormous advantage because it's easier to get real business work done, right? I to get real business work done, right? I to get real business work done, right? I talked about the issues with code talked about the issues with code talked about the issues with code extensively in this video, but code can extensively in this video, but code can extensively in this video, but code can still be tested. Code can still be still be tested. Code can still be still be tested. Code can still be checked and code can still be written in checked and code can still be written in checked and code can still be written in ways that you can forcibly prove the ways that you can forcibly prove the ways that you can forcibly prove the agent did a good job. agent did a good job. agent did a good job. When an agent does a good job on When an agent does a good job on When an agent does a good job on outreach and pipeline builds, you outreach and pipeline builds, you outreach and pipeline builds, you ultimately see that in dollars reaching ultimately see that in dollars reaching ultimately see that in dollars reaching the bank. When a customer responds the bank. When a customer responds the bank. When a customer responds faster because you were able to pick up faster because you were able to pick up faster because you were able to pick up that ticket with an agent faster and you that ticket with an agent faster and you that ticket with an agent faster and you have higher customer satisfaction, have higher customer satisfaction, have higher customer satisfaction, that's something that you can measure. that's something that you can measure. that's something that you can measure. So, So, So, an enterprise can afford to spend a lot an enterprise can afford to spend a lot an enterprise can afford to spend a lot of money turning a lot of their of money turning a lot of their of money turning a lot of their ambiguous work into stuff that that is ambiguous work into stuff that that is ambiguous work into stuff that that is that verifiable, into examples, into that verifiable, into examples, into that verifiable, into examples, into eval sets, into review standards. And eval sets, into review standards. And eval sets, into review standards. And many are. An SMB is going to depend very many are. An SMB is going to depend very many are. An SMB is going to depend very heavily on existing measures and domain heavily on existing measures and domain heavily on existing measures and domain expertise and they're going to depend on expertise and they're going to depend on expertise and they're going to depend on that specifically, typically around code that specifically, typically around code that specifically, typically around code and something in the sales pipeline and something in the sales pipeline and something in the sales pipeline revenue line area. And then they're revenue line area. And then they're revenue line area. And then they're going to buy everything else if they buy going to buy everything else if they buy going to buy everything else if they buy agents. has an even harder task. An agents. has an even harder task. An agents. has an even harder task. An entrepreneur has to know where their entrepreneur has to know where their entrepreneur has to know where their personal expertise is deep and where it personal expertise is deep and where it personal expertise is deep and where it ends. And so if I had to compress this ends. And so if I had to compress this ends. And so if I had to compress this whole video into four questions that I whole video into four questions that I whole video into four questions that I would ask at each of these scales, these would ask at each of these scales, these would ask at each of these scales, these are the four. And these are four are the four. And these are four are the four. And these are four questions I actually sit down and ask

questions I actually sit down and ask questions I actually sit down and ask founders, right? They're questions I founders, right? They're questions I founders, right? They're questions I actually sit down and ask leaders. actually sit down and ask leaders. actually sit down and ask leaders. Question one, can an ordinary competent Question one, can an ordinary competent Question one, can an ordinary competent person inspect this agent's work and person inspect this agent's work and person inspect this agent's work and explain why it works, why it's explain why it works, why it's explain why it works, why it's acceptable, how they can build on it. acceptable, how they can build on it. acceptable, how they can build on it. Don't use your best engineers or the one Don't use your best engineers or the one Don't use your best engineers or the one founder who understands everything. Pick founder who understands everything. Pick founder who understands everything. Pick the person who's kind of a little bit the person who's kind of a little bit the person who's kind of a little bit average, who will actually have to live average, who will actually have to live average, who will actually have to live with the result. Question two, with the result. Question two, with the result. Question two, can you trace the work to the measures can you trace the work to the measures can you trace the work to the measures the business already uses? the business already uses? the business already uses? That might be speed to lead or deal size That might be speed to lead or deal size That might be speed to lead or deal size or revenue or customer resolution or or revenue or customer resolution or or revenue or customer resolution or shipping speed or defect rate or or code shipping speed or defect rate or or code shipping speed or defect rate or or code complexity. But whatever it is, if the complexity. But whatever it is, if the complexity. But whatever it is, if the agent's dashboard is improving while agent's dashboard is improving while agent's dashboard is improving while ordinary measures are flat, believe the ordinary measures are flat, believe the ordinary measures are flat, believe the ordinary measures. You got to push the ordinary measures. You got to push the ordinary measures. You got to push the agent to actually hit the things you agent to actually hit the things you agent to actually hit the things you care about. Question three, do you know care about. Question three, do you know care about. Question three, do you know both your own domain boundary and the both your own domain boundary and the both your own domain boundary and the agent's last important failure? If you agent's last important failure? If you agent's last important failure? If you can't name what you're qualified to can't name what you're qualified to can't name what you're qualified to judge and where the agent broke most judge and where the agent broke most judge and where the agent broke most recently, I don't think you have enough recently, I don't think you have enough recently, I don't think you have enough information to say that you're running information to say that you're running information to say that you're running your agents. I just don't. Question your agents. I just don't. Question your agents. I just don't. Question four, if the work is sitting four, if the work is sitting four, if the work is sitting outside of your area of expertise and it outside of your area of expertise and it outside of your area of expertise and it really carries liability, what is really carries liability, what is really carries liability, what is stopping you from getting a domain stopping you from getting a domain stopping you from getting a domain specific agent? What is stopping you specific agent? What is stopping you specific agent? What is stopping you from getting a managed service instead from getting a managed service instead from getting a managed service instead of trying to do it yourself? Because the of trying to do it yourself? Because the of trying to do it yourself? Because the more expensive a hidden error becomes, more expensive a hidden error becomes, more expensive a hidden error becomes, it's just not worth it to risk the

it's just not worth it to risk the it's just not worth it to risk the business on it. It's just not. Now, I business on it. It's just not. Now, I business on it. It's just not. Now, I have built a much longer checklist to go have built a much longer checklist to go have built a much longer checklist to go with this video. It includes the code with this video. It includes the code with this video. It includes the code audits I talked about. It includes audits I talked about. It includes audits I talked about. It includes enterprise evaluation questions. It enterprise evaluation questions. It enterprise evaluation questions. It includes an entrepreneur self-audit, includes an entrepreneur self-audit, includes an entrepreneur self-audit, buy versus configure decision. buy versus configure decision. buy versus configure decision. I won't read the entire thing here I won't read the entire thing here I won't read the entire thing here because it becomes really, really long. because it becomes really, really long. because it becomes really, really long. But if you want to go and get it and But if you want to go and get it and But if you want to go and get it and actually look at how you can get your actually look at how you can get your actually look at how you can get your agents to get work done, it's over on agents to get work done, it's over on agents to get work done, it's over on the Substack and I'll put that link the Substack and I'll put that link the Substack and I'll put that link there. Now, if we go back to the top, there. Now, if we go back to the top, there. Now, if we go back to the top, the hugging face incident is an extreme the hugging face incident is an extreme the hugging face incident is an extreme story, but I want you as you read it to story, but I want you as you read it to story, but I want you as you read it to think about where that story leads us. think about where that story leads us. think about where that story leads us. Agents can and will learn to pursue Agents can and will learn to pursue Agents can and will learn to pursue their passing evaluation scores with their passing evaluation scores with their passing evaluation scores with astonishing capability and persistence. astonishing capability and persistence. astonishing capability and persistence. A company doesn't need more process from A company doesn't need more process from A company doesn't need more process from that kind of experience in its systems. that kind of experience in its systems. that kind of experience in its systems. Instead, a company needs agents that can Instead, a company needs agents that can Instead, a company needs agents that can actually get work done and we're going actually get work done and we're going actually get work done and we're going to have to figure out how to use the to have to figure out how to use the to have to figure out how to use the score obsession that agents have to put score obsession that agents have to put score obsession that agents have to put in place passing conditions that in place passing conditions that in place passing conditions that represent real value. Good code, a real represent real value. Good code, a real represent real value. Good code, a real customer, revenue that we collected, a customer, revenue that we collected, a customer, revenue that we collected, a good decision. If you can't supply that good decision. If you can't supply that good decision. If you can't supply that to an agent, you don't really have good to an agent, you don't really have good to an agent, you don't really have good agent management practices. You may have agent management practices. You may have agent management practices. You may have a ton of agents, but what you won't have a ton of agents, but what you won't have a ton of agents, but what you won't have is good value and work getting done. And is good value and work getting done. And is good value and work getting done. And that, that is what we should all be that, that is what we should all be that, that is what we should all be caring about and and what I want to see caring about and and what I want to see caring about and and what I want to see in a year is a whole lot less in a year is a whole lot less in a year is a whole lot less differentiation from startups around oh, differentiation from startups around oh, differentiation from startups around oh, our agents get work done. That should

our agents get work done. That should our agents get work done. That should just be the baseline. I want a world just be the baseline. I want a world just be the baseline. I want a world where the baseline is agents getting where the baseline is agents getting where the baseline is agents getting work done, and that's why I made this work done, and that's why I made this work done, and that's why I made this video.