The Last Human Code Review: Building Trust in AI-Generated Code — Itamar Friedman, Qodo

>> Are you all set up with your AI factory? >> Are you all set up with your AI factory? Everything is smooth. Everything is smooth. Everything is smooth. As you're sitting here, your code is As you're sitting here, your code is As you're sitting here, your code is being deployed, errors growing, right? being deployed, errors growing, right? being deployed, errors growing, right? If it's like that, raise your hand. If it's like that, raise your hand. If it's like that, raise your hand. No? Oh, okay. Two people, great. No? Oh, okay. Two people, great. No? Oh, okay. Two people, great. Um Um Um Let us know like before and after this Let us know like before and after this Let us know like before and after this talk what they are and and talk what they are and and talk what they are and and do you feel like there's a bottleneck, a do you feel like there's a bottleneck, a do you feel like there's a bottleneck, a new bottleneck that is not on writing new bottleneck that is not on writing new bottleneck that is not on writing code, rather somewhere else else in the code, rather somewhere else else in the code, rather somewhere else else in the STLC? Is that your biggest thing to STLC? Is that your biggest thing to STLC? Is that your biggest thing to tackle if if you are tackling that right tackle if if you are tackling that right tackle if if you are tackling that right now? now? now? Okay, so you're in the right place. And Okay, so you're in the right place. And Okay, so you're in the right place. And what about like what about like what about like code review, verifying that the code code review, verifying that the code code review, verifying that the code work according to your intent, according work according to your intent, according work according to your intent, according to your architecture, standards, best to your architecture, standards, best to your architecture, standards, best practices, etc. Is this something that practices, etc. Is this something that practices, etc. Is this something that you're tackling day-to-day or you're tackling day-to-day or you're tackling day-to-day or week-by-week? week-by-week? week-by-week? Okay, so you're you're in the right Okay, so you're you're in the right Okay, so you're you're in the right place. So, I'm Adam Ouriel Friedman, the place. So, I'm Adam Ouriel Friedman, the place. So, I'm Adam Ouriel Friedman, the CEO and co-founder of Codo. Um I don't CEO and co-founder of Codo. Um I don't CEO and co-founder of Codo. Um I don't share it too much, but Codo stands for share it too much, but Codo stands for share it too much, but Codo stands for quality of development optimization. Our quality of development optimization. Our quality of development optimization. Our mission and and uh is to help you all mission and and uh is to help you all mission and and uh is to help you all have a code governance code review have a code governance code review have a code governance code review platform that understand your code base, platform that understand your code base, platform that understand your code base, your tribal knowledge, your best your tribal knowledge, your best your tribal knowledge, your best practices, and that does not come off practices, and that does not come off practices, and that does not come off the shelf from a model. It requires a the shelf from a model. It requires a the shelf from a model. It requires a system that optimize for you as we go, system that optimize for you as we go, system that optimize for you as we go, and that's uh why we decided to name our and that's uh why we decided to name our and that's uh why we decided to name our company uh this way.

company uh this way. company uh this way. Uh so, uh I'm going to talk about the Uh so, uh I'm going to talk about the Uh so, uh I'm going to talk about the last human code review, and what do we last human code review, and what do we last human code review, and what do we need to do in order to get there? Okay? need to do in order to get there? Okay? need to do in order to get there? Okay? And um can you switch instead of seeing And um can you switch instead of seeing And um can you switch instead of seeing myself seeing the presentation here? myself seeing the presentation here? myself seeing the presentation here? Thank you. Um so, I I think like first Thank you. Um so, I I think like first Thank you. Um so, I I think like first of all, we need to agree of all, we need to agree of all, we need to agree on on on why do we have code review at all? why do we have code review at all? why do we have code review at all? So, I think basically these are two So, I think basically these are two So, I think basically these are two buckets that we should agree on, and I buckets that we should agree on, and I buckets that we should agree on, and I think I think it's quite common. One is think I think it's quite common. One is think I think it's quite common. One is we want to validate the code that is in we want to validate the code that is in we want to validate the code that is in high quality, safe, maintainable, the high quality, safe, maintainable, the high quality, safe, maintainable, the right architecture, according to our right architecture, according to our right architecture, according to our best practice, etc. The second re- best practice, etc. The second re- best practice, etc. The second re- reason is actually alignment and reason is actually alignment and reason is actually alignment and learning. Right? Like where senior learning. Right? Like where senior learning. Right? Like where senior developers, for example, has developers, for example, has developers, for example, has one last chance, a gateway, a gatekeeper one last chance, a gateway, a gatekeeper one last chance, a gateway, a gatekeeper before before before uh it's being uh code is being pushed to uh it's being uh code is being pushed to uh it's being uh code is being pushed to production to have that alignment and production to have that alignment and production to have that alignment and teaching, right? So, these are the two teaching, right? So, these are the two teaching, right? So, these are the two reasons, and then that's what you need reasons, and then that's what you need reasons, and then that's what you need to think if you're trying to automate to think if you're trying to automate to think if you're trying to automate the code review, can the code review, can the code review, can human still do that? And this if this is human still do that? And this if this is human still do that? And this if this is the right place, the the code review the right place, the the code review the right place, the the code review process, the pull request process, is process, the pull request process, is process, the pull request process, is that the right place to still doing that the right place to still doing that the right place to still doing these two things?

these two things? these two things? If you have If you have If you have uh new tools, new processes that will uh new tools, new processes that will uh new tools, new processes that will help you unblock this bottleneck, but help you unblock this bottleneck, but help you unblock this bottleneck, but let you still do these two buckets of let you still do these two buckets of let you still do these two buckets of tasks, then you're on the right path. tasks, then you're on the right path. tasks, then you're on the right path. Uh so, we're here to ask yes, no, is Uh so, we're here to ask yes, no, is Uh so, we're here to ask yes, no, is human code review human code review human code review still optional end of 2026? still optional end of 2026? still optional end of 2026? Uh is it becoming optional or is it Uh is it becoming optional or is it Uh is it becoming optional or is it still mandatory? still mandatory? still mandatory? Okay, that's that's what we're here to Okay, that's that's what we're here to Okay, that's that's what we're here to answer. Do you think like answer. Do you think like answer. Do you think like every PR or the majority of PRs are every PR or the majority of PRs are every PR or the majority of PRs are going to be reviewed line by line or, going to be reviewed line by line or, going to be reviewed line by line or, you know, bucket by bucket in the in the you know, bucket by bucket in the in the you know, bucket by bucket in the in the chunks and the hunks by our developers, chunks and the hunks by our developers, chunks and the hunks by our developers, raise your hand? Or do you think Do you raise your hand? Or do you think Do you raise your hand? Or do you think Do you think by end of this year your think by end of this year your think by end of this year your developers are still going to review developers are still going to review developers are still going to review diff by diff? diff by diff? diff by diff? Okay. So, try to think why why is that Okay. So, try to think why why is that Okay. So, try to think why why is that happening? Now, I I I wanted to share happening? Now, I I I wanted to share happening? Now, I I I wanted to share with you that yesterday night we did a with you that yesterday night we did a with you that yesterday night we did a drone show, and I had to uh opportunity drone show, and I had to uh opportunity drone show, and I had to uh opportunity to talk to different people during that to talk to different people during that to talk to different people during that drone show, different uh engineering drone show, different uh engineering drone show, different uh engineering leaders.

leaders. leaders. And what I can tell you is that I saw And what I can tell you is that I saw And what I can tell you is that I saw two very two very two very different groups of school of thoughts, different groups of school of thoughts, different groups of school of thoughts, okay? They both agree that bugs okay? They both agree that bugs okay? They both agree that bugs are coming in different shapes. For are coming in different shapes. For are coming in different shapes. For some, this is fine. some, this is fine. some, this is fine. And we're just going to fix that quickly And we're just going to fix that quickly And we're just going to fix that quickly after it hits the production and for after it hits the production and for after it hits the production and for some not. So so actually we do see two some not. So so actually we do see two some not. So so actually we do see two teams. Those that are thinking about the teams. Those that are thinking about the teams. Those that are thinking about the room is split into two. Those are room is split into two. Those are room is split into two. Those are thinking like, "Hey, we have to thinking like, "Hey, we have to thinking like, "Hey, we have to let make sure that every piece of line let make sure that every piece of line let make sure that every piece of line is is trusted is is trusted is is trusted and and the human must review that. Or and and the human must review that. Or and and the human must review that. Or the other group the other group the other group somewhat somewhat somewhat reckless or so saying, "Let's let's like reckless or so saying, "Let's let's like reckless or so saying, "Let's let's like push those bugs into production and push those bugs into production and push those bugs into production and we quickly fix that and that's how we we quickly fix that and that's how we we quickly fix that and that's how we actually do things because it's much actually do things because it's much actually do things because it's much faster. Velocity is more important than faster. Velocity is more important than faster. Velocity is more important than getting getting things right. And I getting getting things right. And I getting getting things right. And I think like you need to think like where think like you need to think like where think like you need to think like where do you sit? Of course I put it in two do you sit? Of course I put it in two do you sit? Of course I put it in two two sides of the spectrum but there's two sides of the spectrum but there's two sides of the spectrum but there's some somewhere in in between and you you some somewhere in in between and you you some somewhere in in between and you you need to think what what's your need to think what what's your need to think what what's your philosophy because that will lead you to philosophy because that will lead you to philosophy because that will lead you to different milestones or different tools different milestones or different tools different milestones or different tools that you need to use in order to get that you need to use in order to get that you need to use in order to get that that that confidence that you can skip over a confidence that you can skip over a confidence that you can skip over a human review in the pull request in the human review in the pull request in the human review in the pull request in the code review.

code review. code review. So So So let's start talking about what is the let's start talking about what is the let's start talking about what is the process? How do we need to start process? How do we need to start process? How do we need to start thinking about it? So I claim that thinking about it? So I claim that thinking about it? So I claim that models are not not a barrier anymore. models are not not a barrier anymore. models are not not a barrier anymore. Like it's not a matter of of the you Like it's not a matter of of the you Like it's not a matter of of the you know a model know a model know a model doing a good good good job or not, doing a good good good job or not, doing a good good good job or not, having the right reasoning or not. having the right reasoning or not. having the right reasoning or not. Basically, the models are improving but Basically, the models are improving but Basically, the models are improving but I'm telling you I just came from one of I'm telling you I just came from one of I'm telling you I just came from one of the leading labs where we are inspecting the leading labs where we are inspecting the leading labs where we are inspecting how how how benchmarks for code review did not benchmarks for code review did not benchmarks for code review did not change a lot throughout the latest change a lot throughout the latest change a lot throughout the latest model. model. model. The The key here is actually context. The The key here is actually context. The The key here is actually context. Okay? Like the models, if you give them Okay? Like the models, if you give them Okay? Like the models, if you give them the right context, and what is that the right context, and what is that the right context, and what is that right context we're going to talk about, right context we're going to talk about, right context we're going to talk about, they could already reason pretty well they could already reason pretty well they could already reason pretty well over what over what over what is the issues that we need What are the is the issues that we need What are the is the issues that we need What are the issues that we need to surface for a issues that we need to surface for a issues that we need to surface for a certain change in the code? certain change in the code? certain change in the code? Otherwise, if you don't have the Otherwise, if you don't have the Otherwise, if you don't have the context, even the best model out there, context, even the best model out there, context, even the best model out there, they will give you different types of of they will give you different types of of they will give you different types of of uh bugs and issues. Some of them are uh bugs and issues. Some of them are uh bugs and issues. Some of them are really good, but in many cases they will really good, but in many cases they will really good, but in many cases they will simply tell you, "Hey, did you consider simply tell you, "Hey, did you consider simply tell you, "Hey, did you consider error handling uh error handling uh error handling uh uh like or not?" By the way, error uh like or not?" By the way, error uh like or not?" By the way, error handling could be like a really good handling could be like a really good handling could be like a really good thing to handle. Depends. In some cases thing to handle. Depends. In some cases thing to handle. Depends. In some cases it's critical, in some cases are not.

it's critical, in some cases are not. it's critical, in some cases are not. And again, the context is what what And again, the context is what what And again, the context is what what matters. Right now, our context is like matters. Right now, our context is like matters. Right now, our context is like spread all across. Like we have spread all across. Like we have spread all across. Like we have agents.mds, cloud.mds, skills.mds. And agents.mds, cloud.mds, skills.mds. And agents.mds, cloud.mds, skills.mds. And And the thing is that each each one of And the thing is that each each one of And the thing is that each each one of them like has like different standards, them like has like different standards, them like has like different standards, uh different or organization and sub uh different or organization and sub uh different or organization and sub organization are dealing with different organization are dealing with different organization are dealing with different differently. Even within a certain team, differently. Even within a certain team, differently. Even within a certain team, you might be using that that you might be using that that you might be using that that differently. And you're actually maybe differently. And you're actually maybe differently. And you're actually maybe using like the same uh one one team is using like the same uh one one team is using like the same uh one one team is using uh the same agent to do code using uh the same agent to do code using uh the same agent to do code review and sorry, coding and code review and sorry, coding and code review and sorry, coding and code review. The other might be using review. The other might be using review. The other might be using something else. And all of that does not something else. And all of that does not something else. And all of that does not bring you the trust and consistency that bring you the trust and consistency that bring you the trust and consistency that you're looking uh towards. Um By the you're looking uh towards. Um By the you're looking uh towards. Um By the way, like you also might have like way, like you also might have like way, like you also might have like coding agents that are running in your coding agents that are running in your coding agents that are running in your IDE, but you're probably building the AI IDE, but you're probably building the AI IDE, but you're probably building the AI factory that running those agents like factory that running those agents like factory that running those agents like in workflows to automate some of the in workflows to automate some of the in workflows to automate some of the coding. I see teams that are already uh coding. I see teams that are already uh coding. I see teams that are already uh having more lines of code being uh having more lines of code being uh having more lines of code being uh shipped that are not generated from the shipped that are not generated from the shipped that are not generated from the CLI or or the IDE. So, how do you CLI or or the IDE. So, how do you CLI or or the IDE. So, how do you control all that? Like that's that's control all that? Like that's that's control all that? Like that's that's missing uh like in right now like in our missing uh like in right now like in our missing uh like in right now like in our in our tools in our infrastructure.

in our tools in our infrastructure. in our tools in our infrastructure. And let alone if you add those MCPs and And let alone if you add those MCPs and And let alone if you add those MCPs and and and rag like style context. I don't and and rag like style context. I don't and and rag like style context. I don't know if you have like great visibility. know if you have like great visibility. know if you have like great visibility. There are ways to tackle that. There's There are ways to tackle that. There's There are ways to tackle that. There's great talks out there. Check how you can great talks out there. Check how you can great talks out there. Check how you can have like MCP versioning and have data have like MCP versioning and have data have like MCP versioning and have data sets for every like a benchmark for sets for every like a benchmark for sets for every like a benchmark for every MCP change, but that's hard to every MCP change, but that's hard to every MCP change, but that's hard to manage. We're missing like a governance manage. We're missing like a governance manage. We're missing like a governance layer for us to move to the next level. layer for us to move to the next level. layer for us to move to the next level. Okay, like where we can actually trust Okay, like where we can actually trust Okay, like where we can actually trust the code without human reviewing it. the code without human reviewing it. the code without human reviewing it. So, where where is that context? So, where where is that context? So, where where is that context? Where is that context? Basically, I Where is that context? Basically, I Where is that context? Basically, I would say experience, tribal knowledge, would say experience, tribal knowledge, would say experience, tribal knowledge, wisdom of your developers is a lot in wisdom of your developers is a lot in wisdom of your developers is a lot in their heads. They are in some documents. their heads. They are in some documents. their heads. They are in some documents. There are documents, infrastructure There are documents, infrastructure There are documents, infrastructure documents, documents, documents, but a lot By the way, a lot of them is but a lot By the way, a lot of them is but a lot By the way, a lot of them is our slacks or or teams or or so. our slacks or or teams or or so. our slacks or or teams or or so. The data is there, but a lot of the The data is there, but a lot of the The data is there, but a lot of the information are are in your developer information are are in your developer information are are in your developer heads. And we need like to to codify heads. And we need like to to codify heads. And we need like to to codify them.

them. them. Now, um I think like basically another thing I think like basically another thing that I'm saying like when I'm saying that I'm saying like when I'm saying that I'm saying like when I'm saying that we need to codify human knowledge, that we need to codify human knowledge, that we need to codify human knowledge, what we're actually saying is that we're what we're actually saying is that we're what we're actually saying is that we're trying to build an interface for agents trying to build an interface for agents trying to build an interface for agents an interface for humans to collaborate an interface for humans to collaborate an interface for humans to collaborate each other on that on that knowledge. each other on that on that knowledge. each other on that on that knowledge. And that's a very important And that's a very important And that's a very important a very important point. When you want to a very important point. When you want to a very important point. When you want to extract that information, the tribal extract that information, the tribal extract that information, the tribal knowledge from your knowledge from your knowledge from your like senior developers, etc., and codify like senior developers, etc., and codify like senior developers, etc., and codify that, do you codify that only in agents' that, do you codify that only in agents' that, do you codify that only in agents' language? Which is very maybe verbose language? Which is very maybe verbose language? Which is very maybe verbose and structured, or you want to codify and structured, or you want to codify and structured, or you want to codify that in a wiki style that in a wiki style that in a wiki style get started and all that what developers get started and all that what developers get started and all that what developers love love doing. And the answer is that love love doing. And the answer is that love love doing. And the answer is that you probably need to build your context you probably need to build your context you probably need to build your context lake, your context engine, as I lake, your context engine, as I lake, your context engine, as I mentioned, that's the the the the real mentioned, that's the the the the real mentioned, that's the the the the real like gold mine here to to get the code like gold mine here to to get the code like gold mine here to to get the code review like auto automated. You have to review like auto automated. You have to review like auto automated. You have to have it fitting for both. So, what have it fitting for both. So, what have it fitting for both. So, what you're seeing here, for example, is that you're seeing here, for example, is that you're seeing here, for example, is that Codo as an example, but you can use Codo as an example, but you can use Codo as an example, but you can use other tools, help you collect all the other tools, help you collect all the other tools, help you collect all the rules and standards that your team rules and standards that your team rules and standards that your team own, that your team is using day-to-day.

own, that your team is using day-to-day. own, that your team is using day-to-day. And then, it will provide that And then, it will provide that And then, it will provide that information during the review for information during the review for information during the review for humans. Hey, notice that Codo used four humans. Hey, notice that Codo used four humans. Hey, notice that Codo used four rules. rules. rules. Uh sorry, uh used uh uh many rules and Uh sorry, uh used uh uh many rules and Uh sorry, uh used uh uh many rules and four are violated, and that includes a four are violated, and that includes a four are violated, and that includes a link to all the rules that are were link to all the rules that are were link to all the rules that are were being used. That's for human in order to being used. That's for human in order to being used. That's for human in order to trust. Okay, in order to trust the trust. Okay, in order to trust the trust. Okay, in order to trust the results that coming from your code results that coming from your code results that coming from your code review tool, etc., you have to build review tool, etc., you have to build review tool, etc., you have to build that interface for for human. You have that interface for for human. You have that interface for for human. You have to accumulate that knowledge and have an to accumulate that knowledge and have an to accumulate that knowledge and have an interface for human. But, you also want interface for human. But, you also want interface for human. But, you also want to have an interface that is dedicated to have an interface that is dedicated to have an interface that is dedicated for agents. What you're seeing here, uh for agents. What you're seeing here, uh for agents. What you're seeing here, uh for example, is a com- comment, for for example, is a com- comment, for for example, is a com- comment, for example, by Codo that is speaking to example, by Codo that is speaking to example, by Codo that is speaking to another agent. Hey, dear agent, Codo another agent. Hey, dear agent, Codo another agent. Hey, dear agent, Codo just reviewed this uh PR and has found just reviewed this uh PR and has found just reviewed this uh PR and has found five different issues. Codo already five different issues. Codo already five different issues. Codo already spend uh uh like uh some back background spend uh uh like uh some back background spend uh uh like uh some back background task and use cloud code, for example, task and use cloud code, for example, task and use cloud code, for example, harness in order to do fixes, and there harness in order to do fixes, and there harness in order to do fixes, and there is a closed PR, like you can see here in is a closed PR, like you can see here in is a closed PR, like you can see here in the top right, there is some closed PR the top right, there is some closed PR the top right, there is some closed PR with all the fixes, and now, when an with all the fixes, and now, when an with all the fixes, and now, when an agent is coming to review this PR once agent is coming to review this PR once agent is coming to review this PR once again, then it has like a cherry-picking again, then it has like a cherry-picking again, then it has like a cherry-picking moment uh with everything that all the moment uh with everything that all the moment uh with everything that all the code that is actually is passing your code that is actually is passing your code that is actually is passing your rules, your standard, uh and everything rules, your standard, uh and everything rules, your standard, uh and everything that we're like more architectural that we're like more architectural that we're like more architectural decision that we're going to talk about.

decision that we're going to talk about. decision that we're going to talk about. So, So, So, what what I said so far is that what what I said so far is that what what I said so far is that just to to just to to just to to like recap so far, if you want to trust like recap so far, if you want to trust like recap so far, if you want to trust and you want to get to a point where and you want to get to a point where and you want to get to a point where you're trusting the code that's being you're trusting the code that's being you're trusting the code that's being shipped, you need to have the right shipped, you need to have the right shipped, you need to have the right context that is being gathered and being context that is being gathered and being context that is being gathered and being used during the code review process. used during the code review process. used during the code review process. Then, Then, Then, you will see that this code review you will see that this code review you will see that this code review process have links and information for process have links and information for process have links and information for human and have links and information for human and have links and information for human and have links and information for for agents, okay? And when that is in for agents, okay? And when that is in for agents, okay? And when that is in place, you will see that developers are place, you will see that developers are place, you will see that developers are writing less and less comments in the writing less and less comments in the writing less and less comments in the pull request. And then after 100 of pull request. And then after 100 of pull request. And then after 100 of these pull requests, there's human no these pull requests, there's human no these pull requests, there's human no more human review, you know that you're more human review, you know that you're more human review, you know that you're ready for for automation. Okay? Now, ready for for automation. Okay? Now, ready for for automation. Okay? Now, back to back to the context. back to back to the context. back to back to the context. So far, I talked about relatively simple So far, I talked about relatively simple So far, I talked about relatively simple context. I talked about context. I talked about context. I talked about uh rules and standards and skills, but uh rules and standards and skills, but uh rules and standards and skills, but actual the human knowledge that the actual the human knowledge that the actual the human knowledge that the tribal knowledge in your organization tribal knowledge in your organization tribal knowledge in your organization sits in understanding the system sits in understanding the system sits in understanding the system architecture.

architecture. architecture. What are the P0s, the the bugs that What are the P0s, the the bugs that What are the P0s, the the bugs that actually made an outage outage for for actually made an outage outage for for actually made an outage outage for for you, unfortunately, in the last like 3 you, unfortunately, in the last like 3 you, unfortunately, in the last like 3 months or so? When a microservice uh one months or so? When a microservice uh one months or so? When a microservice uh one changed its contract and broke a changed its contract and broke a changed its contract and broke a microservice two, right? microservice two, right? microservice two, right? That is does not exist in most code That is does not exist in most code That is does not exist in most code review and and like if you try to build review and and like if you try to build review and and like if you try to build yourself, it's really hard to build, but yourself, it's really hard to build, but yourself, it's really hard to build, but it is available in some of the those it is available in some of the those it is available in some of the those code review uh solution that is code review uh solution that is code review uh solution that is dedicated for that. For example, what dedicated for that. For example, what dedicated for that. For example, what you're seeing here is the graph being you're seeing here is the graph being you're seeing here is the graph being built for a certain microservice and all built for a certain microservice and all built for a certain microservice and all the repos and and their connection. And the repos and and their connection. And the repos and and their connection. And and each node and edge, there is what is and each node and edge, there is what is and each node and edge, there is what is the con- if it's an edge, what is the the con- if it's an edge, what is the the con- if it's an edge, what is the contract between two uh piece of your contract between two uh piece of your contract between two uh piece of your software, but also links to history of software, but also links to history of software, but also links to history of discussions between developers that they discussions between developers that they discussions between developers that they had when they fixed an issue because a had when they fixed an issue because a had when they fixed an issue because a root cause analysis and now you need to root cause analysis and now you need to root cause analysis and now you need to codify that. codify that. codify that. Okay? Okay? Okay? And when you get to to that level of a And when you get to to that level of a And when you get to to that level of a context engine, context engine, context engine, now you're ready now you're ready now you're ready to start approving and blocking PRs to start approving and blocking PRs to start approving and blocking PRs automatically.

automatically. automatically. And you want to do that not just by And you want to do that not just by And you want to do that not just by letting AI some like choose by yourself, letting AI some like choose by yourself, letting AI some like choose by yourself, rather giving some semantic rather giving some semantic rather giving some semantic rules rules rules that that for example, when you when do that that for example, when you when do that that for example, when you when do you guys approve or or or block a PR? you guys approve or or or block a PR? you guys approve or or or block a PR? And that knowledge also needs to be And that knowledge also needs to be And that knowledge also needs to be accumulated as part of your your accumulated as part of your your accumulated as part of your your context. context. context. What I'm actually saying is that What I'm actually saying is that What I'm actually saying is that software development, at least code software development, at least code software development, at least code governance, governance, governance, is going to change from reviewing your is going to change from reviewing your is going to change from reviewing your pull request to actually reviewing your pull request to actually reviewing your pull request to actually reviewing your entire software development from a graph entire software development from a graph entire software development from a graph abstraction, abstraction, abstraction, where you're seeing your PRs as bubbles where you're seeing your PRs as bubbles where you're seeing your PRs as bubbles with all the issues that might happen with all the issues that might happen with all the issues that might happen even with three different PRs are in on even with three different PRs are in on even with three different PRs are in on the fly, when they which which contract the fly, when they which which contract the fly, when they which which contract they might bridge they they might ruin. they might bridge they they might ruin. they might bridge they they might ruin. Okay? Okay? Okay? And that's how the software development And that's how the software development And that's how the software development future is going to look like.

future is going to look like. future is going to look like. What you need to do in order to get What you need to do in order to get What you need to do in order to get there is to codify your standards. It there is to codify your standards. It there is to codify your standards. It needs to be built in a way that humans needs to be built in a way that humans needs to be built in a way that humans can trust and audit and control. You can trust and audit and control. You can trust and audit and control. You need to build real-time self-learning need to build real-time self-learning need to build real-time self-learning context, learning from PR history, context, learning from PR history, context, learning from PR history, learning from accepted and unaccepted learning from accepted and unaccepted learning from accepted and unaccepted learning from discussions between learning from discussions between learning from discussions between between developers, learning between between developers, learning between between developers, learning between learning like from those cases that learning like from those cases that learning like from those cases that broke your production. And that context broke your production. And that context broke your production. And that context needs to be needs to be needs to be not just like thrown into files. It not just like thrown into files. It not just like thrown into files. It needs to to to to sit and locate it in a needs to to to to sit and locate it in a needs to to to to sit and locate it in a place that agent understand where is place that agent understand where is place that agent understand where is that context fitting. that context fitting. that context fitting. And then you need the governance And then you need the governance And then you need the governance infrastructure that gives you that infrastructure that gives you that infrastructure that gives you that visibility of what's happening. Rather visibility of what's happening. Rather visibility of what's happening. Rather the graph I presented, but there's more the graph I presented, but there's more the graph I presented, but there's more to that. to that. to that. Okay, I'm going to show you very soon Okay, I'm going to show you very soon Okay, I'm going to show you very soon another visualization that helps you another visualization that helps you another visualization that helps you understand the overall status of your understand the overall status of your understand the overall status of your PRs and software. PRs and software. PRs and software. If you're already shipping AI-generated If you're already shipping AI-generated If you're already shipping AI-generated code faster than your human can review, code faster than your human can review, code faster than your human can review, I'm actually saying that you are in the I'm actually saying that you are in the I'm actually saying that you are in the problem. You're not like ahead of the problem. You're not like ahead of the problem. You're not like ahead of the problem.

problem. problem. You you if you You you if you You you if you put the infrastructure, if you gather put the infrastructure, if you gather put the infrastructure, if you gather the context, if you start accumulating the context, if you start accumulating the context, if you start accumulating how code review can be automated for how code review can be automated for how code review can be automated for you, then that's where you're going to you, then that's where you're going to you, then that's where you're going to get the 10x velocity that you're being get the 10x velocity that you're being get the 10x velocity that you're being promising your CEO or yourself or your promising your CEO or yourself or your promising your CEO or yourself or your or your developers, because otherwise or your developers, because otherwise or your developers, because otherwise it's a bottleneck. it's a bottleneck. it's a bottleneck. You need to own You need to own You need to own your rules and standards and codify them your rules and standards and codify them your rules and standards and codify them and get analytics and statistics about and get analytics and statistics about and get analytics and statistics about each one of them. How many times they're each one of them. How many times they're each one of them. How many times they're being caught, which rules and standards being caught, which rules and standards being caught, which rules and standards and skill is actually being used during and skill is actually being used during and skill is actually being used during the review process, and it is useful or the review process, and it is useful or the review process, and it is useful or not, or does it need to get an update. not, or does it need to get an update. not, or does it need to get an update. You need to have visual full You need to have visual full You need to have visual full visualization of your software graph. visualization of your software graph. visualization of your software graph. What is the connection? What are the What is the connection? What are the What is the connection? What are the contracts? What is working? What is not contracts? What is working? What is not contracts? What is working? What is not working? What are two PRs are going to working? What are two PRs are going to working? What are two PRs are going to crash very soon because they're touching crash very soon because they're touching crash very soon because they're touching the same agent and they don't and they the same agent and they don't and they the same agent and they don't and they don't know. don't know. don't know. And you need to start learning how to And you need to start learning how to And you need to start learning how to auto approve and auto block. That won't auto approve and auto block. That won't auto approve and auto block. That won't happen in like immediately. That needs happen in like immediately. That needs happen in like immediately. That needs to gradually to gradually to gradually being automated for you step by step by being automated for you step by step by being automated for you step by step by adding more adding more adding more rules for blocking and more rules for rules for blocking and more rules for rules for blocking and more rules for for for approving over time.

So, trustworthy automated review, So, trustworthy automated review, your rules, your standards, in your your rules, your standards, in your your rules, your standards, in your software graph software graph software graph placed in the right edge and the right placed in the right edge and the right placed in the right edge and the right node of your software. node of your software. node of your software. Learning from the tribal knowledge of Learning from the tribal knowledge of Learning from the tribal knowledge of discussions, of history of your discussions, of history of your discussions, of history of your software, and codifying that. software, and codifying that. software, and codifying that. There's so much tribal knowledge what to There's so much tribal knowledge what to There's so much tribal knowledge what to do and not to do, and that needs to be do and not to do, and that needs to be do and not to do, and that needs to be codified as well, both for the agents codified as well, both for the agents codified as well, both for the agents and for the humans. and for the humans. and for the humans. And basically, you need to accumulate And basically, you need to accumulate And basically, you need to accumulate that experience and codify that. that experience and codify that. that experience and codify that. And sorry for the buzz or whatever you And sorry for the buzz or whatever you And sorry for the buzz or whatever you want to call it, that's moving from want to call it, that's moving from want to call it, that's moving from artificial intelligence to artificial artificial intelligence to artificial artificial intelligence to artificial wisdom. wisdom. wisdom. Because right now, Because right now, Because right now, you know better than I that your you know better than I that your you know better than I that your developer holds the judgement of what's developer holds the judgement of what's developer holds the judgement of what's bad and what's good. It's not your your bad and what's good. It's not your your bad and what's good. It's not your your software not your AI tools. If you want software not your AI tools. If you want software not your AI tools. If you want to get to a point where judgement is to get to a point where judgement is to get to a point where judgement is moving to your moving to your moving to your AI tools, that's AI wisdom where that AI tools, that's AI wisdom where that AI tools, that's AI wisdom where that experience need to be codified in the experience need to be codified in the experience need to be codified in the right way, the right place for agents right way, the right place for agents right way, the right place for agents and humans.

and humans. and humans. And that's where we are here, Codota, to And that's where we are here, Codota, to And that's where we are here, Codota, to help you. And our help you. And our help you. And our vision and mission is not far away from vision and mission is not far away from vision and mission is not far away from now. We want to reach in 2027 to a place now. We want to reach in 2027 to a place now. We want to reach in 2027 to a place where you have zero outages, zero bugs where you have zero outages, zero bugs where you have zero outages, zero bugs in production, at least the critical and in production, at least the critical and in production, at least the critical and the high ones. That's why we're here. the high ones. That's why we're here. the high ones. That's why we're here. Thank you so much for having me.