AI Agents for Performance: Ship Faster, Pay Less — Rajat Shah, Netflix

Hi there. Hi there. Welcome to AI Engineer World's Fair 2026 Welcome to AI Engineer World's Fair 2026 Welcome to AI Engineer World's Fair 2026 event. I am Rajat Shah. I'm a staff event. I am Rajat Shah. I'm a staff event. I am Rajat Shah. I'm a staff software engineer at Netflix where I software engineer at Netflix where I software engineer at Netflix where I work in the AI platform organization work in the AI platform organization work in the AI platform organization building large-scale distributed systems building large-scale distributed systems building large-scale distributed systems for machine learning model hosting. for machine learning model hosting. for machine learning model hosting. In this talk, I'm here to um In this talk, I'm here to um In this talk, I'm here to um share how we did uh improve our share how we did uh improve our share how we did uh improve our performance engineering performance engineering performance engineering uh throughput by introducing AI agents uh throughput by introducing AI agents uh throughput by introducing AI agents into the mix. And this is more of a uh into the mix. And this is more of a uh into the mix. And this is more of a uh playbook or a practitioner's guide uh to playbook or a practitioner's guide uh to playbook or a practitioner's guide uh to help you also replicate similar help you also replicate similar help you also replicate similar learnings in your own organizations uh learnings in your own organizations uh learnings in your own organizations uh to improve uh the infrastructure cost to improve uh the infrastructure cost to improve uh the infrastructure cost and uh ship faster. Let's first talk about the problem. Why Let's first talk about the problem. Why does performance engineering doesn't does performance engineering doesn't does performance engineering doesn't scale and uh what does it cost to scale and uh what does it cost to scale and uh what does it cost to actually do it right? actually do it right? actually do it right? The problem is arising arising from the The problem is arising arising from the The problem is arising arising from the fact is that you are authoring code now fact is that you are authoring code now fact is that you are authoring code now at a 10x faster speed. The at a 10x faster speed. The at a 10x faster speed. The uh coding agents are getting better and uh coding agents are getting better and uh coding agents are getting better and better uh at uh solving problems. And um better uh at uh solving problems. And um better uh at uh solving problems. And um as more and more uh engineers uh adopt as more and more uh engineers uh adopt as more and more uh engineers uh adopt it, it it gets very easy to produce code it, it it gets very easy to produce code it, it it gets very easy to produce code in your system. And this is slight in your system. And this is slight in your system. And this is slight exaggeration, but the compute cost also exaggeration, but the compute cost also exaggeration, but the compute cost also uh is increasing at a similar pace uh uh is increasing at a similar pace uh uh is increasing at a similar pace uh because uh it doesn't always because uh it doesn't always because uh it doesn't always uh write the fastest code.

uh write the fastest code. uh write the fastest code. Uh so, this is where the problem arises Uh so, this is where the problem arises Uh so, this is where the problem arises because of that new wide coding era. Uh because of that new wide coding era. Uh because of that new wide coding era. Uh the AI agent ships code. Uh it is the AI agent ships code. Uh it is the AI agent ships code. Uh it is pretty much tuned to ship code fast. Um pretty much tuned to ship code fast. Um pretty much tuned to ship code fast. Um and of course, you could say that uh as and of course, you could say that uh as and of course, you could say that uh as the uh the uh the uh the coding agents are evolving, newer the coding agents are evolving, newer the coding agents are evolving, newer models are coming into play. Uh they get models are coming into play. Uh they get models are coming into play. Uh they get better and better at uh simply writing better and better at uh simply writing better and better at uh simply writing performant code. But, that's not always performant code. But, that's not always performant code. But, that's not always true. Your um true. Your um true. Your um agent doesn't know specific details agent doesn't know specific details agent doesn't know specific details about your platforms and your frameworks about your platforms and your frameworks about your platforms and your frameworks uh and your internal code base patterns. uh and your internal code base patterns. uh and your internal code base patterns. So, it it tends to just produce code So, it it tends to just produce code So, it it tends to just produce code based on what it might have already based on what it might have already based on what it might have already seen other code bases using or inventing seen other code bases using or inventing seen other code bases using or inventing new new new patterns in your code bases that you did patterns in your code bases that you did patterns in your code bases that you did not anticipate not anticipate not anticipate an engineer to use an engineer to use an engineer to use as a pattern to use your framework. So, let's look at what a performance So, let's look at what a performance engineer typically does. engineer typically does. engineer typically does. I'm calling this as a human performance I'm calling this as a human performance I'm calling this as a human performance engineer, which is responsible for engineer, which is responsible for engineer, which is responsible for identifying bottlenecks in a service and identifying bottlenecks in a service and identifying bottlenecks in a service and fixing them.

Typically, a a human would trigger Typically, a a human would trigger profiling on a single production profiling on a single production profiling on a single production instance of a fleet of production instance of a fleet of production instance of a fleet of production instances. instances. instances. You would go and download it, You would go and download it, You would go and download it, potentially open it in a visualizer. The potentially open it in a visualizer. The potentially open it in a visualizer. The raw data that you download typically is raw data that you download typically is raw data that you download typically is not great to look at. It It could be, not great to look at. It It could be, not great to look at. It It could be, for example, a JSON structure data of for example, a JSON structure data of for example, a JSON structure data of the call stack and where the CPU is the call stack and where the CPU is the call stack and where the CPU is spent. So, using a visualizer spent. So, using a visualizer spent. So, using a visualizer helps you at least see and visualize the helps you at least see and visualize the helps you at least see and visualize the call stack and CPU call stack and CPU call stack and CPU time of various um time of various um time of various um method in your services better. method in your services better. method in your services better. Once you have that visualizer open, you Once you have that visualizer open, you Once you have that visualizer open, you pretty much end up spending a lot of pretty much end up spending a lot of pretty much end up spending a lot of time in uh like just looking at and and time in uh like just looking at and and time in uh like just looking at and and finding in this in this treasure finding in this in this treasure finding in this in this treasure hunt on hunt on hunt on on the path on the potential places on the path on the potential places on the path on the potential places where you could where you could where you could improve the code to improve the code to improve the code to to make it more performant. This takes a to make it more performant. This takes a to make it more performant. This takes a lot of time in order to even learn how lot of time in order to even learn how lot of time in order to even learn how to look at it. Um to look at it. Um to look at it. Um And there's a learning curve to And there's a learning curve to And there's a learning curve to curve curve to it. And this is where the curve curve to it. And this is where the curve curve to it. And this is where the real bottleneck ends up being. You end real bottleneck ends up being. You end real bottleneck ends up being. You end up you end up having to spend straight up you end up having to spend straight up you end up having to spend straight many, many minutes to many, many minutes to many, many minutes to uh to identify the bottlenecks.

uh to identify the bottlenecks. uh to identify the bottlenecks. Once you've identified some code paths Once you've identified some code paths Once you've identified some code paths and some packages that and some packages that and some packages that that are spending significant CPU that are spending significant CPU that are spending significant CPU cycles, you you would end up searching cycles, you you would end up searching cycles, you you would end up searching it in your code bases in your code repos it in your code bases in your code repos it in your code bases in your code repos and see if it and see if it and see if it has a potential to improvement. has a potential to improvement. has a potential to improvement. Hopefully, you you hit Hopefully, you you hit Hopefully, you you hit you have luckier and you find a you have luckier and you find a you have luckier and you find a root cause and you produce a code review root cause and you produce a code review root cause and you produce a code review out. out. out. You merge it and You merge it and You merge it and you get some performance wins. you get some performance wins. you get some performance wins. And then you repeat all of this again. And then you repeat all of this again. And then you repeat all of this again. You see the problem, right? This is You see the problem, right? This is You see the problem, right? This is a very manual effort and very tedious a very manual effort and very tedious a very manual effort and very tedious effort to get right and this ends up effort to get right and this ends up effort to get right and this ends up being a bottleneck if you were to do it being a bottleneck if you were to do it being a bottleneck if you were to do it across across across many of your code bases and code paths many of your code bases and code paths many of your code bases and code paths and that's why this is done very rarely. and that's why this is done very rarely. and that's why this is done very rarely. People typically end up looking at People typically end up looking at People typically end up looking at profiling data only when something is profiling data only when something is profiling data only when something is going wrong at 2:00 a.m. and going wrong at 2:00 a.m. and going wrong at 2:00 a.m. and somebody needs to fix a problem because somebody needs to fix a problem because somebody needs to fix a problem because your CPU is your CPU is your CPU is unbearable. unbearable. unbearable. So, we asked this question internally. So, we asked this question internally. So, we asked this question internally. Can we Can we Can we Can an LLM read this profiling data? The Can an LLM read this profiling data? The Can an LLM read this profiling data? The 20 minutes that I mentioned an engineer 20 minutes that I mentioned an engineer 20 minutes that I mentioned an engineer spends in identifying hot paths, can an spends in identifying hot paths, can an spends in identifying hot paths, can an LLM agent which is fed that data also do LLM agent which is fed that data also do LLM agent which is fed that data also do it much faster?

it much faster? it much faster? And we tried to answer this And we tried to answer this And we tried to answer this question through some live services. So, question through some live services. So, question through some live services. So, the next couple of slides will be about the next couple of slides will be about the next couple of slides will be about this experiment and how we do it, right? this experiment and how we do it, right? this experiment and how we do it, right? Before we get to it, the the foundation Before we get to it, the the foundation Before we get to it, the the foundation of of of of our assumption of our assumption of our assumption is that every profiler essentially is that every profiler essentially is that every profiler essentially speaks the same language. Even though speaks the same language. Even though speaks the same language. Even though you would profile and you could have you would profile and you could have you would profile and you could have your services written in Java, Python, your services written in Java, Python, your services written in Java, Python, Go, etc. The profiler's example that I Go, etc. The profiler's example that I Go, etc. The profiler's example that I mentioned here mentioned here mentioned here do the same same thing. They would run do the same same thing. They would run do the same same thing. They would run it it it in your in your in your they would run against your production they would run against your production they would run against your production service on a single instance and they service on a single instance and they service on a single instance and they would essentially try to capture the would essentially try to capture the would essentially try to capture the call stack, the self CPU and inclusive call stack, the self CPU and inclusive call stack, the self CPU and inclusive CPU that is spent on CPU that is spent on CPU that is spent on on each of the on each of the on each of the methods in your call stack, Uh sample methods in your call stack, Uh sample methods in your call stack, Uh sample data very high frequency. So, data very high frequency. So, data very high frequency. So, irrespective of which language, which irrespective of which language, which irrespective of which language, which runtime you're using in production, the runtime you're using in production, the runtime you're using in production, the output of a profiling data is actually output of a profiling data is actually output of a profiling data is actually very similar and very well-structured very similar and very well-structured very similar and very well-structured for an LLM agent to use. This is one for an LLM agent to use. This is one for an LLM agent to use. This is one assumption that we had going into the assumption that we had going into the assumption that we had going into the experiment.

experiment. experiment. The second assumption we had going into The second assumption we had going into The second assumption we had going into the experiment is that there are many the experiment is that there are many the experiment is that there are many common patterns that common patterns that common patterns that the coding agents have learned through the coding agents have learned through the coding agents have learned through their training sites across all the their training sites across all the their training sites across all the publicly available code sources that publicly available code sources that publicly available code sources that they might have had. So, things like O they might have had. So, things like O they might have had. So, things like O of N squared loops, loop invariants, of N squared loops, loop invariants, of N squared loops, loop invariants, things that you're computing every time things that you're computing every time things that you're computing every time in your loop which could be computed in your loop which could be computed in your loop which could be computed just once outside it. If you have just once outside it. If you have just once outside it. If you have certain places where you are doing the certain places where you are doing the certain places where you are doing the object allocation object allocation object allocation very repeatedly and if you could move it very repeatedly and if you could move it very repeatedly and if you could move it out as well. Um out as well. Um out as well. Um Or there are times when you would be Or there are times when you would be Or there are times when you would be like having some contention and like having some contention and like having some contention and places where you could optimize through places where you could optimize through places where you could optimize through better batching. All of these are very better batching. All of these are very better batching. All of these are very common patterns that an agent already common patterns that an agent already common patterns that an agent already knows. Uh it uh knows. Uh it uh knows. Uh it uh I mean, assuming it is trained on very I mean, assuming it is trained on very I mean, assuming it is trained on very good quality software code. good quality software code. good quality software code. The better the quality, the better agent The better the quality, the better agent The better the quality, the better agent knows knows knows which patterns are which patterns are which patterns are not great. Um and this is a very good not great. Um and this is a very good not great. Um and this is a very good point for an LLM agent because it point for an LLM agent because it point for an LLM agent because it essentially means that it can essentially means that it can essentially means that it can technically technically technically look at the look at the look at the the code base and it can identify those the code base and it can identify those the code base and it can identify those patterns very easily. So, this is more patterns very easily. So, this is more patterns very easily. So, this is more about like pattern recognition, like about like pattern recognition, like about like pattern recognition, like finding out these code patterns to finding out these code patterns to finding out these code patterns to find some code smells that that are find some code smells that that are find some code smells that that are worth fixing.

worth fixing. worth fixing. So, when you have that profiling output So, when you have that profiling output So, when you have that profiling output that I mentioned and you have coding that I mentioned and you have coding that I mentioned and you have coding agent that could understand it, you feed agent that could understand it, you feed agent that could understand it, you feed it into it. Um it into it. Um it into it. Um The profiling data that it needs to read The profiling data that it needs to read The profiling data that it needs to read is actually very well structured in a is actually very well structured in a is actually very well structured in a way that it could see that the in this way that it could see that the in this way that it could see that the in this case uh the case uh the case uh the um the profiling data is uh in in the um the profiling data is uh in in the um the profiling data is uh in in the case of row one and row three, they are case of row one and row three, they are case of row one and row three, they are the same call path and the immutable map the same call path and the immutable map the same call path and the immutable map copy of that I have in my row three is copy of that I have in my row three is copy of that I have in my row three is actually actually actually uh used inside uh a tensor merge method uh used inside uh a tensor merge method uh used inside uh a tensor merge method that we have. Uh so um don't think too that we have. Uh so um don't think too that we have. Uh so um don't think too much about the method names here, but much about the method names here, but much about the method names here, but the key point that I wanted to call out the key point that I wanted to call out the key point that I wanted to call out is that it was able to use these is that it was able to use these is that it was able to use these functions. functions. functions. Uh Uh Uh it knows the meaning of these functions it knows the meaning of these functions it knows the meaning of these functions and when it sees it is being used in a a and when it sees it is being used in a a and when it sees it is being used in a a poor way, it can actually identify that poor way, it can actually identify that poor way, it can actually identify that this is a quadratic uh this is a quadratic uh this is a quadratic uh um um um algorithm and not a linear algorithm.

algorithm and not a linear algorithm. algorithm and not a linear algorithm. So, this is where uh So, this is where uh So, this is where uh our first aha moment was that okay, if our first aha moment was that okay, if our first aha moment was that okay, if you feed a profiling data, it if it you feed a profiling data, it if it you feed a profiling data, it if it knows the pattern, it assuming the knows the pattern, it assuming the knows the pattern, it assuming the coding agent was trained on a very good coding agent was trained on a very good coding agent was trained on a very good quality data, it can actually identify quality data, it can actually identify quality data, it can actually identify and up and find out that uh find in your and up and find out that uh find in your and up and find out that uh find in your code base where that bad uh patterns are code base where that bad uh patterns are code base where that bad uh patterns are actually uh running. And this is not by actually uh running. And this is not by actually uh running. And this is not by looking at the code base. This is purely looking at the code base. This is purely looking at the code base. This is purely by looking at the call stack that the by looking at the call stack that the by looking at the call stack that the profiling uh profiling uh profiling uh uh data produced. uh data produced. uh data produced. And once And once And once uh uh uh a prof a an AI agent gets this, what a prof a an AI agent gets this, what a prof a an AI agent gets this, what what are the steps that it needs to do what are the steps that it needs to do what are the steps that it needs to do in order to get to the potential fix? in order to get to the potential fix? in order to get to the potential fix? So, we we we have mentioned that it can So, we we we have mentioned that it can So, we we we have mentioned that it can identify uh identify uh identify uh a pattern and say that okay, this is a a pattern and say that okay, this is a a pattern and say that okay, this is a potential problem to be fixed. What what potential problem to be fixed. What what potential problem to be fixed. What what happens next? happens next? happens next? Um Um Um it could knowing where that method it could knowing where that method it could knowing where that method lives, it could do a code search and lives, it could do a code search and lives, it could do a code search and first of all, before uh first of all, before uh first of all, before uh I mean, yeah, it could do a code search I mean, yeah, it could do a code search I mean, yeah, it could do a code search and find out the code repo where this and find out the code repo where this and find out the code repo where this method is defined uh or this code is method is defined uh or this code is method is defined uh or this code is defined.

defined. defined. And it it needs to extract out the exact And it it needs to extract out the exact And it it needs to extract out the exact commit that is currently running in commit that is currently running in commit that is currently running in production, which is typically easy to production, which is typically easy to production, which is typically easy to get. You know which build is running in get. You know which build is running in get. You know which build is running in your production, and you could point your production, and you could point your production, and you could point that to the LLM to uh check out the Git that to the LLM to uh check out the Git that to the LLM to uh check out the Git repo at that same commit. repo at that same commit. repo at that same commit. And then uh once it has uh find found And then uh once it has uh find found And then uh once it has uh find found out uh out uh out uh and has the Git repo cloned, uh it can and has the Git repo cloned, uh it can and has the Git repo cloned, uh it can look for uh that exact code path, uh look for uh that exact code path, uh look for uh that exact code path, uh skip any internal library details, and skip any internal library details, and skip any internal library details, and find out uh the code patterns. And now find out uh the code patterns. And now find out uh the code patterns. And now that it now once it has the uh full uh that it now once it has the uh full uh that it now once it has the uh full uh uh code repo cloned out, the um the uh code repo cloned out, the um the uh code repo cloned out, the um the methods uh definition and implementation methods uh definition and implementation methods uh definition and implementation identified, it can trace the entire call identified, it can trace the entire call identified, it can trace the entire call call path of that method. call path of that method. call path of that method. And this And this And this uh structured data uh structured data uh structured data once it uh has this information, becomes once it uh has this information, becomes once it uh has this information, becomes uh like very powerful, and we'll we are uh like very powerful, and we'll we are uh like very powerful, and we'll we are going to talk about how uh it leverages going to talk about how uh it leverages going to talk about how uh it leverages this. this. this. So, So, So, these were our understanding going into these were our understanding going into these were our understanding going into the experiment. We uh the experiment. We uh the experiment. We uh we knew it could find we knew it could find we knew it could find uh the patterns. We tested it with with uh the patterns. We tested it with with uh the patterns. We tested it with with a couple of uh of our services to see if a couple of uh of our services to see if a couple of uh of our services to see if it can actually not just find the it can actually not just find the it can actually not just find the pattern, but also take it one step pattern, but also take it one step pattern, but also take it one step further and uh produce a real fix and further and uh produce a real fix and further and uh produce a real fix and provide a real fix that uh an engineer provide a real fix that uh an engineer provide a real fix that uh an engineer could approve and merge.

could approve and merge. could approve and merge. So, first finding uh the the O of N So, first finding uh the the O of N So, first finding uh the the O of N squared um squared um squared um uh example that I just mentioned, it uh uh example that I just mentioned, it uh uh example that I just mentioned, it uh once it knew once it knew once it knew uh that this is a problem, and looking uh that this is a problem, and looking uh that this is a problem, and looking at the profile profiling data, it knows at the profile profiling data, it knows at the profile profiling data, it knows that this is consuming 8.8% of the CPU that this is consuming 8.8% of the CPU that this is consuming 8.8% of the CPU time during that uh time during that uh time during that uh uh period of profiling. uh period of profiling. uh period of profiling. So, for us, uh the coding agent not just So, for us, uh the coding agent not just So, for us, uh the coding agent not just ended up finding the problem, given ended up finding the problem, given ended up finding the problem, given enough instructions as a form of let's enough instructions as a form of let's enough instructions as a form of let's say skill or prompt, uh you could say skill or prompt, uh you could say skill or prompt, uh you could actually have it uh do those four steps actually have it uh do those four steps actually have it uh do those four steps that I just mentioned, which is checking that I just mentioned, which is checking that I just mentioned, which is checking out the Git repo, out the Git repo, out the Git repo, uh finding where that code is uh finding where that code is uh finding where that code is implemented, and actually uh sending a implemented, and actually uh sending a implemented, and actually uh sending a code review out. code review out. code review out. All of it could be done uh in a very All of it could be done uh in a very All of it could be done uh in a very large code base with powerful enough large code base with powerful enough large code base with powerful enough code agents in less than 5 minutes. Um code agents in less than 5 minutes. Um code agents in less than 5 minutes. Um and in this case, uh we also, which I'll and in this case, uh we also, which I'll and in this case, uh we also, which I'll talk about a little bit in the later talk about a little bit in the later talk about a little bit in the later slides, uh were able to identify what is slides, uh were able to identify what is slides, uh were able to identify what is the savings uh if you were to uh the savings uh if you were to uh the savings uh if you were to uh reimplement it with a uh with an reimplement it with a uh with an reimplement it with a uh with an optimized implementation in terms of optimized implementation in terms of optimized implementation in terms of both CPU as well as latency savings. Uh both CPU as well as latency savings. Uh both CPU as well as latency savings. Uh I I've shown the numbers here of our I I've shown the numbers here of our I I've shown the numbers here of our real production uh savings that we were real production uh savings that we were real production uh savings that we were able to observe uh through this uh able to observe uh through this uh able to observe uh through this uh exercise.

exercise. exercise. So, this was like a first uh proof for So, this was like a first uh proof for So, this was like a first uh proof for us that you could actually indeed us that you could actually indeed us that you could actually indeed introduce an an AI agent into the mix introduce an an AI agent into the mix introduce an an AI agent into the mix and get uh and get uh and get uh your productivity wins of not just your productivity wins of not just your productivity wins of not just identifying, but also identifying, but also identifying, but also going all the way to a code review. going all the way to a code review. going all the way to a code review. Uh the second uh Uh the second uh Uh the second uh great benefit is that once you have this great benefit is that once you have this great benefit is that once you have this well set up, um well set up, um well set up, um the uh other example that we are able to the uh other example that we are able to the uh other example that we are able to take it forward to it is, let's say it take it forward to it is, let's say it take it forward to it is, let's say it identifies the uh pattern in one of the identifies the uh pattern in one of the identifies the uh pattern in one of the services that we profile, um and services that we profile, um and services that we profile, um and the learning that we had is that it the learning that we had is that it the learning that we had is that it could actually uh could actually uh could actually uh look at all your potential places where look at all your potential places where look at all your potential places where that same bad pattern uh is being that same bad pattern uh is being that same bad pattern uh is being reused. So, in the previous example, it reused. So, in the previous example, it reused. So, in the previous example, it it just identified one bad it just identified one bad it just identified one bad implementation. You fix it, you get all implementation. You fix it, you get all implementation. You fix it, you get all the benefits. In this case, it the benefits. In this case, it the benefits. In this case, it identified not just a bad identified not just a bad identified not just a bad code, but because that pattern, in this code, but because that pattern, in this code, but because that pattern, in this case, which is a counter object uh for case, which is a counter object uh for case, which is a counter object uh for our uh spectator metrics, our uh spectator metrics, our uh spectator metrics, uh that are uh being created on every uh that are uh being created on every uh that are uh being created on every single uh uh single uh uh single uh uh iteration of a hot path in our service iteration of a hot path in our service iteration of a hot path in our service stack, um stack, um stack, um it could essentially see that this is uh it could essentially see that this is uh it could essentially see that this is uh a real problem and it could a real problem and it could a real problem and it could double down and search multiple services double down and search multiple services double down and search multiple services where that same pattern is being where that same pattern is being where that same pattern is being repeated and repeated and repeated and uh uh uh scale up the effort to fix it in scale up the effort to fix it in scale up the effort to fix it in multiple services. In this case, we multiple services. In this case, we multiple services. In this case, we found that same bad pattern were

found that same bad pattern were found that same bad pattern were actually implemented in seven different actually implemented in seven different actually implemented in seven different services uh services uh services uh through cross repo code searches. And if through cross repo code searches. And if through cross repo code searches. And if fixed, it could actually uh if fixed fixed, it could actually uh if fixed fixed, it could actually uh if fixed across all of those different code across all of those different code across all of those different code repos, it could have savings between 0.5 repos, it could have savings between 0.5 repos, it could have savings between 0.5 to 4.6% to 4.6% to 4.6% of CPU cycles. of CPU cycles. of CPU cycles. Um Um Um so so so we have discussed the problem, our we have discussed the problem, our we have discussed the problem, our hypothesis, and the experiment which hypothesis, and the experiment which hypothesis, and the experiment which proved out that a uh LLM coding agent proved out that a uh LLM coding agent proved out that a uh LLM coding agent could actually help improve uh could actually help improve uh could actually help improve uh the uh the finding and fixing of your the uh the finding and fixing of your the uh the finding and fixing of your um um um suboptimal code. Let's see how we can suboptimal code. Let's see how we can suboptimal code. Let's see how we can actually build this into the development actually build this into the development actually build this into the development cycle. Uh it doesn't have to be a very cycle. Uh it doesn't have to be a very cycle. Uh it doesn't have to be a very retrospective retrospective retrospective uh uh uh problem where once a production service problem where once a production service problem where once a production service is uh having issues, you end up doing is uh having issues, you end up doing is uh having issues, you end up doing all of those exercises. I'll talk about all of those exercises. I'll talk about all of those exercises. I'll talk about how you can actually how you can actually how you can actually leverage in your end-to-end software leverage in your end-to-end software leverage in your end-to-end software development cycle development cycle development cycle so that you get the benefits uh much so that you get the benefits uh much so that you get the benefits uh much early and potentially avoid having that early and potentially avoid having that early and potentially avoid having that suboptimal code reach production suboptimal code reach production suboptimal code reach production altogether.

altogether. altogether. Uh the first uh real problem is that Uh the first uh real problem is that Uh the first uh real problem is that LLMs if you invoke them, they do have LLMs if you invoke them, they do have LLMs if you invoke them, they do have some memory these days, but it's very some memory these days, but it's very some memory these days, but it's very compact memory and it will uh it it compact memory and it will uh it it compact memory and it will uh it it won't have all the uh information that won't have all the uh information that won't have all the uh information that uh you are uh uh that you potentially as uh you are uh uh that you potentially as uh you are uh uh that you potentially as a performance engineer know when you're a performance engineer know when you're a performance engineer know when you're trying to debug. So, you you trying to debug. So, you you trying to debug. So, you you when you you if you think of a coding when you you if you think of a coding when you you if you think of a coding agent and if you want to build it at par agent and if you want to build it at par agent and if you want to build it at par uh uh as sufficient as as efficient as a uh uh as sufficient as as efficient as a uh uh as sufficient as as efficient as a human engineer would typically be, you human engineer would typically be, you human engineer would typically be, you want to introduce a long-term memory want to introduce a long-term memory want to introduce a long-term memory that it could reuse. So that's our first that it could reuse. So that's our first that it could reuse. So that's our first real problem to solve for and I'll talk real problem to solve for and I'll talk real problem to solve for and I'll talk about how you solve that. about how you solve that. about how you solve that. The simplest way is to first think of The simplest way is to first think of The simplest way is to first think of you want to build a catalog. you want to build a catalog. you want to build a catalog. Patterns, anti-patterns. Patterns, anti-patterns. Patterns, anti-patterns. Once you have identified enough of them, Once you have identified enough of them, Once you have identified enough of them, you could put it in a catalog that you could put it in a catalog that you could put it in a catalog that that users uh that is used by an LLM that users uh that is used by an LLM that users uh that is used by an LLM agent which can be stateless of its own. agent which can be stateless of its own. agent which can be stateless of its own. And together a stateful catalog and a And together a stateful catalog and a And together a stateful catalog and a state stateless LLM can become a full state stateless LLM can become a full state stateless LLM can become a full fleet-wide memory um fleet-wide memory um fleet-wide memory um for for for for a coding agent to use.

for a coding agent to use. for a coding agent to use. The The The foundation for this and and to foundation for this and and to foundation for this and and to foundation for incorporating in this foundation for incorporating in this foundation for incorporating in this development cycle is practically just development cycle is practically just development cycle is practically just this. You want to keep this very central this. You want to keep this very central this. You want to keep this very central so that it's not very team-specific, not so that it's not very team-specific, not so that it's not very team-specific, not very product-specific, but rather a very product-specific, but rather a very product-specific, but rather a central ever-growing catalog. As the central ever-growing catalog. As the central ever-growing catalog. As the catalog patterns get written, if there catalog patterns get written, if there catalog patterns get written, if there are certain things that could be are certain things that could be are certain things that could be generalized. generalized. generalized. In that example, the first example that In that example, the first example that In that example, the first example that I the first finding that I provided I the first finding that I provided I the first finding that I provided around O of N squared implementation is around O of N squared implementation is around O of N squared implementation is actually very generalizable. All coding actually very generalizable. All coding actually very generalizable. All coding languages will potentially have the same languages will potentially have the same languages will potentially have the same problem if it if the code is written and problem if it if the code is written and problem if it if the code is written and implemented in that way. implemented in that way. implemented in that way. So you you can take it a step further in So you you can take it a step further in So you you can take it a step further in that your catalog can become even more that your catalog can become even more that your catalog can become even more ubiquitous and usable across languages ubiquitous and usable across languages ubiquitous and usable across languages and frameworks as well. and frameworks as well. and frameworks as well. And you And you And you think of it as a blueprint. think of it as a blueprint. think of it as a blueprint. As As As as those as that catalog starts growing, as those as that catalog starts growing, as those as that catalog starts growing, it becomes your building block for it becomes your building block for it becomes your building block for future coding agents to actually use for future coding agents to actually use for future coding agents to actually use for identifying the problem in your identifying the problem in your identifying the problem in your development cycle. And that's the next development cycle. And that's the next development cycle. And that's the next thing I'm going to talk about. The thing I'm going to talk about. The thing I'm going to talk about. The solution that I mentioned so far of a solution that I mentioned so far of a solution that I mentioned so far of a memory is not very fancy vector search memory is not very fancy vector search memory is not very fancy vector search or or a or or a or or a vector database that needs to store all vector database that needs to store all vector database that needs to store all the catalog of patterns and the catalog of patterns and the catalog of patterns and anti-patterns. Rather, you can start anti-patterns. Rather, you can start anti-patterns. Rather, you can start with just a markdown with just a markdown with just a markdown files in a in a centralized Git repo.

files in a in a centralized Git repo. files in a in a centralized Git repo. Um, as the production findings appear, Um, as the production findings appear, Um, as the production findings appear, have the have the have the coding agent put more and more coding agent put more and more coding agent put more and more patterns and anti-patterns into this. patterns and anti-patterns into this. patterns and anti-patterns into this. And the benefit is that even if one And the benefit is that even if one And the benefit is that even if one service finds this, even if one service finds this, even if one service finds this, even if one uh, profiling, even if one service did uh, profiling, even if one service did uh, profiling, even if one service did the profiling to find this pattern, the profiling to find this pattern, the profiling to find this pattern, multiple services that in the future are multiple services that in the future are multiple services that in the future are going to run profiling, the agent that going to run profiling, the agent that going to run profiling, the agent that is doing the profiling of that other is doing the profiling of that other is doing the profiling of that other service can use the pattern catalog from service can use the pattern catalog from service can use the pattern catalog from the first service to the first service to the first service to find the patterns more optimally and not find the patterns more optimally and not find the patterns more optimally and not having to redo all the exercise that the having to redo all the exercise that the having to redo all the exercise that the first first first agent did in order to identify that agent did in order to identify that agent did in order to identify that anti-pattern. Uh, you might think of like how how do I Uh, you might think of like how how do I get started with this? Well, you you get started with this? Well, you you get started with this? Well, you you don't always have to start fresh, but don't always have to start fresh, but don't always have to start fresh, but you could start fresh. There is no harm you could start fresh. There is no harm you could start fresh. There is no harm in having a catalog that is just empty in having a catalog that is just empty in having a catalog that is just empty in the beginning and as more and more in the beginning and as more and more in the beginning and as more and more profiling happens, profiling happens, profiling happens, uh, more uh, more uh, more uh, uh, uh, powerful the powerful the powerful the coding agents that are referring it coding agents that are referring it coding agents that are referring it could become.

could become. could become. But there are few, many in fact, But there are few, many in fact, But there are few, many in fact, sources of how you could sources of how you could sources of how you could public sources of how you could improve public sources of how you could improve public sources of how you could improve your coding agents. If you're into C++ your coding agents. If you're into C++ your coding agents. If you're into C++ optimizations, Jeff Dean had this optimizations, Jeff Dean had this optimizations, Jeff Dean had this wonderful wonderful wonderful blog post around how to look for certain blog post around how to look for certain blog post around how to look for certain optimization opportunities in your C++ optimization opportunities in your C++ optimization opportunities in your C++ code. PyTorch has a torch fix code repo code. PyTorch has a torch fix code repo code. PyTorch has a torch fix code repo where it catalogs several anti-patterns where it catalogs several anti-patterns where it catalogs several anti-patterns that could help optimize kernels and that could help optimize kernels and that could help optimize kernels and the model graph of your PyTorch model. the model graph of your PyTorch model. the model graph of your PyTorch model. And then you might have your own And then you might have your own And then you might have your own performance playbooks as well. performance playbooks as well. performance playbooks as well. Typically, as your organization grows Typically, as your organization grows Typically, as your organization grows and your enterprise software evolves, and your enterprise software evolves, and your enterprise software evolves, you end up noting down certain patterns you end up noting down certain patterns you end up noting down certain patterns and anti-patterns. Those all could and anti-patterns. Those all could and anti-patterns. Those all could become the bootstrapping point for that become the bootstrapping point for that become the bootstrapping point for that uh uh uh catalog Git repo that I mentioned that catalog Git repo that I mentioned that catalog Git repo that I mentioned that can act as a centralized um can act as a centralized um can act as a centralized um place for all uh catalog all patterns place for all uh catalog all patterns place for all uh catalog all patterns and anti-patterns to be noted. and anti-patterns to be noted. and anti-patterns to be noted. And what would an entry in that catalog And what would an entry in that catalog And what would an entry in that catalog look like? Uh look like? Uh look like? Uh the example is right here. Uh as the example is right here. Uh as the example is right here. Uh as the left-hand side it mentions the uh the left-hand side it mentions the uh the left-hand side it mentions the uh small hints that an LLM uh coding agent small hints that an LLM uh coding agent small hints that an LLM uh coding agent in the future could use to easily query in the future could use to easily query in the future could use to easily query that uh catalog. It can mention a list that uh catalog. It can mention a list that uh catalog. It can mention a list of symbols, uh the services where it was of symbols, uh the services where it was of symbols, uh the services where it was confirmed. Some confidence level is confirmed. Some confidence level is confirmed. Some confidence level is great here as well because as more and great here as well because as more and great here as well because as more and more services confirm this, you want to more services confirm this, you want to more services confirm this, you want to keep updating this so that uh the future keep updating this so that uh the future keep updating this so that uh the future agents get more and more confident that, agents get more and more confident that, agents get more and more confident that, "Okay, this is worth a human uh

"Okay, this is worth a human uh "Okay, this is worth a human uh uh uh uh human review, and I would want to more human review, and I would want to more human review, and I would want to more confidently send out a code review to a confidently send out a code review to a confidently send out a code review to a human." human." human." Uh I'm still keeping the confidence uh Uh I'm still keeping the confidence uh Uh I'm still keeping the confidence uh bar to just send a code review and not bar to just send a code review and not bar to just send a code review and not directly uh push it to production. directly uh push it to production. directly uh push it to production. That's by intent. I That's by intent. I That's by intent. I uh I'll talk a little bit on why, but I uh I'll talk a little bit on why, but I uh I'll talk a little bit on why, but I do feel like there is still need for a do feel like there is still need for a do feel like there is still need for a human approval because you're modifying human approval because you're modifying human approval because you're modifying an existing code that is running just an existing code that is running just an existing code that is running just fine in in production fine in in production fine in in production in order to optimize it, which is which in order to optimize it, which is which in order to optimize it, which is which is very risky. If you don't know the is very risky. If you don't know the is very risky. If you don't know the business context, if you don't have business context, if you don't have business context, if you don't have enough test coverage, uh you might end enough test coverage, uh you might end enough test coverage, uh you might end up breaking. So, there is a need for up breaking. So, there is a need for up breaking. So, there is a need for human uh to actually be the one human uh to actually be the one human uh to actually be the one responsible for approving that code responsible for approving that code responsible for approving that code review. review. review. Um and on the right here, I mentioned Um and on the right here, I mentioned Um and on the right here, I mentioned what the entry in that what the entry in that what the entry in that pattern-anti-pattern catalog could look pattern-anti-pattern catalog could look pattern-anti-pattern catalog could look like. Uh like. Uh like. Uh a anti-pattern and a good pattern so a anti-pattern and a good pattern so a anti-pattern and a good pattern so that both the things are well noted that both the things are well noted that both the things are well noted here. here. here. Um so, I mentioned a code review and a Um so, I mentioned a code review and a Um so, I mentioned a code review and a human uh human uh human uh involvement in order to approve and involvement in order to approve and involvement in order to approve and merge that code review is still needed. merge that code review is still needed. merge that code review is still needed. So, how do you optimize for noise there? So, how do you optimize for noise there? So, how do you optimize for noise there? There could be enough signals that the There could be enough signals that the There could be enough signals that the uh AI agent uh could could think of uh uh AI agent uh could could think of uh uh AI agent uh could could think of uh improving and it could improving and it could improving and it could like arbitrarily send code reviews for like arbitrarily send code reviews for like arbitrarily send code reviews for them without knowing enough about your them without knowing enough about your them without knowing enough about your system.

system. system. There are a few tips that I mentioned There are a few tips that I mentioned There are a few tips that I mentioned here could be very powerful in here could be very powerful in here could be very powerful in in making sure that the agent first has in making sure that the agent first has in making sure that the agent first has enough verifications done before looking enough verifications done before looking enough verifications done before looking for an human's attention. The integration test, unit test, the The integration test, unit test, the most basic functional test that your most basic functional test that your most basic functional test that your system needs to have should be run ahead system needs to have should be run ahead system needs to have should be run ahead of time of time of time by the coding agent itself so that it by the coding agent itself so that it by the coding agent itself so that it knows that as it is making code changes knows that as it is making code changes knows that as it is making code changes to optimize it, it is also not to optimize it, it is also not to optimize it, it is also not causing any production business logic to causing any production business logic to causing any production business logic to fail. So, as as you have good coverage fail. So, as as you have good coverage fail. So, as as you have good coverage and unit test, it gets more confidence and unit test, it gets more confidence and unit test, it gets more confidence that it gets more confidence that your that it gets more confidence that your that it gets more confidence that your that the change it is going to propose that the change it is going to propose that the change it is going to propose is actually right. is actually right. is actually right. Once it has made that functional Once it has made that functional Once it has made that functional correctness check, you still correctness check, you still correctness check, you still the our code coverage isn't typically the our code coverage isn't typically the our code coverage isn't typically ideal. There are still surprises in ideal. There are still surprises in ideal. There are still surprises in production production production and canary deployments could be very and canary deployments could be very and canary deployments could be very powerful in this case. If you provide powerful in this case. If you provide powerful in this case. If you provide enough enough enough automation around this, the canary automation around this, the canary automation around this, the canary deployment could be a prerequisite deployment could be a prerequisite deployment could be a prerequisite before the code review before the code review before the code review requires human attention.

requires human attention. requires human attention. What is a canary? You typically have What is a canary? You typically have What is a canary? You typically have you typically have two machines, one you typically have two machines, one you typically have two machines, one containing your old code, another containing your old code, another containing your old code, another containing your new code, which could be containing your new code, which could be containing your new code, which could be the performance fixed code. And you send the performance fixed code. And you send the performance fixed code. And you send the same traffic to both of them over a the same traffic to both of them over a the same traffic to both of them over a period of let's say period of let's say period of let's say 10 minutes and try to compare 10 minutes and try to compare 10 minutes and try to compare the the the CPU usage across between them. CPU usage across between them. CPU usage across between them. The comparison report typically is is The comparison report typically is is The comparison report typically is is what you would want the AI agent to make what you would want the AI agent to make what you would want the AI agent to make decision on and judgment on whether it decision on and judgment on whether it decision on and judgment on whether it should open a code review for it. should open a code review for it. should open a code review for it. In this case, the observability report In this case, the observability report In this case, the observability report could contain could contain could contain standard infra standard infra standard infra things like how how much CPU reduction, things like how how much CPU reduction, things like how how much CPU reduction, how much latency reduced, how much latency reduced, how much latency reduced, is there an increase in error rate? If is there an increase in error rate? If is there an increase in error rate? If if that's the case, it should if that's the case, it should if that's the case, it should see that as a red signal to not proceed see that as a red signal to not proceed see that as a red signal to not proceed because it might have gotten the because it might have gotten the because it might have gotten the business logic incorrect in an attempt business logic incorrect in an attempt business logic incorrect in an attempt to optimize the code. to optimize the code. to optimize the code. And what I mentioned here is that this And what I mentioned here is that this And what I mentioned here is that this is not an AI problem. The observability, is not an AI problem. The observability, is not an AI problem. The observability, canary, verify logic, these are all canary, verify logic, these are all canary, verify logic, these are all standard checks that you need need to standard checks that you need need to standard checks that you need need to have in your system.

have in your system. have in your system. And then final guardrail that you want And then final guardrail that you want And then final guardrail that you want to have is an engineer decision, like I to have is an engineer decision, like I to have is an engineer decision, like I mentioned, still very important, mentioned, still very important, mentioned, still very important, critical critical critical uh uh uh in the mix. in the mix. in the mix. Um Um Um Here's a mental model. Profiler gives Here's a mental model. Profiler gives Here's a mental model. Profiler gives the estimate, canary gives ground truth. the estimate, canary gives ground truth. the estimate, canary gives ground truth. Uh canary is the way you verify and Uh canary is the way you verify and Uh canary is the way you verify and evaluate that your evaluate that your evaluate that your uh suggested fixes actually uh going to uh suggested fixes actually uh going to uh suggested fixes actually uh going to improve and have some uh positive impact improve and have some uh positive impact improve and have some uh positive impact and no negative impact. An engineer and no negative impact. An engineer and no negative impact. An engineer makes the eventual decision. makes the eventual decision. makes the eventual decision. So, So, So, the the the uh path so far has been about how do you uh path so far has been about how do you uh path so far has been about how do you uh uh uh improve the improve the improve the efficiency of getting efficiency of getting efficiency of getting from a profiled from a profiled from a profiled uh uh uh profiling data to producing a code profiling data to producing a code profiling data to producing a code review fix. Uh that's the reactive path review fix. Uh that's the reactive path review fix. Uh that's the reactive path that I just covered so far, uh that I just covered so far, uh that I just covered so far, uh which means you have the code already which means you have the code already which means you have the code already running in production, and now you're running in production, and now you're running in production, and now you're trying to improve it. That's a trying to improve it. That's a trying to improve it. That's a very very very that's typically too late in the game.

that's typically too late in the game. that's typically too late in the game. Now, anytime you try to change a thing Now, anytime you try to change a thing Now, anytime you try to change a thing that is running in production in order that is running in production in order that is running in production in order to improve the performance of it, you to improve the performance of it, you to improve the performance of it, you have to be very, very careful. So, you have to be very, very careful. So, you have to be very, very careful. So, you want want want that to shift left. You want that that to shift left. You want that that to shift left. You want that reactive path to be your initial guide reactive path to be your initial guide reactive path to be your initial guide to build that initial catalog. And as to build that initial catalog. And as to build that initial catalog. And as your pattern catalog grows, you want to your pattern catalog grows, you want to your pattern catalog grows, you want to move closer and closer to the proactive move closer and closer to the proactive move closer and closer to the proactive path where path where path where uh as a human or a coding agent authors uh as a human or a coding agent authors uh as a human or a coding agent authors new code uh the the reviewer agent could new code uh the the reviewer agent could new code uh the the reviewer agent could actually look up that catalog and actually look up that catalog and actually look up that catalog and provide an inline uh comment uh by provide an inline uh comment uh by provide an inline uh comment uh by understanding the code change and ensure understanding the code change and ensure understanding the code change and ensure and providing a suggestion that hey, and providing a suggestion that hey, and providing a suggestion that hey, based on the pattern catalog and the based on the pattern catalog and the based on the pattern catalog and the observed uh profiling data that I have, observed uh profiling data that I have, observed uh profiling data that I have, I feel confident that this uh is an I feel confident that this uh is an I feel confident that this uh is an anti-pattern to introduce. Can you uh anti-pattern to introduce. Can you uh anti-pattern to introduce. Can you uh rewrite this code? That could be a a rewrite this code? That could be a a rewrite this code? That could be a a review comment that a reviewer agent review comment that a reviewer agent review comment that a reviewer agent could be uh providing. could be uh providing. could be uh providing. And if you shift left all the way, like And if you shift left all the way, like And if you shift left all the way, like why even wait until a code review? Uh why even wait until a code review? Uh why even wait until a code review? Uh that that's where you get the most that that's where you get the most that that's where you get the most power.

power. power. Because you know, the anti-patterns are Because you know, the anti-patterns are Because you know, the anti-patterns are already in a catalog and if you are already in a catalog and if you are already in a catalog and if you are using a coding agent to write your code, using a coding agent to write your code, using a coding agent to write your code, you could actually ask the coding agent you could actually ask the coding agent you could actually ask the coding agent and hook that catalog and hook that catalog and hook that catalog in it so that when it is uh producing in it so that when it is uh producing in it so that when it is uh producing newer tokens and producing newer code, newer tokens and producing newer code, newer tokens and producing newer code, it could actually, before writing out it could actually, before writing out it could actually, before writing out the inefficient code, directly reference the inefficient code, directly reference the inefficient code, directly reference the catalog up front and uh the catalog up front and uh the catalog up front and uh and write your uh and write your uh and write your uh code in an optimal way to start with. code in an optimal way to start with. code in an optimal way to start with. Uh this could sometimes slow down the uh Uh this could sometimes slow down the uh Uh this could sometimes slow down the uh speed of uh newer code uh being written speed of uh newer code uh being written speed of uh newer code uh being written and might end up consuming more tokens. and might end up consuming more tokens. and might end up consuming more tokens. And that's why the pattern catalog needs And that's why the pattern catalog needs And that's why the pattern catalog needs to be very well uh structured and to be very well uh structured and to be very well uh structured and indexed so that the the way parses and indexed so that the the way parses and indexed so that the the way parses and reads the catalog doesn't fill up the reads the catalog doesn't fill up the reads the catalog doesn't fill up the agent context too much, but rather it agent context too much, but rather it agent context too much, but rather it can navigate it in a very hierarchical can navigate it in a very hierarchical can navigate it in a very hierarchical format and only uh and be able to find format and only uh and be able to find format and only uh and be able to find out the exact uh places it needs to look out the exact uh places it needs to look out the exact uh places it needs to look for a uh a given pattern or an for a uh a given pattern or an for a uh a given pattern or an anti-pattern.

anti-pattern. anti-pattern. This means that if you can This means that if you can This means that if you can catch this in the code authoring phase catch this in the code authoring phase catch this in the code authoring phase itself, you skip all of the unnecessary itself, you skip all of the unnecessary itself, you skip all of the unnecessary uh overhead of uh somebody reviewing uh overhead of uh somebody reviewing uh overhead of uh somebody reviewing your code and providing a uh suggestion your code and providing a uh suggestion your code and providing a uh suggestion or even having that code change reach or even having that code change reach or even having that code change reach production. So, the sooner you catch it production. So, the sooner you catch it production. So, the sooner you catch it or even sooner you or even sooner you or even sooner you uh uh uh uh introduce that pattern catalog into uh introduce that pattern catalog into uh introduce that pattern catalog into your software development end-to-end your software development end-to-end your software development end-to-end life cycle, the easier it gets for you life cycle, the easier it gets for you life cycle, the easier it gets for you to keep the uh code and service running to keep the uh code and service running to keep the uh code and service running optimally. optimally. optimally. And uh I do want to highlight this that And uh I do want to highlight this that And uh I do want to highlight this that uh it's easier to imagine putting AI uh it's easier to imagine putting AI uh it's easier to imagine putting AI agent into every single step of your agent into every single step of your agent into every single step of your software development life cycle for software development life cycle for software development life cycle for performance improvements, but that's not performance improvements, but that's not performance improvements, but that's not the intent of this uh discussion or the the intent of this uh discussion or the the intent of this uh discussion or the uh the thing uh the thing uh the thing uh or or the playbook that I'm uh or or the playbook that I'm uh or or the playbook that I'm mentioning. You still want these mentioning. You still want these mentioning. You still want these foundations very, very right. Your test foundations very, very right. Your test foundations very, very right. Your test coverage needs to be rock solid. Uh coverage needs to be rock solid. Uh coverage needs to be rock solid. Uh business logic all needs to be very well business logic all needs to be very well business logic all needs to be very well encoded in your test. Your canary encoded in your test. Your canary encoded in your test. Your canary automation needs to be amazing. Uh and automation needs to be amazing. Uh and automation needs to be amazing. Uh and this is where you don't need AI. You this is where you don't need AI. You this is where you don't need AI. You pretty much just want AI agent to know pretty much just want AI agent to know pretty much just want AI agent to know how to invoke canary, where to get the how to invoke canary, where to get the how to invoke canary, where to get the downloaded report from canary canary, so downloaded report from canary canary, so downloaded report from canary canary, so that it can directly read the numbers that it can directly read the numbers that it can directly read the numbers from the report and uh make a judgment from the report and uh make a judgment from the report and uh make a judgment call.

call. call. And that pattern catalog that I And that pattern catalog that I And that pattern catalog that I mentioned is also a foundational piece mentioned is also a foundational piece mentioned is also a foundational piece which needs to be gotten uh which which which needs to be gotten uh which which which needs to be gotten uh which which you need to get right. It It is by you need to get right. It It is by you need to get right. It It is by design a Git repo, so that it's both a design a Git repo, so that it's both a design a Git repo, so that it's both a human and a coding agent readable and uh human and a coding agent readable and uh human and a coding agent readable and uh authorable, and you could actually have authorable, and you could actually have authorable, and you could actually have humans also authoring patterns and humans also authoring patterns and humans also authoring patterns and anti-pattern as they're let's say anti-pattern as they're let's say anti-pattern as they're let's say building new frameworks and new platform building new frameworks and new platform building new frameworks and new platform components that uh they have enough components that uh they have enough components that uh they have enough context on at the time of building. The takeaways uh that I do want to The takeaways uh that I do want to mention, first build your code mention, first build your code mention, first build your code foundations. Uh foundations. Uh foundations. Uh think of uh all of the uh automation think of uh all of the uh automation think of uh all of the uh automation that you could do, uh but before that you could do, uh but before that you could do, uh but before introducing an LLM, try to see if there introducing an LLM, try to see if there introducing an LLM, try to see if there are good uh uh are good uh uh are good uh uh integration points that an agent could integration points that an agent could integration points that an agent could uh use such as how can it automatically uh use such as how can it automatically uh use such as how can it automatically trigger profiling on an instance on a trigger profiling on an instance on a trigger profiling on an instance on a uh CPU new that you have? Can it uh CPU new that you have? Can it uh CPU new that you have? Can it download the profiling data effectively, download the profiling data effectively, download the profiling data effectively, uh then feed it into uh an LLM uh uh then feed it into uh an LLM uh uh then feed it into uh an LLM uh uh model. uh model. uh model. And can it, once it has identified the And can it, once it has identified the And can it, once it has identified the fixes by cloning the Git repo, etc., can fixes by cloning the Git repo, etc., can fixes by cloning the Git repo, etc., can it actually validate that through a real it actually validate that through a real it actually validate that through a real production canary through shadow traffic production canary through shadow traffic production canary through shadow traffic or real traffic? All of those or real traffic? All of those or real traffic? All of those integration and foundation pieces needs integration and foundation pieces needs integration and foundation pieces needs to be to be to be really good in order for this entire really good in order for this entire really good in order for this entire life to life and this entire uh life to life and this entire uh life to life and this entire uh life cycle improvement that I'm life cycle improvement that I'm life cycle improvement that I'm mentioning. Otherwise, it will just mentioning. Otherwise, it will just mentioning. Otherwise, it will just cause more friction and more bugs in the cause more friction and more bugs in the cause more friction and more bugs in the production. So, you have to make these production. So, you have to make these production. So, you have to make these uh foundation pieces investment a lot uh uh foundation pieces investment a lot uh uh foundation pieces investment a lot uh more important. And start from here.

more important. And start from here. more important. And start from here. Don't think of Don't think of Don't think of uh the reactive path as a bad approach. uh the reactive path as a bad approach. uh the reactive path as a bad approach. Uh reactive path is where you want to Uh reactive path is where you want to Uh reactive path is where you want to start with and then gradually move start with and then gradually move start with and then gradually move towards towards towards uh towards the left and uh into the uh towards the left and uh into the uh towards the left and uh into the proactive path. proactive path. proactive path. >> [clears throat] >> [clears throat] >> [clears throat] >> And the other takeaway, like I >> And the other takeaway, like I >> And the other takeaway, like I mentioned, uh is important is that as mentioned, uh is important is that as mentioned, uh is important is that as you build this uh memory of or you build this uh memory of or you build this uh memory of or essentially a catalog, the essentially a catalog, the essentially a catalog, the um um um the sooner you catch it, the better it the sooner you catch it, the better it the sooner you catch it, the better it gets for your uh end-to-end software gets for your uh end-to-end software gets for your uh end-to-end software development life cycle to have optimus development life cycle to have optimus development life cycle to have optimus optimal code running in production. optimal code running in production. optimal code running in production. Um Um Um This is a I'll uh this is a great mental This is a I'll uh this is a great mental This is a I'll uh this is a great mental model for how you step-by-step proceed model for how you step-by-step proceed model for how you step-by-step proceed in automating in automating in automating um or improving your uh efficiency of um or improving your uh efficiency of um or improving your uh efficiency of software development life cycle. The software development life cycle. The software development life cycle. The current state, uh which is typically the current state, uh which is typically the current state, uh which is typically the norm, uh you spend hours finding the norm, uh you spend hours finding the norm, uh you spend hours finding the problems. It's all very manual, no LLM problems. It's all very manual, no LLM problems. It's all very manual, no LLM in the mix. in the mix. in the mix. Start with introducing an LLM into the Start with introducing an LLM into the Start with introducing an LLM into the mix just for the mix just for the mix just for the um um um the identification of um the identification of um the identification of um uh of a potential fix. It might be okay uh of a potential fix. It might be okay uh of a potential fix. It might be okay if you are still having to trigger a if you are still having to trigger a if you are still having to trigger a profiler uh and produce output and feed profiler uh and produce output and feed profiler uh and produce output and feed it into an AI agent and then having to it into an AI agent and then having to it into an AI agent and then having to manually run canary, but it still gives manually run canary, but it still gives manually run canary, but it still gives you a lot of uh leeway to spend time on you a lot of uh leeway to spend time on you a lot of uh leeway to spend time on the other parts of the workflow as the other parts of the workflow as the other parts of the workflow as opposed to just staring at those flame opposed to just staring at those flame opposed to just staring at those flame graphs to get the um graphs to get the um graphs to get the um hot parts identified.

hot parts identified. hot parts identified. And then this talk mainly covered the uh And then this talk mainly covered the uh And then this talk mainly covered the uh level two orchestration, which is level two orchestration, which is level two orchestration, which is um our level two automation in an um our level two automation in an um our level two automation in an agentic spectrum, which is more about agentic spectrum, which is more about agentic spectrum, which is more about the tools, the integrations, the the key the tools, the integrations, the the key the tools, the integrations, the the key harnesses, and the hooks that our AI harnesses, and the hooks that our AI harnesses, and the hooks that our AI agent needs to have in order to uh have agent needs to have in order to uh have agent needs to have in order to uh have full power in order to have like good uh full power in order to have like good uh full power in order to have like good uh uh uh uh uh capability to identify a problem and uh capability to identify a problem and uh capability to identify a problem and give you in various development life give you in various development life give you in various development life cycle areas of uh opportunities to uh cycle areas of uh opportunities to uh cycle areas of uh opportunities to uh improve your code base. improve your code base. improve your code base. And uh once you have this in place, once And uh once you have this in place, once And uh once you have this in place, once you have this standard step-by-step you have this standard step-by-step you have this standard step-by-step workflow, which could be very static, workflow, which could be very static, workflow, which could be very static, right? In this case, the workflow that I right? In this case, the workflow that I right? In this case, the workflow that I mentioned is is very uh well-defined, mentioned is is very uh well-defined, mentioned is is very uh well-defined, predefined. There is no uh predefined. There is no uh predefined. There is no uh uh LLM to reason and plan and uh do the uh LLM to reason and plan and uh do the uh LLM to reason and plan and uh do the further level of automations. All it has further level of automations. All it has further level of automations. All it has to do is uh you provide it a to do is uh you provide it a to do is uh you provide it a uh fixed workflow, which is download the uh fixed workflow, which is download the uh fixed workflow, which is download the profile profile profile uh from certain place, um uh from certain place, um uh from certain place, um or rather trigger the profile, download or rather trigger the profile, download or rather trigger the profile, download the profile, the profile, the profile, um and then uh analyze it, uh run the um and then uh analyze it, uh run the um and then uh analyze it, uh run the canary, and provide a suggested fix. Uh canary, and provide a suggested fix. Uh canary, and provide a suggested fix. Uh you could once you have that in place, you could once you have that in place, you could once you have that in place, you could essentially you could essentially you could essentially have a scheduled run on your services have a scheduled run on your services have a scheduled run on your services every week, so that it can keep every week, so that it can keep every week, so that it can keep continuously identifying newer problems continuously identifying newer problems continuously identifying newer problems uh that might have not been uh uh uh that might have not been uh uh uh that might have not been uh uh uh or that might have been introduced in uh or that might have been introduced in uh or that might have been introduced in the last one week, for example, and keep the last one week, for example, and keep the last one week, for example, and keep uh the the identification of the problem uh the the identification of the problem uh the the identification of the problem closer to when you wrote that code.

closer to when you wrote that code. closer to when you wrote that code. And if you were to think of uh more And if you were to think of uh more And if you were to think of uh more automation automation automation uh uh uh in this place, you could in this place, you could in this place, you could uh um uh um uh um imagine a a a level where the workflow imagine a a a level where the workflow imagine a a a level where the workflow is not very fixed, and rather the AI is not very fixed, and rather the AI is not very fixed, and rather the AI agent has capabilities to even uh plan, agent has capabilities to even uh plan, agent has capabilities to even uh plan, reason, act, uh but you have to invest a reason, act, uh but you have to invest a reason, act, uh but you have to invest a lot more in evaluation sandboxing. lot more in evaluation sandboxing. lot more in evaluation sandboxing. You can't simply run AI agents You can't simply run AI agents You can't simply run AI agents without the right level of security without the right level of security without the right level of security guardrails so that prompt injection and guardrails so that prompt injection and guardrails so that prompt injection and other security attacks that a typical other security attacks that a typical other security attacks that a typical agent infrastructure currently agent infrastructure currently agent infrastructure currently cannot always solve for. Those cannot always solve for. Those cannot always solve for. Those things need to be heavily invested in if things need to be heavily invested in if things need to be heavily invested in if you were to go with level three you were to go with level three you were to go with level three autonomy. So, start with level one. Try autonomy. So, start with level one. Try autonomy. So, start with level one. Try to move to level two and you get maximum to move to level two and you get maximum to move to level two and you get maximum benefits. If you need more automation, benefits. If you need more automation, benefits. If you need more automation, you could think of the next step as you could think of the next step as you could think of the next step as needed. Thank you so much. Thank you so much. I hope you got some learnings out of I hope you got some learnings out of I hope you got some learnings out of this small talk that I had.

this small talk that I had. this small talk that I had. If if you want to connect more and If if you want to connect more and If if you want to connect more and connect further, connect further, connect further, I might end up putting a more detailed I might end up putting a more detailed I might end up putting a more detailed blog post on this on my shaharrajat.com blog post on this on my shaharrajat.com blog post on this on my shaharrajat.com personal website. I'll be happy to personal website. I'll be happy to personal website. I'll be happy to connect with you and provide more connect with you and provide more connect with you and provide more information. information. information. Take care. Thank you.