What to Do When Software Engineering Becomes Context Engineering

So hi everyone and welcome to blueprint So hi everyone and welcome to blueprint plus at the intersection of business and plus at the intersection of business and plus at the intersection of business and technology. My name is Kira Burn and I'm technology. My name is Kira Burn and I'm technology. My name is Kira Burn and I'm the director of engineering productivity the director of engineering productivity the director of engineering productivity and AI impact at Jet Brains and my guest and AI impact at Jet Brains and my guest and AI impact at Jet Brains and my guest today is Maxim Salnikov who from today is Maxim Salnikov who from today is Maxim Salnikov who from Microsoft who helps enterprise Microsoft who helps enterprise Microsoft who helps enterprise development teams to uh move to AI development teams to uh move to AI development teams to uh move to AI native development at scale. So you're native development at scale. So you're native development at scale. So you're very welcome Axim. Uh can you tell us a very welcome Axim. Uh can you tell us a very welcome Axim. Uh can you tell us a little bit more about your work? little bit more about your work? little bit more about your work? >> Absolutely. Thank you so much for having >> Absolutely. Thank you so much for having >> Absolutely. Thank you so much for having me, for inviting me and hello everyone. me, for inviting me and hello everyone. me, for inviting me and hello everyone. I feel myself that I'm a full-time I feel myself that I'm a full-time I feel myself that I'm a full-time learner these days. learner these days. learner these days. >> We all are. >> We all are. >> We all are. >> And uh my my personal approach for >> And uh my my personal approach for >> And uh my my personal approach for learning is learning by teaching, learning is learning by teaching, learning is learning by teaching, learning by training, learning by learning by training, learning by learning by training, learning by enabling. And I do it at scale at enabling. And I do it at scale at enabling. And I do it at scale at Microsoft as a senior solution engineer Microsoft as a senior solution engineer Microsoft as a senior solution engineer with a very kind of niche area that is with a very kind of niche area that is with a very kind of niche area that is developer tools. And today we can safely developer tools. And today we can safely developer tools. And today we can safely say that by saying developer tools tools say that by saying developer tools tools say that by saying developer tools tools we say we mean AI native or AI powered we say we mean AI native or AI powered we say we mean AI native or AI powered developer tools. So this is uh what developer tools. So this is uh what developer tools. So this is uh what keeps me busy these days. But uh beyond keeps me busy these days. But uh beyond keeps me busy these days. But uh beyond that before joining Microsoft I spent a that before joining Microsoft I spent a that before joining Microsoft I spent a couple of decades in uh development in couple of decades in uh development in couple of decades in uh development in web development. So like and still build web development. So like and still build web development. So like and still build every single day. And I'm super happy to every single day. And I'm super happy to every single day. And I'm super happy to look at my GitHub contribution graph look at my GitHub contribution graph look at my GitHub contribution graph that is uh open up that is uh open up that is uh open up >> darker and darker green these days. Um >> darker and darker green these days. Um >> darker and darker green these days. Um yeah at um doing my best to research, yeah at um doing my best to research, yeah at um doing my best to research, explore and share with uh around 100

explore and share with uh around 100 explore and share with uh around 100 enterprise customers I work with like enterprise customers I work with like enterprise customers I work with like specifically with their developer teams, specifically with their developer teams, specifically with their developer teams, technical leads. We are all learners. technical leads. We are all learners. technical leads. We are all learners. >> Amazing. Uh so one thing that we see >> Amazing. Uh so one thing that we see >> Amazing. Uh so one thing that we see customers come up against is this sort customers come up against is this sort customers come up against is this sort of uh individual productivity you know a of uh individual productivity you know a of uh individual productivity you know a lot of speed up in terms of PR lot of speed up in terms of PR lot of speed up in terms of PR throughput and stuff like this for throughput and stuff like this for throughput and stuff like this for individual developers and then teams individual developers and then teams individual developers and then teams kind of hit a wall where they're not kind of hit a wall where they're not kind of hit a wall where they're not really seeing the team level or the really seeing the team level or the really seeing the team level or the organization level productivity. So what organization level productivity. So what organization level productivity. So what why do you think that's happening and why do you think that's happening and why do you think that's happening and how can we tackle it? That's a very big how can we tackle it? That's a very big how can we tackle it? That's a very big question that also keeps busy lots of question that also keeps busy lots of question that also keeps busy lots of technical leads I work with. So they see technical leads I work with. So they see technical leads I work with. So they see okay there is a individual developer and okay there is a individual developer and okay there is a individual developer and their productivity their productivity their productivity say 50% say 50% say 50% better these days but the point is better these days but the point is better these days but the point is system is not ready right and uh if we system is not ready right and uh if we system is not ready right and uh if we talk specifically about coding part and talk specifically about coding part and talk specifically about coding part and by the way I am not big fan of calling by the way I am not big fan of calling by the way I am not big fan of calling um these AI dev tools just code um these AI dev tools just code um these AI dev tools just code assistants we can call it like that to assistants we can call it like that to assistants we can call it like that to to simplify u but coding itself um in to simplify u but coding itself um in to simplify u but coding itself um in enterprise development Maybe it's uh I enterprise development Maybe it's uh I enterprise development Maybe it's uh I different teams different uh ratios but different teams different uh ratios but different teams different uh ratios but maybe 30% maximum right and maybe 30% maximum right and maybe 30% maximum right and >> we've had 15 to 20% and >> we've had 15 to 20% and >> we've had 15 to 20% and >> yeah yeah so I I I try to be very >> yeah yeah so I I I try to be very >> yeah yeah so I I I try to be very optimistic in that sense um so if you optimistic in that sense um so if you optimistic in that sense um so if you boost up uh this part for 50% it will boost up uh this part for 50% it will boost up uh this part for 50% it will you know be only a fraction of um what you know be only a fraction of um what you know be only a fraction of um what you see as overall productivity.

you see as overall productivity. you see as overall productivity. >> Yeah. Uh and there is a deeper problem >> Yeah. Uh and there is a deeper problem >> Yeah. Uh and there is a deeper problem more code and even like if every team more code and even like if every team more code and even like if every team every team member now produces uh new every team member now produces uh new every team member now produces uh new lines of the code with uh this lines of the code with uh this lines of the code with uh this incredible speed like let's keep quality incredible speed like let's keep quality incredible speed like let's keep quality conversations aside a bit we'll return conversations aside a bit we'll return conversations aside a bit we'll return to that later. to that later. to that later. >> You hit uh next uh part in this software >> You hit uh next uh part in this software >> You hit uh next uh part in this software development pipeline next bottleneck development pipeline next bottleneck development pipeline next bottleneck that is review and it's a that is review and it's a that is review and it's a multi-dimensional multi-dimensional multi-dimensional problem. So it's not only about problem. So it's not only about problem. So it's not only about quantity of the code, it's also about quantity of the code, it's also about quantity of the code, it's also about specific specific specific let's say uh specific nature of this AI let's say uh specific nature of this AI let's say uh specific nature of this AI generated code that is uh that could be generated code that is uh that could be generated code that is uh that could be let's say confidently wrong and the code let's say confidently wrong and the code let's say confidently wrong and the code could uh be okay let's say maybe maybe could uh be okay let's say maybe maybe could uh be okay let's say maybe maybe not perfect but good enough when you not perfect but good enough when you not perfect but good enough when you look at this code from in in like local look at this code from in in like local look at this code from in in like local perspective, right? Um so it could be perspective, right? Um so it could be perspective, right? Um so it could be what we can call locally coherent but what we can call locally coherent but what we can call locally coherent but globally inconsistent and of course globally inconsistent and of course globally inconsistent and of course again um no AI agent will leave you with again um no AI agent will leave you with again um no AI agent will leave you with some uh red um tests right so it the some uh red um tests right so it the some uh red um tests right so it the code checks all the check marks but not code checks all the check marks but not code checks all the check marks but not obligatory in the context of whole obligatory in the context of whole obligatory in the context of whole application and these nature of um of application and these nature of um of application and these nature of um of bugs of defects is much more challenging bugs of defects is much more challenging bugs of defects is much more challenging to to capture. So again this is why team to to capture. So again this is why team to to capture. So again this is why team productivity, unit productivity,

productivity, unit productivity, productivity, unit productivity, organizational productivity organizational productivity organizational productivity saying gentle is not uh could not be saying gentle is not uh could not be saying gentle is not uh could not be that scale like individual productivity. that scale like individual productivity. that scale like individual productivity. The answer is changing the system The answer is changing the system The answer is changing the system rewiring the system rearchitecturing the rewiring the system rearchitecturing the rewiring the system rearchitecturing the system and I believe this is where we system and I believe this is where we system and I believe this is where we dedicate spend much time today during dedicate spend much time today during dedicate spend much time today during our conversation. our conversation. our conversation. >> Absolutely. So like it's engineering the >> Absolutely. So like it's engineering the >> Absolutely. So like it's engineering the system that's absolutely instead of system that's absolutely instead of system that's absolutely instead of engineering the software engineering the software engineering the software >> you could say. Um it sounds like then >> you could say. Um it sounds like then >> you could say. Um it sounds like then one of the issues here is lack of one of the issues here is lack of one of the issues here is lack of context right so you have local context context right so you have local context context right so you have local context or maybe your agents have local context or maybe your agents have local context or maybe your agents have local context but you don't have the wider context and but you don't have the wider context and but you don't have the wider context and I think you speak a lot about context uh I think you speak a lot about context uh I think you speak a lot about context uh as like one of the basic things you need as like one of the basic things you need as like one of the basic things you need to scale. So can you tell us a bit about to scale. So can you tell us a bit about to scale. So can you tell us a bit about more about that? 100% context more about that? 100% context more about that? 100% context engineering. I believe this is what uh I engineering. I believe this is what uh I engineering. I believe this is what uh I will mention multiple times during today will mention multiple times during today will mention multiple times during today and this is what uh really builds um and this is what uh really builds um and this is what uh really builds um today's intellectual property of the of today's intellectual property of the of today's intellectual property of the of the company. So how well you build this the company. So how well you build this the company. So how well you build this layer or these layers uh is directly layer or these layers uh is directly layer or these layers uh is directly affected the affecting quality of the affected the affecting quality of the affected the affecting quality of the output from uh from AI agents 100%. and output from uh from AI agents 100%. and output from uh from AI agents 100%. and uh talking about how software uh talking about how software uh talking about how software engineering role is evolving to much engineering role is evolving to much engineering role is evolving to much extent.

extent. extent. Software developers become context Software developers become context Software developers become context engineers especially if you talk about engineers especially if you talk about engineers especially if you talk about senior um software engineers uh who also senior um software engineers uh who also senior um software engineers uh who also owns and u navigates well domain owns and u navigates well domain owns and u navigates well domain knowledge of uh of the solution they knowledge of uh of the solution they knowledge of uh of the solution they build. Yeah, they play crucial role in build. Yeah, they play crucial role in build. Yeah, they play crucial role in uh uh building this uh context layers uh uh building this uh context layers uh uh building this uh context layers and it's much more say multi- faceted and it's much more say multi- faceted and it's much more say multi- faceted problem than just writing markdown problem than just writing markdown problem than just writing markdown files. So let's uh look at this as a files. So let's uh look at this as a files. So let's uh look at this as a just create this instruction create this just create this instruction create this just create this instruction create this uh find this agent skill no it's a it's uh find this agent skill no it's a it's uh find this agent skill no it's a it's a skill it's a knowledge and it requires a skill it's a knowledge and it requires a skill it's a knowledge and it requires iterations. So what are the most iterations. So what are the most iterations. So what are the most important aspects of that for you if you important aspects of that for you if you important aspects of that for you if you had to say the top three things beyond had to say the top three things beyond had to say the top three things beyond like instruction files or whatever or like instruction files or whatever or like instruction files or whatever or maybe rag you know this kind of thing maybe rag you know this kind of thing maybe rag you know this kind of thing that people are familiar with. that people are familiar with. that people are familiar with. >> Yes. Um what I see today as uh one of >> Yes. Um what I see today as uh one of >> Yes. Um what I see today as uh one of the biggest gaps in understanding this the biggest gaps in understanding this the biggest gaps in understanding this context engineering layer is enterprise context engineering layer is enterprise context engineering layer is enterprise approach to the way how you manage all approach to the way how you manage all approach to the way how you manage all these assets. We can call them AI native these assets. We can call them AI native these assets. We can call them AI native dev primitives because um yeah uh for dev primitives because um yeah uh for dev primitives because um yeah uh for sure all this becomes inevitable part of sure all this becomes inevitable part of sure all this becomes inevitable part of your code repository right but um the your code repository right but um the your code repository right but um the question is how you how do you make sure question is how you how do you make sure question is how you how do you make sure that u it's all always up to date that u it's all always up to date that u it's all always up to date >> yeah how is it maintained and who >> yeah how is it maintained and who >> yeah how is it maintained and who maintains it maintains it maintains it >> yes yes yes and uh it also um like um >> yes yes yes and uh it also um like um >> yes yes yes and uh it also um like um invokes questions about security right

invokes questions about security right invokes questions about security right because I've seen too many times how uh because I've seen too many times how uh because I've seen too many times how uh folks found agent skilled There are folks found agent skilled There are folks found agent skilled There are plenty of repositories of agent skills plenty of repositories of agent skills plenty of repositories of agent skills these days, right? Yeah. Uh some of them these days, right? Yeah. Uh some of them these days, right? Yeah. Uh some of them are from trusted vendors, some of them are from trusted vendors, some of them are from trusted vendors, some of them are just uh popping up and uh even even are just uh popping up and uh even even are just uh popping up and uh even even trusted vendors can be compromised, trusted vendors can be compromised, trusted vendors can be compromised, right? So right? So right? So >> uh for sure you don't want to rely on >> uh for sure you don't want to rely on >> uh for sure you don't want to rely on this third party code. We can call it this third party code. We can call it this third party code. We can call it code, right? So um um even like these code, right? So um um even like these code, right? So um um even like these markdown files even though they look markdown files even though they look markdown files even though they look like text for us this is text for like text for us this is text for like text for us this is text for agents. This is executable code. agents. This is executable code. agents. This is executable code. >> It's it's it's instructions, right? >> It's it's it's instructions, right? >> It's it's it's instructions, right? Exactly. Exactly. And uh specifically if Exactly. Exactly. And uh specifically if Exactly. Exactly. And uh specifically if you talk about agent skills part of its you talk about agent skills part of its you talk about agent skills part of its specification is scripts themselves to specification is scripts themselves to specification is scripts themselves to to keep them a bit more deterministic. to keep them a bit more deterministic. to keep them a bit more deterministic. Right. And um did anyone like did did Right. And um did anyone like did did Right. And um did anyone like did did everyone validate every single time everyone validate every single time everyone validate every single time what's going on in this script that um what's going on in this script that um what's going on in this script that um like downloaded from from internet. So like downloaded from from internet. So like downloaded from from internet. So uh gap is here. Luckily there are tools uh gap is here. Luckily there are tools uh gap is here. Luckily there are tools plenty of them on the market already. plenty of them on the market already. plenty of them on the market already. One of them I can uh mention agent One of them I can uh mention agent One of them I can uh mention agent package manager uh from from my good uh package manager uh from from my good uh package manager uh from from my good uh friend my colleague Daniel Meila.

friend my colleague Daniel Meila. friend my colleague Daniel Meila. >> Okay. >> Okay. >> Okay. >> Uh that applies >> Uh that applies >> Uh that applies let's say same approach to from from let's say same approach to from from let's say same approach to from from let's say any package manager to let's say any package manager to let's say any package manager to agent uh primitives management. It's u agent uh primitives management. It's u agent uh primitives management. It's u governance. It's uh versioning. It's for governance. It's uh versioning. It's for governance. It's uh versioning. It's for example pinning to specific commit example pinning to specific commit example pinning to specific commit versions. Um so basically you apply this versions. Um so basically you apply this versions. Um so basically you apply this all the same techniques like you apply all the same techniques like you apply all the same techniques like you apply for example when you use npm and what's for example when you use npm and what's for example when you use npm and what's the best it uh successfully works within the best it uh successfully works within the best it uh successfully works within your enterprise boundary right so once your enterprise boundary right so once your enterprise boundary right so once you for example uh got some skill uh and you for example uh got some skill uh and you for example uh got some skill uh and validated this and fine-tuned this for validated this and fine-tuned this for validated this and fine-tuned this for your own context uh you kind of detach your own context uh you kind of detach your own context uh you kind of detach it from uh from upstream right and you it from uh from upstream right and you it from uh from upstream right and you accumulate your very own team or maybe accumulate your very own team or maybe accumulate your very own team or maybe organization boundary of uh of all these organization boundary of uh of all these organization boundary of uh of all these uh assets. So, agent skills is just one uh assets. So, agent skills is just one uh assets. So, agent skills is just one of examples, right? There are plenty of of examples, right? There are plenty of of examples, right? There are plenty of these things that directly affects these things that directly affects these things that directly affects quality and you basically you you you quality and you basically you you you quality and you basically you you you accumulate this knowledge and that that accumulate this knowledge and that that accumulate this knowledge and that that compounds and so we there are plenty of compounds and so we there are plenty of compounds and so we there are plenty of use cases when you use instructions to use cases when you use instructions to use cases when you use instructions to improve instructions. You use agent improve instructions. You use agent improve instructions. You use agent skills to build agent skills. Uh but skills to build agent skills. Uh but skills to build agent skills. Uh but yeah again could be separate yeah again could be separate yeah again could be separate conversation. I'm super excited about conversation. I'm super excited about conversation. I'm super excited about that.

that. that. >> Things like code review comments from >> Things like code review comments from >> Things like code review comments from human developers andor agents that human developers andor agents that human developers andor agents that repeat and then you incorporate those repeat and then you incorporate those repeat and then you incorporate those and you're harnessed and so on. Okay. So and you're harnessed and so on. Okay. So and you're harnessed and so on. Okay. So two things seem to be happening like you two things seem to be happening like you two things seem to be happening like you say there's uh you're speeding up uh say there's uh you're speeding up uh say there's uh you're speeding up uh code generation for individual code generation for individual code generation for individual developers but the whole system around developers but the whole system around developers but the whole system around them is not evolving or needs to evolve. them is not evolving or needs to evolve. them is not evolving or needs to evolve. >> Yes. >> Yes. >> Yes. >> And then you have this lack of context. >> And then you have this lack of context. >> And then you have this lack of context. So what is the advice that you give So what is the advice that you give So what is the advice that you give teams on how they need to like redesign teams on how they need to like redesign teams on how they need to like redesign their system in order to deal with these their system in order to deal with these their system in order to deal with these two realities so that they can do a two realities so that they can do a two realities so that they can do a agent native you could say about it? agent native you could say about it? agent native you could say about it? >> Uh we could start like a step back from >> Uh we could start like a step back from >> Uh we could start like a step back from from redesigning the system redesign the from redesigning the system redesign the from redesigning the system redesign the team that u for for many teams for many team that u for for many teams for many team that u for for many teams for many individual developers for many individual developers for many individual developers for many organizations it's going out of their organizations it's going out of their organizations it's going out of their comfort zone. Yes, let's be realistic. comfort zone. Yes, let's be realistic. comfort zone. Yes, let's be realistic. Let's be honest. Uh but uh that's the Let's be honest. Uh but uh that's the Let's be honest. Uh but uh that's the only way to kind of move forward. And um only way to kind of move forward. And um only way to kind of move forward. And um thinking about AI agents like about thinking about AI agents like about thinking about AI agents like about tools like tools only is a bit legacy tools like tools only is a bit legacy tools like tools only is a bit legacy thinking. Maybe it was uh true like two thinking. Maybe it was uh true like two thinking. Maybe it was uh true like two three years ago in the very beginning of three years ago in the very beginning of three years ago in the very beginning of this tooling. But now this tooling. But now this tooling. But now >> uh conceptually we think about these um >> uh conceptually we think about these um >> uh conceptually we think about these um tools. they still technically still tools. they still technically still tools. they still technically still tools like uh our digital team members tools like uh our digital team members tools like uh our digital team members uh but this is only conceptually uh uh but this is only conceptually uh uh but this is only conceptually uh looking a bit ahead make sure that your looking a bit ahead make sure that your looking a bit ahead make sure that your AI agent is not operating with the same

AI agent is not operating with the same AI agent is not operating with the same permissions as the developer who is permissions as the developer who is permissions as the developer who is running this AI agent okay it's so easy running this AI agent okay it's so easy running this AI agent okay it's so easy to get into this trap and again luckily to get into this trap and again luckily to get into this trap and again luckily we have set of tools set of uh measures we have set of tools set of uh measures we have set of tools set of uh measures techniques to run these AI agents with techniques to run these AI agents with techniques to run these AI agents with the least possible privileges the least possible privileges the least possible privileges >> let's talk about that a a little bit. So >> let's talk about that a a little bit. So >> let's talk about that a a little bit. So AI guardrails like AI guardrails like AI guardrails like >> but but yeah let >> but but yeah let >> but but yeah let before we drift away before you drift before we drift away before you drift before we drift away before you drift away let me uh finalize on um on that so away let me uh finalize on um on that so away let me uh finalize on um on that so and it it's about changing the team and it it's about changing the team and it it's about changing the team right and u you think about how do you right and u you think about how do you right and u you think about how do you on board new team member in your team on board new team member in your team on board new team member in your team you need someone who will properly on you need someone who will properly on you need someone who will properly on board this uh say junior team member and board this uh say junior team member and board this uh say junior team member and uh AI agents again let's be honest they uh AI agents again let's be honest they uh AI agents again let's be honest they are capable they are fast but they have are capable they are fast but they have are capable they are fast but they have kind of amnesia right so constant uh kind of amnesia right so constant uh kind of amnesia right so constant uh kind of onboarding is needed um next kind of onboarding is needed um next kind of onboarding is needed um next person should review output of these AI person should review output of these AI person should review output of these AI agents right agents right agents right >> while they're on boarding particularly >> while they're on boarding particularly >> while they're on boarding particularly [laughter] [laughter] [laughter] >> yeah including this this process >> yeah including this this process >> yeah including this this process >> uh and review of the code is a is >> uh and review of the code is a is >> uh and review of the code is a is another big big big topic I hope that another big big big topic I hope that another big big big topic I hope that we'll have some time to cover that we'll have some time to cover that we'll have some time to cover that because today's reviews are very because today's reviews are very because today's reviews are very different from yesterday's reviews.

different from yesterday's reviews. different from yesterday's reviews. Yeah. And another person is needed to Yeah. And another person is needed to Yeah. And another person is needed to kind of react on all possible kind of react on all possible kind of react on all possible escalations because uh again let's be escalations because uh again let's be escalations because uh again let's be realistic today's AI agents they need realistic today's AI agents they need realistic today's AI agents they need some u some some care right so they can some u some some care right so they can some u some some care right so they can stuck they can go into infinite loops stuck they can go into infinite loops stuck they can go into infinite loops etc etc. So I mentioned like three etc etc. So I mentioned like three etc etc. So I mentioned like three different people in reality of course different people in reality of course different people in reality of course these are not three different people not these are not three different people not these are not three different people not even one separate person but these are even one separate person but these are even one separate person but these are three new roles for every single three new roles for every single three new roles for every single developer in the team. So that's today's developer in the team. So that's today's developer in the team. So that's today's reality. This is uh yeah um how how that reality. This is uh yeah um how how that reality. This is uh yeah um how how that not disrupts the team but augments the not disrupts the team but augments the not disrupts the team but augments the team and also uh to to mention about uh team and also uh to to mention about uh team and also uh to to mention about uh what different uh maturity developers do what different uh maturity developers do what different uh maturity developers do these days how I envision that and by these days how I envision that and by these days how I envision that and by the way by the way it's very important the way by the way it's very important the way by the way it's very important to say that uh lots of statements lots to say that uh lots of statements lots to say that uh lots of statements lots of ideas lots of um say concepts I of ideas lots of um say concepts I of ideas lots of um say concepts I mentioned today are taken by me uh from mentioned today are taken by me uh from mentioned today are taken by me uh from the book called the book called the book called Agentic SDLC handbook uh by my colleague Agentic SDLC handbook uh by my colleague Agentic SDLC handbook uh by my colleague Daniel. Uh and uh this is not just you Daniel. Uh and uh this is not just you Daniel. Uh and uh this is not just you know um ideas I uh I took from this know um ideas I uh I took from this know um ideas I uh I took from this book. These are ideas that I validated book. These are ideas that I validated book. These are ideas that I validated >> with with the teams that you work with >> with with the teams that you work with >> with with the teams that you work with >> the teams I work with and yeah I think >> the teams I work with and yeah I think >> the teams I work with and yeah I think this year I spent like well over 100 this year I spent like well over 100 this year I spent like well over 100 hours of of just you know hands-on hours of of just you know hands-on hours of of just you know hands-on workshops uh technical sessions workshops uh technical sessions workshops uh technical sessions whatever. So yeah I I know this from whatever. So yeah I I know this from whatever. So yeah I I know this from from my experience. Yeah. So back to

from my experience. Yeah. So back to from my experience. Yeah. So back to what new developers let's say what what what new developers let's say what what what new developers let's say what what developers in a new environment do developers in a new environment do developers in a new environment do senior ones the most senior ones they senior ones the most senior ones they senior ones the most senior ones they are becoming really more contact are becoming really more contact are becoming really more contact engineers. engineers. engineers. >> Yeah. And system designers. >> Yeah. And system designers. >> Yeah. And system designers. >> Yeah. So we we we actually we already >> Yeah. So we we we actually we already >> Yeah. So we we we actually we already mentioned that uh briefly midlevel mentioned that uh briefly midlevel mentioned that uh briefly midlevel developers they focus today on uh developers they focus today on uh developers they focus today on uh detailed specifications detailed specifications detailed specifications uh uh uh >> for the for the tasks that they won't. >> for the for the tasks that they won't. >> for the for the tasks that they won't. Yeah, because they also know uh domain Yeah, because they also know uh domain Yeah, because they also know uh domain well and um with without very explicit well and um with without very explicit well and um with without very explicit very very very >> uh step by step and very constrained >> uh step by step and very constrained >> uh step by step and very constrained specification your AI agent will say specification your AI agent will say specification your AI agent will say will not behave will not behave will not behave >> what uh junior engineers do these days >> what uh junior engineers do these days >> what uh junior engineers do these days that's a big question let's take it a that's a big question let's take it a that's a big question let's take it a bit later um and um I can tell that um bit later um and um I can tell that um bit later um and um I can tell that um there are also a couple of antiatterns there are also a couple of antiatterns there are also a couple of antiatterns in how these teams could be structured in how these teams could be structured in how these teams could be structured also that I see it left and it and it also that I see it left and it and it also that I see it left and it and it doesn't work well at least at long term.

doesn't work well at least at long term. doesn't work well at least at long term. uh for example uh identifying steam of uh for example uh identifying steam of uh for example uh identifying steam of AI superheroes across all organization AI superheroes across all organization AI superheroes across all organization and say hey these are say all knowledge and say hey these are say all knowledge and say hey these are say all knowledge uh owners around um agentic development uh owners around um agentic development uh owners around um agentic development agentic AI development and go to them agentic AI development and go to them agentic AI development and go to them and ask their questions and they only and ask their questions and they only and ask their questions and they only they will build this contextual layer they will build this contextual layer they will build this contextual layer could work on uh short term but these could work on uh short term but these could work on uh short term but these folks are not obligatory do the main folks are not obligatory do the main folks are not obligatory do the main knowledge owners from every single team, knowledge owners from every single team, knowledge owners from every single team, right? So they they might know right? So they they might know right? So they they might know everything about AI but not about all everything about AI but not about all everything about AI but not about all pieces all parts of your solution. pieces all parts of your solution. pieces all parts of your solution. >> So does this tend to be like AI >> So does this tend to be like AI >> So does this tend to be like AI champions distributed throughout the champions distributed throughout the champions distributed throughout the organization or do people set aside a organization or do people set aside a organization or do people set aside a couple of teams and say learn everything couple of teams and say learn everything couple of teams and say learn everything about how to do agentic development or about how to do agentic development or about how to do agentic development or how does it normally work this anti how does it normally work this anti how does it normally work this anti pattern? pattern? pattern? >> Yes. Uh so if I if I mention I I should >> Yes. Uh so if I if I mention I I should >> Yes. Uh so if I if I mention I I should mention uh pattern right? uh the best mention uh pattern right? uh the best mention uh pattern right? uh the best practice is to have uh context practice is to have uh context practice is to have uh context engineering knowledge, skill, engineering knowledge, skill, engineering knowledge, skill, responsibility embedded into every responsibility embedded into every responsibility embedded into every single team.

single team. single team. >> So the your everyday regular development >> So the your everyday regular development >> So the your everyday regular development teams. teams. teams. >> Yes. Yes. Yes. I mean not not obligatory >> Yes. Yes. Yes. I mean not not obligatory >> Yes. Yes. Yes. I mean not not obligatory every member of uh the particular team every member of uh the particular team every member of uh the particular team uh are full day full-time context uh are full day full-time context uh are full day full-time context engineers but across every single team engineers but across every single team engineers but across every single team um let's say kind of domain uh knowledge um let's say kind of domain uh knowledge um let's say kind of domain uh knowledge uh boundaries because uh uh boundaries because uh uh boundaries because uh >> it's not again about uh technical >> it's not again about uh technical >> it's not again about uh technical writing only it's about deep deep writing only it's about deep deep writing only it's about deep deep knowledge and not not only kind of knowledge and not not only kind of knowledge and not not only kind of snapshot of this knowledge but evolving snapshot of this knowledge but evolving snapshot of this knowledge but evolving knowledge your your your product evolves knowledge your your your product evolves knowledge your your your product evolves and uh your context layer should evolve and uh your context layer should evolve and uh your context layer should evolve uh And this tends to be where agents uh And this tends to be where agents uh And this tends to be where agents fall down as well like you say because fall down as well like you say because fall down as well like you say because they don't they have amnesia. They don't they don't they have amnesia. They don't they don't they have amnesia. They don't have have have >> Yes. Yes. Yes. Yes. >> Yes. Yes. Yes. Yes. >> Yes. Yes. Yes. Yes. >> Uh very good memory yet etc. So >> Uh very good memory yet etc. So >> Uh very good memory yet etc. So developers have to be that memory and developers have to be that memory and developers have to be that memory and provide that context. provide that context. provide that context. >> Yes. Yes. Yes. So there's uh the say >> Yes. Yes. Yes. So there's uh the say >> Yes. Yes. Yes. So there's uh the say creating this uh context layer is creating this uh context layer is creating this uh context layer is becoming if not a separate role but I becoming if not a separate role but I becoming if not a separate role but I will not be surprised that in large will not be surprised that in large will not be surprised that in large teams larger organizations there will be teams larger organizations there will be teams larger organizations there will be separate roles. context engineer maybe separate roles. context engineer maybe separate roles. context engineer maybe uh agent ops will evolve into its its uh agent ops will evolve into its its uh agent ops will evolve into its its own role it's a bit like early days but own role it's a bit like early days but own role it's a bit like early days but I will not be surprised again I will not be surprised again I will not be surprised again >> and what was your second anti pattern >> and what was your second anti pattern >> and what was your second anti pattern that you mentioned that you mentioned that you mentioned >> uh the the second one is uh building two >> uh the the second one is uh building two >> uh the the second one is uh building two parallel tracks this is also I see quite parallel tracks this is also I see quite parallel tracks this is also I see quite often when uh often when uh often when uh >> some tech leads say so this code is uh >> some tech leads say so this code is uh >> some tech leads say so this code is uh too important it's crucial it's critical too important it's crucial it's critical too important it's crucial it's critical so no AI here but this is uh something so no AI here but this is uh something so no AI here but this is uh something Okay to to apply AI on wrong pattern

Okay to to apply AI on wrong pattern Okay to to apply AI on wrong pattern again might work in the short term but again might work in the short term but again might work in the short term but uh there will be always tension like uh there will be always tension like uh there will be always tension like what code is which code is uh to build what code is which code is uh to build what code is which code is uh to build which category category right uh and which category category right uh and which category category right uh and there will be some disagreements and it there will be some disagreements and it there will be some disagreements and it also uh kind of contaminates all the also uh kind of contaminates all the also uh kind of contaminates all the idea of uh true AI native development idea of uh true AI native development idea of uh true AI native development for sure there there should be risk for sure there there should be risk for sure there there should be risk management right and reviews of this uh management right and reviews of this uh management right and reviews of this uh high like potent like um critical track high like potent like um critical track high like potent like um critical track should be done differently but um yeah should be done differently but um yeah should be done differently but um yeah maybe uh completely get rid of AI from maybe uh completely get rid of AI from maybe uh completely get rid of AI from particular code track is antipart as particular code track is antipart as particular code track is antipart as well well well >> so it's more about how you do the >> so it's more about how you do the >> so it's more about how you do the verification and validation for verification and validation for verification and validation for particular types of code rather than particular types of code rather than particular types of code rather than you're saying you should only use AI you're saying you should only use AI you're saying you should only use AI here because here because here because >> it's uh it's not critical. >> it's uh it's not critical. >> it's uh it's not critical. >> Yes. >> Yes. >> Yes. >> Okay. Um you talked a little bit about >> Okay. Um you talked a little bit about >> Okay. Um you talked a little bit about uh one of the reasons that people can't uh one of the reasons that people can't uh one of the reasons that people can't move from uh AI from an in as an move from uh AI from an in as an move from uh AI from an in as an individual to an organizational individual to an organizational individual to an organizational capability is these bottlenecks, right?

capability is these bottlenecks, right? capability is these bottlenecks, right? Review is an obvious one. Um we can talk Review is an obvious one. Um we can talk Review is an obvious one. Um we can talk so we can talk about that a little bit so we can talk about that a little bit so we can talk about that a little bit and then maybe some of the other and then maybe some of the other and then maybe some of the other bottlenecks that you see emerging with bottlenecks that you see emerging with bottlenecks that you see emerging with customers and how to handle them. customers and how to handle them. customers and how to handle them. >> Yes. uh to be honest maybe this uh >> Yes. uh to be honest maybe this uh >> Yes. uh to be honest maybe this uh review is what what's on top of mind for review is what what's on top of mind for review is what what's on top of mind for majority of technical leaders majority majority of technical leaders majority majority of technical leaders majority of the organizations because uh just I of the organizations because uh just I of the organizations because uh just I mean it's naturally next uh next stage mean it's naturally next uh next stage mean it's naturally next uh next stage after the coding itself was more or less after the coding itself was more or less after the coding itself was more or less solved I would say uh folks if you're solved I would say uh folks if you're solved I would say uh folks if you're comfortable or not if you might agree or comfortable or not if you might agree or comfortable or not if you might agree or or disagree but coding becomes commodity or disagree but coding becomes commodity or disagree but coding becomes commodity again if if if there are many again if if if there are many again if if if there are many disclaimers and exactly this is what we disclaimers and exactly this is what we disclaimers and exactly this is what we exactly talk about right context next exactly talk about right context next exactly talk about right context next layer, proper team structure. But if you layer, proper team structure. But if you layer, proper team structure. But if you do things right, the coding Yeah. is uh do things right, the coding Yeah. is uh do things right, the coding Yeah. is uh like uh something that will worry you like uh something that will worry you like uh something that will worry you less and less. But reviews less and less. But reviews less and less. But reviews >> and um your your first kind of instinct >> and um your your first kind of instinct >> and um your your first kind of instinct could be we have more code we and and could be we have more code we and and could be we have more code we and and this code is u more and more complex.

this code is u more and more complex. this code is u more and more complex. Let's bring more people for reviews. Let's bring more people for reviews. Let's bring more people for reviews. again might uh save you for a very short again might uh save you for a very short again might uh save you for a very short period of time but uh both technically period of time but uh both technically period of time but uh both technically and also uh like uh burnout wise it's um and also uh like uh burnout wise it's um and also uh like uh burnout wise it's um it's a dead end uh and of course I mean it's a dead end uh and of course I mean it's a dead end uh and of course I mean uh I'm not even talking about some some uh I'm not even talking about some some uh I'm not even talking about some some like trivial uh things like obvious like trivial uh things like obvious like trivial uh things like obvious things like uh first uh review by um AI things like uh first uh review by um AI things like uh first uh review by um AI by for example another model. Yeah, I by for example another model. Yeah, I by for example another model. Yeah, I mean it it should be just kind of a mean it it should be just kind of a mean it it should be just kind of a checkbox, right? Uh before human checkbox, right? Uh before human checkbox, right? Uh before human >> like like static analysis or whatever. >> like like static analysis or whatever. >> like like static analysis or whatever. >> Exactly. So I Yeah, I I I just keep this >> Exactly. So I Yeah, I I I just keep this >> Exactly. So I Yeah, I I I just keep this as default these days. as default these days. as default these days. >> But um if you talk about uh reviews >> But um if you talk about uh reviews >> But um if you talk about uh reviews specifically I already mentioned that uh specifically I already mentioned that uh specifically I already mentioned that uh the facts that bugs in AI generated code the facts that bugs in AI generated code the facts that bugs in AI generated code have different let's say failure have different let's say failure have different let's say failure signature. If uh let's say bugs created signature. If uh let's say bugs created signature. If uh let's say bugs created by by us humans could be something uh I by by us humans could be something uh I by by us humans could be something uh I overlooked something I forgot about some overlooked something I forgot about some overlooked something I forgot about some edge case edge case edge case >> there is nothing like that in AI >> there is nothing like that in AI >> there is nothing like that in AI generated code because I mean it will uh generated code because I mean it will uh generated code because I mean it will uh make sure that uh before you see uh pull make sure that uh before you see uh pull make sure that uh before you see uh pull request all tests are green if you if request all tests are green if you if request all tests are green if you if you say that I need more tests you say that I need more tests you say that I need more tests >> AI agent will add as many tests as >> AI agent will add as many tests as >> AI agent will add as many tests as needed right needed right needed right >> agents are good at checklists right >> agents are good at checklists right >> agents are good at checklists right better than humans better than humans better than humans >> yeah but but um Again, it could be >> yeah but but um Again, it could be >> yeah but but um Again, it could be locally perfect or like close to that, locally perfect or like close to that, locally perfect or like close to that, but doesn't follow

but doesn't follow but doesn't follow >> overall uh kind of uh software software >> overall uh kind of uh software software >> overall uh kind of uh software software uh your your project conventions, right? uh your your project conventions, right? uh your your project conventions, right? Uh it could for example pick some Uh it could for example pick some Uh it could for example pick some outdated APIs that someone keeps in um outdated APIs that someone keeps in um outdated APIs that someone keeps in um in some folder for kind of legacy reason in some folder for kind of legacy reason in some folder for kind of legacy reason and everyone knows in the teams that and everyone knows in the teams that and everyone knows in the teams that don't touch this folder but agent don't touch this folder but agent don't touch this folder but agent >> the tribal knowledge, right? agent. >> the tribal knowledge, right? agent. >> the tribal knowledge, right? agent. >> Yes. Yeah. Tribal knowledge. Wow. It's a >> Yes. Yeah. Tribal knowledge. Wow. It's a >> Yes. Yeah. Tribal knowledge. Wow. It's a big big big kind of uh blocker in in big big big kind of uh blocker in in big big big kind of uh blocker in in that sense. So um that sense. So um that sense. So um the I I just want to emphasize that this the I I just want to emphasize that this the I I just want to emphasize that this kind of uh drop more resources approach kind of uh drop more resources approach kind of uh drop more resources approach will not work. What will work is again will not work. What will work is again will not work. What will work is again we can call this constraints we can call this constraints we can call this constraints engineering. Again your f your your your engineering. Again your f your your your engineering. Again your f your your your your first instinct could be like let's your first instinct could be like let's your first instinct could be like let's let's bring more resources. But it's let's bring more resources. But it's let's bring more resources. But it's opposite. You you need to narrow down. opposite. You you need to narrow down. opposite. You you need to narrow down. And basically in today's AI native And basically in today's AI native And basically in today's AI native development era review you you in ideal development era review you you in ideal development era review you you in ideal situation you review not the code but situation you review not the code but situation you review not the code but you review how AI agent obeys these you review how AI agent obeys these you review how AI agent obeys these constraints. How does it follow these uh constraints. How does it follow these uh constraints. How does it follow these uh instructions? Uh does it behave? And um instructions? Uh does it behave? And um instructions? Uh does it behave? And um yeah, that that totally changes the yeah, that that totally changes the yeah, that that totally changes the nature of uh of the reviews. And um yeah nature of uh of the reviews. And um yeah nature of uh of the reviews. And um yeah after after you get some experience with after after you get some experience with after after you get some experience with that you will have much more confidence that you will have much more confidence that you will have much more confidence because if your AI agent follows your because if your AI agent follows your because if your AI agent follows your carefully created and nurtured and carefully created and nurtured and carefully created and nurtured and always up-to-date instructions most always up-to-date instructions most always up-to-date instructions most likely the code is uh again please

likely the code is uh again please likely the code is uh again please understand me right not absolute uh understand me right not absolute uh understand me right not absolute uh definition but the code should be okay definition but the code should be okay definition but the code should be okay and uh yeah so this is where teams and uh yeah so this is where teams and uh yeah so this is where teams should uh should uh strive and like this should uh should uh strive and like this should uh should uh strive and like this is uh could be aspiration. is uh could be aspiration. is uh could be aspiration. >> One thing you've mentioned and which >> One thing you've mentioned and which >> One thing you've mentioned and which we've seen come up a lot with uh we've seen come up a lot with uh we've seen come up a lot with uh developers that we talked to as well is developers that we talked to as well is developers that we talked to as well is like architecture for example that like architecture for example that like architecture for example that that's one of the places that agents that's one of the places that agents that's one of the places that agents often fall down or over complex code often fall down or over complex code often fall down or over complex code like we had one developer who described like we had one developer who described like we had one developer who described this as he used to be a painter with this as he used to be a painter with this as he used to be a painter with code and now he's a sculptor. He's code and now he's a sculptor. He's code and now he's a sculptor. He's getting rid of stuff rather than adding getting rid of stuff rather than adding getting rid of stuff rather than adding in stuff. So how do you think code in stuff. So how do you think code in stuff. So how do you think code review needs to change in terms of the review needs to change in terms of the review needs to change in terms of the emphasis shall we say uh or when human emphasis shall we say uh or when human emphasis shall we say uh or when human developers are doing you know manual developers are doing you know manual developers are doing you know manual review because there will be some review because there will be some review because there will be some probably still some of that like what probably still some of that like what probably still some of that like what they need to concentrate on that maybe they need to concentrate on that maybe they need to concentrate on that maybe they didn't before they didn't before they didn't before >> um again in uh today's AI native >> um again in uh today's AI native >> um again in uh today's AI native reviewing world you check does AI agent reviewing world you check does AI agent reviewing world you check does AI agent follow architecture that you created follow architecture that you created follow architecture that you created that you provided and again back to your that you provided and again back to your that you provided and again back to your statement about tribal knowledge not statement about tribal knowledge not statement about tribal knowledge not everything or what like kind of everyone everything or what like kind of everyone everything or what like kind of everyone knows in the team right and uh knows in the team right and uh knows in the team right and uh >> not everything is uh kind of >> not everything is uh kind of >> not everything is uh kind of externalized and not everything is externalized and not everything is externalized and not everything is machine readable not everything is agent machine readable not everything is agent machine readable not everything is agent consumable and in that sense if you pick consumable and in that sense if you pick consumable and in that sense if you pick any random AI a AI system these days uh any random AI a AI system these days uh any random AI a AI system these days uh there is common something like slash there is common something like slash there is common something like slash init or something like that that will init or something like that that will init or something like that that will that that claims um just run me and I

that that claims um just run me and I that that claims um just run me and I will scan your project you will get uh will scan your project you will get uh will scan your project you will get uh instruction you will get but this will instruction you will get but this will instruction you will get but this will be just uh it could work only as a draft be just uh it could work only as a draft be just uh it could work only as a draft because um the say value of this because um the say value of this because um the say value of this instruction there is some value at least instruction there is some value at least instruction there is some value at least next time agent uh doesn't need to scan next time agent uh doesn't need to scan next time agent uh doesn't need to scan all your codebase right uh but what's all your codebase right uh but what's all your codebase right uh but what's possible to extract from the codebase possible to extract from the codebase possible to extract from the codebase theoretically can be extracted by the theoretically can be extracted by the theoretically can be extracted by the same or different agent on um any any same or different agent on um any any same or different agent on um any any run right so you can keep some of uh run right so you can keep some of uh run right so you can keep some of uh this autogenerated knowledge but put it this autogenerated knowledge but put it this autogenerated knowledge but put it somewhere down and uh the primary um somewhere down and uh the primary um somewhere down and uh the primary um value of your instruction is something value of your instruction is something value of your instruction is something that is not scannable not not possible that is not scannable not not possible that is not scannable not not possible to extract from your existing code and to extract from your existing code and to extract from your existing code and believe me every team in enterprise believe me every team in enterprise believe me every team in enterprise development has lots of this tribal development has lots of this tribal development has lots of this tribal knowledge and like implicit knowledge knowledge and like implicit knowledge knowledge and like implicit knowledge that kind of everyone knows now make that kind of everyone knows now make that kind of everyone knows now make sure that your agent agent knows about sure that your agent agent knows about sure that your agent agent knows about that as well that as well that as well >> by externalizing it and creating an >> by externalizing it and creating an >> by externalizing it and creating an interactive interactive interactive >> totally totally totally so we we see >> totally totally totally so we we see >> totally totally totally so we we see this related to this this explosion of this related to this this explosion of this related to this this explosion of different like prompt libraries and different like prompt libraries and different like prompt libraries and skills registries and like uh maybe skills registries and like uh maybe skills registries and like uh maybe workflows or task templates like where workflows or task templates like where workflows or task templates like where do you think things are going with all do you think things are going with all do you think things are going with all of that stuff? Oh to be honest uh when I of that stuff? Oh to be honest uh when I of that stuff? Oh to be honest uh when I open my uh LinkedIn feed of course I open my uh LinkedIn feed of course I open my uh LinkedIn feed of course I mean I live in my own bubble it's in its mean I live in my own bubble it's in its mean I live in my own bubble it's in its algorithmic feed but I have strong FOMO algorithmic feed but I have strong FOMO algorithmic feed but I have strong FOMO when I see that every single developer when I see that every single developer when I see that every single developer every single team every like every single team every like every single team every like organization large or small is building organization large or small is building organization large or small is building this external harness layer and it's for

this external harness layer and it's for this external harness layer and it's for good reason. Yeah, first of all, uh we good reason. Yeah, first of all, uh we good reason. Yeah, first of all, uh we have like set of uh how to call this have like set of uh how to call this have like set of uh how to call this foundational essential um essential uh foundational essential um essential uh foundational essential um essential uh AI def um tools, right? And this market AI def um tools, right? And this market AI def um tools, right? And this market is maturing, right? So maybe we can name is maturing, right? So maybe we can name is maturing, right? So maybe we can name three, four, five big players uh these three, four, five big players uh these three, four, five big players uh these days, but that's not enough. They even days, but that's not enough. They even days, but that's not enough. They even with incredible pace they develop their with incredible pace they develop their with incredible pace they develop their tools, it's still not enough for for tools, it's still not enough for for tools, it's still not enough for for today's developers. And uh also they today's developers. And uh also they today's developers. And uh also they develop let's say uh the most general develop let's say uh the most general develop let's say uh the most general level right uh but developers teams level right uh but developers teams level right uh but developers teams today they accumulate this uh company today they accumulate this uh company today they accumulate this uh company knowledge and this what becomes true IP knowledge and this what becomes true IP knowledge and this what becomes true IP for the teams because uh what for the teams because uh what for the teams because uh what differentiates your particular team your differentiates your particular team your differentiates your particular team your particular company from from others not particular company from from others not particular company from from others not a dev tool not the model you use because a dev tool not the model you use because a dev tool not the model you use because they are all available right they are all available right they are all available right >> not even anymore >> not even anymore >> not even anymore >> yeah there is no mode uh we can call it >> yeah there is no mode uh we can call it >> yeah there is no mode uh we can call it like like that but your external like like that but your external like like that but your external hardness knowledge Yeah, hardness knowledge Yeah, hardness knowledge Yeah, >> this is what what builds your your kind >> this is what what builds your your kind >> this is what what builds your your kind of confidence and um again what actually of confidence and um again what actually of confidence and um again what actually kind of u boosts uh uh like evaluation kind of u boosts uh uh like evaluation kind of u boosts uh uh like evaluation of your your company and the best part of your your company and the best part of your your company and the best part of that is you can actively use AI to of that is you can actively use AI to of that is you can actively use AI to build these levels of AI. So this is build these levels of AI. So this is build these levels of AI. So this is what uh we can call meta AI. Uh and I what uh we can call meta AI. Uh and I what uh we can call meta AI. Uh and I actively use this myself. I of course I actively use this myself. I of course I actively use this myself. I of course I mean I I try to jump this train uh and I mean I I try to jump this train uh and I mean I I try to jump this train uh and I maintain a set of agent skills myself.

maintain a set of agent skills myself. maintain a set of agent skills myself. I'm big fan of web platform. I spent two I'm big fan of web platform. I spent two I'm big fan of web platform. I spent two decades in building web and I'm big fan decades in building web and I'm big fan decades in building web and I'm big fan of emerging API. So maybe um uh folks of emerging API. So maybe um uh folks of emerging API. So maybe um uh folks remember me as a strong advocate for remember me as a strong advocate for remember me as a strong advocate for progressive web applications service progressive web applications service progressive web applications service workers even though like it became workers even though like it became workers even though like it became commodity and now I I advocate for web commodity and now I I advocate for web commodity and now I I advocate for web AI what's built in in your browser and AI what's built in in your browser and AI what's built in in your browser and enough APIs that you can um leverage to enough APIs that you can um leverage to enough APIs that you can um leverage to get lots of interesting use cases get lots of interesting use cases get lots of interesting use cases straight on your device straight in the straight on your device straight in the straight on your device straight in the browser like web neural networks web MCP browser like web neural networks web MCP browser like web neural networks web MCP plenty of and they all kind of emerging plenty of and they all kind of emerging plenty of and they all kind of emerging right uh they are all drafts in W3C and right uh they are all drafts in W3C and right uh they are all drafts in W3C and uh of course you you don't want to be uh of course you you don't want to be uh of course you you don't want to be just sit you don't want to just sit and just sit you don't want to just sit and just sit you don't want to just sit and wait while they go for uh like wait while they go for uh like wait while they go for uh like recommendation or standard it might take recommendation or standard it might take recommendation or standard it might take years right but u with u progressive years right but u with u progressive years right but u with u progressive enhancement and the feature detection enhancement and the feature detection enhancement and the feature detection yeah you're just good to go web yeah you're just good to go web yeah you're just good to go web solutions so I maintain this a AI agent solutions so I maintain this a AI agent solutions so I maintain this a AI agent skills and I actively use uh for example skills and I actively use uh for example skills and I actively use uh for example agent skill to build agent skills okay I agent skill to build agent skills okay I agent skill to build agent skills okay I actively use agentic workflows to every actively use agentic workflows to every actively use agentic workflows to every Monday it's hands updates in Monday it's hands updates in Monday it's hands updates in specification for me compares what specification for me compares what specification for me compares what exists in current agent skill and my exists in current agent skill and my exists in current agent skill and my only goal is just to accept uh pull only goal is just to accept uh pull only goal is just to accept uh pull requests right or or amend it and uh requests right or or amend it and uh requests right or or amend it and uh yeah I see that uh it's kind of yeah I see that uh it's kind of yeah I see that uh it's kind of avalanche of these dev tools and of avalanche of these dev tools and of avalanche of these dev tools and of course that's really amazing that course that's really amazing that course that's really amazing that majority of them are kind of open source majority of them are kind of open source majority of them are kind of open source they just go and use this uh new fancy they just go and use this uh new fancy they just go and use this uh new fancy harness on top of this or that agent um

harness on top of this or that agent um harness on top of this or that agent um there is set of the skills available for there is set of the skills available for there is set of the skills available for you based on best practices in you based on best practices in you based on best practices in engineering just go and install but um engineering just go and install but um engineering just go and install but um again we had discussion about how to again we had discussion about how to again we had discussion about how to manage this uh AI in FD primitives in manage this uh AI in FD primitives in manage this uh AI in FD primitives in enterprise way right so yeah and again enterprise way right so yeah and again enterprise way right so yeah and again it compounds so it's kind of uh it compounds so it's kind of uh it compounds so it's kind of uh exponential you you use next layers of exponential you you use next layers of exponential you you use next layers of AI to build even better AI on top of AI to build even better AI on top of AI to build even better AI on top of that that that >> uh we call that automations at jet brain >> uh we call that automations at jet brain >> uh we call that automations at jet brain so basically triggered agents that do so basically triggered agents that do so basically triggered agents that do things at a particular schedule or yeah things at a particular schedule or yeah things at a particular schedule or yeah in line with certain uh events but what in line with certain uh events but what in line with certain uh events but what we've heard from some customers we've heard from some customers we've heard from some customers customers though is that uh like you say customers though is that uh like you say customers though is that uh like you say developers get very enthusiastic and developers get very enthusiastic and developers get very enthusiastic and they start creating their own like they start creating their own like they start creating their own like custom agents, custom code review agents custom agents, custom code review agents custom agents, custom code review agents like automations to do all kinds of like automations to do all kinds of like automations to do all kinds of things etc. But the same thing is being things etc. But the same thing is being things etc. But the same thing is being done in seven different teams in the done in seven different teams in the done in seven different teams in the company which is both a kind of waste of company which is both a kind of waste of company which is both a kind of waste of developer time and also you're spending developer time and also you're spending developer time and also you're spending loads of tokens on that. loads of tokens on that. loads of tokens on that. >> So I was wondering if that's something >> So I was wondering if that's something >> So I was wondering if that's something you've seen at the customers you work you've seen at the customers you work you've seen at the customers you work with as well and like how how do you with as well and like how how do you with as well and like how how do you tackle that? Yeah, I mean uh here we we tackle that? Yeah, I mean uh here we we tackle that? Yeah, I mean uh here we we could touch a bit uh return on could touch a bit uh return on could touch a bit uh return on investment, right? And uh licenses for investment, right? And uh licenses for investment, right? And uh licenses for these tools and tokens. Let's be these tools and tokens. Let's be these tools and tokens. Let's be realistic like on on cost center side.

realistic like on on cost center side. realistic like on on cost center side. If you only count this, yeah, it's not If you only count this, yeah, it's not If you only count this, yeah, it's not enough, right? Because you also must enough, right? Because you also must enough, right? Because you also must count and of course invest into for count and of course invest into for count and of course invest into for example building this contextual lay and example building this contextual lay and example building this contextual lay and it takes time. It takes works hours, it takes time. It takes works hours, it takes time. It takes works hours, right? It's not something that comes right? It's not something that comes right? It's not something that comes just okay as as as granted. It's as as just okay as as as granted. It's as as just okay as as as granted. It's as as just default. No, if you do not dedicate just default. No, if you do not dedicate just default. No, if you do not dedicate time, if you don't dedicate kind of time, if you don't dedicate kind of time, if you don't dedicate kind of billable hours, it's just uh again you billable hours, it's just uh again you billable hours, it's just uh again you risk to get mediocre results. There is risk to get mediocre results. There is risk to get mediocre results. There is upskilling and also of course we are not upskilling and also of course we are not upskilling and also of course we are not talking about hey just take this uh talking about hey just take this uh talking about hey just take this uh random course or whatever. No in um if random course or whatever. No in um if random course or whatever. No in um if we talk about right adoption of AI we talk about right adoption of AI we talk about right adoption of AI native dev tools at scale this is whole native dev tools at scale this is whole native dev tools at scale this is whole motion and um it's not end to end motion and um it's not end to end motion and um it's not end to end journey and this is what I call flywheel journey and this is what I call flywheel journey and this is what I call flywheel because uh you have new developers join because uh you have new developers join because uh you have new developers join right uh and uh every developer is uh on right uh and uh every developer is uh on right uh and uh every developer is uh on different stage of uh understanding this different stage of uh understanding this different stage of uh understanding this tool. So you tool. So you tool. So you >> yeah it's it's not a kind of one short >> yeah it's it's not a kind of one short >> yeah it's it's not a kind of one short training it should be whole program but training it should be whole program but training it should be whole program but again uh if you talk about enterprise again uh if you talk about enterprise again uh if you talk about enterprise companies it's not something new for companies it's not something new for companies it's not something new for them I just kindly ask them to to always them I just kindly ask them to to always them I just kindly ask them to to always have something related to AI native dev have something related to AI native dev have something related to AI native dev um in their regular flywheel um so they um in their regular flywheel um so they um in their regular flywheel um so they and most likely there are more points and most likely there are more points and most likely there are more points you need to get on this kind of cost you need to get on this kind of cost you need to get on this kind of cost center side but on profit center how do center side but on profit center how do center side but on profit center how do we measure well there are there are we measure well there are there are we measure well there are there are multiple methodologies but it all starts multiple methodologies but it all starts multiple methodologies but it all starts with uh metrics, right? And with uh metrics, right? And with uh metrics, right? And >> metrics inative development are are >> metrics inative development are are >> metrics inative development are are different compared to say classic different compared to say classic different compared to say classic software development because we want to

software development because we want to software development because we want to focus on focus on focus on >> uh on measurement not uh obligatory >> uh on measurement not uh obligatory >> uh on measurement not uh obligatory software software itself but systems uh software software itself but systems uh software software itself but systems uh you build to build this software and uh you build to build this software and uh you build to build this software and uh it's also easy to get uh trapped into it's also easy to get uh trapped into it's also easy to get uh trapped into some kind of uh vanity metrics we can some kind of uh vanity metrics we can some kind of uh vanity metrics we can call it like that or lowest hanging call it like that or lowest hanging call it like that or lowest hanging fruit ones. fruit ones. fruit ones. I hope that uh lines of the code as I hope that uh lines of the code as I hope that uh lines of the code as metric is more or less anecdotal these metric is more or less anecdotal these metric is more or less anecdotal these days. days. days. >> No, [laughter] >> No, [laughter] >> No, [laughter] >> sadly no still. >> sadly no still. >> sadly no still. >> Uh but seriously uh for example I see >> Uh but seriously uh for example I see >> Uh but seriously uh for example I see that uh lots of companies measure that uh lots of companies measure that uh lots of companies measure um num let's say number of lines or um num let's say number of lines or um num let's say number of lines or number of acceptances of the code number of acceptances of the code number of acceptances of the code activity metrics. activity metrics. activity metrics. >> Yeah. Uh kind of generated by AI. So >> Yeah. Uh kind of generated by AI. So >> Yeah. Uh kind of generated by AI. So what what do you measure by that? You what what do you measure by that? You what what do you measure by that? You measure tool adoption. measure tool adoption. measure tool adoption. >> Could it be interesting? Well, in in >> Could it be interesting? Well, in in >> Could it be interesting? Well, in in some cases, yes, some cases, yes, some cases, yes, >> in the early stages it is, but mo most >> in the early stages it is, but mo most >> in the early stages it is, but mo most companies have that more or less down at companies have that more or less down at companies have that more or less down at this point. this point. this point. >> Yeah. But um again, it's it doesn't say >> Yeah. But um again, it's it doesn't say >> Yeah. But um again, it's it doesn't say about quality of software and especially about quality of software and especially about quality of software and especially quality of the system you build to build quality of the system you build to build quality of the system you build to build the software. So uh today's actual like the software. So uh today's actual like the software. So uh today's actual like relevant metrics could be ratio of uh relevant metrics could be ratio of uh relevant metrics could be ratio of uh human interventions uh let's say the human interventions uh let's say the human interventions uh let's say the less times you need to uh kind of jump less times you need to uh kind of jump less times you need to uh kind of jump in and check how this AI agent behaves in and check how this AI agent behaves in and check how this AI agent behaves along its life cycle the better right along its life cycle the better right along its life cycle the better right and of course you want to see kind of and of course you want to see kind of and of course you want to see kind of downline uh another metric could be kind downline uh another metric could be kind downline uh another metric could be kind of confident time to merge to make sure

of confident time to merge to make sure of confident time to merge to make sure that uh your time from uh creating pull that uh your time from uh creating pull that uh your time from uh creating pull request to like confident merge not request to like confident merge not request to like confident merge not something like we can fix this later is something like we can fix this later is something like we can fix this later is reducing. reducing. reducing. >> Oh, so you're actually fixing anything >> Oh, so you're actually fixing anything >> Oh, so you're actually fixing anything that that that >> Yeah. So, so basically you improve your >> Yeah. So, so basically you improve your >> Yeah. So, so basically you improve your context layer. So AI agents have better context layer. So AI agents have better context layer. So AI agents have better kind of chance to to fail and um like kind of chance to to fail and um like kind of chance to to fail and um like better need for rework and rework itself better need for rework and rework itself better need for rework and rework itself could be another super interesting could be another super interesting could be another super interesting metric right so because I mean uh so metric right so because I mean uh so metric right so because I mean uh so many times especially in the early days many times especially in the early days many times especially in the early days of uh this AI native dev tools we heard of uh this AI native dev tools we heard of uh this AI native dev tools we heard that they do not generate code they that they do not generate code they that they do not generate code they generate technical depth for you. Yes, generate technical depth for you. Yes, generate technical depth for you. Yes, it was true and that's still true if you it was true and that's still true if you it was true and that's still true if you do not don't apply this thinking that we do not don't apply this thinking that we do not don't apply this thinking that we are talking about. Uh and still some are talking about. Uh and still some are talking about. Uh and still some rework might be needed. Again I'm I'm rework might be needed. Again I'm I'm rework might be needed. Again I'm I'm super realistic. I'm uh I'm not you know super realistic. I'm uh I'm not you know super realistic. I'm uh I'm not you know this hyp person that ah everything is uh this hyp person that ah everything is uh this hyp person that ah everything is uh solved. No uh rework might be needed but solved. No uh rework might be needed but solved. No uh rework might be needed but again it could be metric again it could be metric again it could be metric >> uh and the but anyway they they are not >> uh and the but anyway they they are not >> uh and the but anyway they they are not very kind of um still business related very kind of um still business related very kind of um still business related one. So it's a efficiency adoption.

one. So it's a efficiency adoption. one. So it's a efficiency adoption. >> These are still engineering metrics. >> These are still engineering metrics. >> These are still engineering metrics. Yeah. But if uh if you talk about say Yeah. But if uh if you talk about say Yeah. But if uh if you talk about say dollars, euros, um whatever dollars, euros, um whatever dollars, euros, um whatever we so my best recommendation is to we so my best recommendation is to we so my best recommendation is to measure productivity of uh what we call measure productivity of uh what we call measure productivity of uh what we call garden not gardener. So again from garden not gardener. So again from garden not gardener. So again from individual productivity to team individual productivity to team individual productivity to team organization productivity organization productivity organization productivity >> it's always been a best practice which >> it's always been a best practice which >> it's always been a best practice which has kind of been thrown out the window. has kind of been thrown out the window. has kind of been thrown out the window. >> Yeah. Yeah. Yeah. and uh kind of um I >> Yeah. Yeah. Yeah. and uh kind of um I >> Yeah. Yeah. Yeah. and uh kind of um I afraid to sound a bit too generic but afraid to sound a bit too generic but afraid to sound a bit too generic but still in that sense the best metric is still in that sense the best metric is still in that sense the best metric is your own KPIs comparison to baseline your own KPIs comparison to baseline your own KPIs comparison to baseline like how that was before AI and how it like how that was before AI and how it like how that was before AI and how it evolves with AI and there is like no one evolves with AI and there is like no one evolves with AI and there is like no one size fits all for for the companies so size fits all for for the companies so size fits all for for the companies so on like so we drifted a bit from your on like so we drifted a bit from your on like so we drifted a bit from your question right but what I fine question right but what I fine question right but what I fine [laughter] [laughter] [laughter] >> yeah I tend to lot of side step >> yeah I tend to lot of side step >> yeah I tend to lot of side step >> as long as we're talking about >> as long as we're talking about >> as long as we're talking about interesting stuff it's all fine interesting stuff it's all fine interesting stuff it's all fine >> um back to your um initial question >> um back to your um initial question >> um back to your um initial question about uh how companies look at uh like about uh how companies look at uh like about uh how companies look at uh like individual teams experiment. Yes, if uh individual teams experiment. Yes, if uh individual teams experiment. Yes, if uh it's somehow attached to potential it's somehow attached to potential it's somehow attached to potential impact, it's it's totally fine, right?

impact, it's it's totally fine, right? impact, it's it's totally fine, right? And um again in in many cases in And um again in in many cases in And um again in in many cases in majority of the cases experiments of majority of the cases experiments of majority of the cases experiments of team building their like uh new team building their like uh new team building their like uh new experimental external hardness over this experimental external hardness over this experimental external hardness over this like default harness of their AI agent. like default harness of their AI agent. like default harness of their AI agent. uh experimenting with uh different uh experimenting with uh different uh experimenting with uh different interesting ways to manage AI native dev interesting ways to manage AI native dev interesting ways to manage AI native dev primitives could be primitives could be primitives could be uh booster of uh the next steps if not uh booster of uh the next steps if not uh booster of uh the next steps if not the new revenue stream. I mean so many the new revenue stream. I mean so many the new revenue stream. I mean so many so many times we've seen like how so many times we've seen like how so many times we've seen like how company pivoted from uh building their company pivoted from uh building their company pivoted from uh building their their own product but suddenly they their own product but suddenly they their own product but suddenly they started to sell particular pro tool they started to sell particular pro tool they started to sell particular pro tool they built for their product. So it it's so built for their product. So it it's so built for their product. So it it's so white space now. I for me this feels white space now. I for me this feels white space now. I for me this feels like you know early days of NodeJS like like you know early days of NodeJS like like you know early days of NodeJS like people who uh work with JavaScript know people who uh work with JavaScript know people who uh work with JavaScript know that like once this u say framework that like once this u say framework that like once this u say framework platform appeared that was like kind of platform appeared that was like kind of platform appeared that was like kind of zero package right but and and everyone zero package right but and and everyone zero package right but and and everyone had chance to create something very had chance to create something very had chance to create something very essential that will be adopted by essential that will be adopted by essential that will be adopted by millions of developers in very short millions of developers in very short millions of developers in very short time and uh this is my also my strong time and uh this is my also my strong time and uh this is my also my strong call to action to everyone experiment call to action to everyone experiment call to action to everyone experiment faults and externalize. But it also faults and externalize. But it also faults and externalize. But it also sounds like um having a structured way sounds like um having a structured way sounds like um having a structured way to approach building this uh to approach building this uh to approach building this uh organization or team level harness you organization or team level harness you organization or team level harness you could say including the context could say including the context could say including the context engineering is crucial engineering is crucial engineering is crucial >> and that probably can't be done entirely >> and that probably can't be done entirely >> and that probably can't be done entirely like bottom up right you you probably like bottom up right you you probably like bottom up right you you probably need some support top down as well need some support top down as well need some support top down as well because people need time people need because people need time people need because people need time people need like like like >> yes yes yes I'm I'm a big fan of uh >> yes yes yes I'm I'm a big fan of uh >> yes yes yes I'm I'm a big fan of uh change management course by pro and uh

change management course by pro and uh change management course by pro and uh I'm super happy that I went through this I'm super happy that I went through this I'm super happy that I went through this course before this kind of big big big course before this kind of big big big course before this kind of big big big uh hockey stick uh hockey stick uh hockey stick >> because it all still applies right >> because it all still applies right >> because it all still applies right >> because it's it's >> because it's it's >> because it's it's >> a change management change management >> a change management change management >> a change management change management absolutely this is not technological absolutely this is not technological absolutely this is not technological problem it's a change management problem problem it's a change management problem problem it's a change management problem and uh what what kind of uh footprinted and uh what what kind of uh footprinted and uh what what kind of uh footprinted in my head from this course is main in my head from this course is main in my head from this course is main factor that affects uh that directly factor that affects uh that directly factor that affects uh that directly drives success of the change is drives success of the change is drives success of the change is executive sponsorship. executive sponsorship. executive sponsorship. >> Yeah. >> Yeah. >> Yeah. >> And uh it's not just buying pizzas uh >> And uh it's not just buying pizzas uh >> And uh it's not just buying pizzas uh for some evening event. No, not that for some evening event. No, not that for some evening event. No, not that level. full commitment, full level. full commitment, full level. full commitment, full communicating, full transparency uh from communicating, full transparency uh from communicating, full transparency uh from um technical leaders um technical leaders um technical leaders >> and I would say trust as well because >> and I would say trust as well because >> and I would say trust as well because this is something that seems to be this is something that seems to be this is something that seems to be emerging as a problem. Distrust between emerging as a problem. Distrust between emerging as a problem. Distrust between engineering and maybe you know uh engineering and maybe you know uh engineering and maybe you know uh non-technical management especially once non-technical management especially once non-technical management especially once the AI spend goes up the AI spend goes up the AI spend goes up >> that you still need to trust your >> that you still need to trust your >> that you still need to trust your engineering team that they will use engineering team that they will use engineering team that they will use their resources wisely. Yes. their resources wisely. Yes. their resources wisely. Yes. >> Yes. And and this is uh multi-layered. >> Yes. And and this is uh multi-layered. >> Yes. And and this is uh multi-layered. So if you talk about trust about So if you talk about trust about So if you talk about trust about governance of these tools governance of these tools governance of these tools >> um also could easy like there are like >> um also could easy like there are like >> um also could easy like there are like two two sides of the spectrum like two two sides of the spectrum like two two sides of the spectrum like everything is prohibited right you only everything is prohibited right you only everything is prohibited right you only have like this kind of mediocre model have like this kind of mediocre model have like this kind of mediocre model available there is no way for you to use available there is no way for you to use available there is no way for you to use MCP servers there is no for you to use MCP servers there is no for you to use MCP servers there is no for you to use uh CLI just because we are a bit a bit uh CLI just because we are a bit a bit uh CLI just because we are a bit a bit careful about all these things so this careful about all these things so this careful about all these things so this is one side of of the story second we is one side of of the story second we is one side of of the story second we don't have any any policies at all hey don't have any any policies at all hey don't have any any policies at all hey folks bring your own subscription of uh folks bring your own subscription of uh folks bring your own subscription of uh random tool. Bring your free

random tool. Bring your free random tool. Bring your free subscription. Um use anything and um subscription. Um use anything and um subscription. Um use anything and um this will not last for for too long this will not last for for too long this will not last for for too long before like something bad happens, before like something bad happens, before like something bad happens, right? So governance um is is crucial right? So governance um is is crucial right? So governance um is is crucial and of course it should not be burden on and of course it should not be burden on and of course it should not be burden on the shoulders of developers. the shoulders of developers. the shoulders of developers. >> No, they should take about it most of >> No, they should take about it most of >> No, they should take about it most of the time. security engineers, CESOs need the time. security engineers, CESOs need the time. security engineers, CESOs need to have full clarity, full understanding to have full clarity, full understanding to have full clarity, full understanding and be 100% transparent on uh what's and be 100% transparent on uh what's and be 100% transparent on uh what's available and why available and why is available and why available and why is available and why available and why is something is is not available. something is is not available. something is is not available. >> Uh and uh I believe today any AIM >> Uh and uh I believe today any AIM >> Uh and uh I believe today any AIM assistant on the market uh has enough assistant on the market uh has enough assistant on the market uh has enough knows to enable disable things. knows to enable disable things. knows to enable disable things. >> Um so yes, so I call this uh guard rails >> Um so yes, so I call this uh guard rails >> Um so yes, so I call this uh guard rails journey without blockers. So every journey without blockers. So every journey without blockers. So every single individual developer should not single individual developer should not single individual developer should not uh worry too much about they do uh worry too much about they do uh worry too much about they do something wrong because they this will something wrong because they this will something wrong because they this will stop experimentation stop experimentation stop experimentation >> because also then they don't have their >> because also then they don't have their >> because also then they don't have their personal responsibility of making some personal responsibility of making some personal responsibility of making some mistake that they didn't know about mistake that they didn't know about mistake that they didn't know about >> and accountability and responsibility. >> and accountability and responsibility. >> and accountability and responsibility. This is something that changed This is something that changed This is something that changed completely again compared to classic completely again compared to classic completely again compared to classic software development. Uh because this is software development. Uh because this is software development. Uh because this is kind of blurred across uh multiple kind of blurred across uh multiple kind of blurred across uh multiple layers now, right? And if you talk about layers now, right? And if you talk about layers now, right? And if you talk about uh this specific part of governance that uh this specific part of governance that uh this specific part of governance that is compliancy and all these you know is compliancy and all these you know is compliancy and all these you know audits because what was uh audits because what was uh audits because what was uh >> knowing exactly what an agent did and >> knowing exactly what an agent did and >> knowing exactly what an agent did and >> Yeah. Yeah. So in old gold times uh at >> Yeah. Yeah. So in old gold times uh at >> Yeah. Yeah. So in old gold times uh at every single uh kind kind of decision every single uh kind kind of decision every single uh kind kind of decision point there was a human right and uh if point there was a human right and uh if point there was a human right and uh if uh auditors come to I don't know certify

uh auditors come to I don't know certify uh auditors come to I don't know certify your company for for this certification your company for for this certification your company for for this certification yeah it's audit trail is clearly visible yeah it's audit trail is clearly visible yeah it's audit trail is clearly visible now AI agent again if we think about now AI agent again if we think about now AI agent again if we think about this not as a tool not like a llinter this not as a tool not like a llinter this not as a tool not like a llinter right but it behaves like a decision right but it behaves like a decision right but it behaves like a decision maker in many cases maker in many cases maker in many cases >> yes >> yes >> yes >> and uh if you don't have this u >> and uh if you don't have this u >> and uh if you don't have this u governance layer explained meaning why governance layer explained meaning why governance layer explained meaning why this AI agent took this decision because this AI agent took this decision because this AI agent took this decision because after all uh you know there there is after all uh you know there there is after all uh you know there there is human right who wrote this instruction human right who wrote this instruction human right who wrote this instruction etc and if there is no audit trail to to etc and if there is no audit trail to to etc and if there is no audit trail to to that level you're in trouble again um that level you're in trouble again um that level you're in trouble again um there is no no one no one could say uh there is no no one no one could say uh there is no no one no one could say uh the the simplest part is why AI agent the the simplest part is why AI agent the the simplest part is why AI agent generated this code it's it's the the generated this code it's it's the the generated this code it's it's the the simplest right problem but uh how it simplest right problem but uh how it simplest right problem but uh how it could pass the review like where is could pass the review like where is could pass the review like where is responsibility was it AI agent uh that responsibility was it AI agent uh that responsibility was it AI agent uh that uh generated this wrong code or was it uh generated this wrong code or was it uh generated this wrong code or was it harness related to review? We have harness related to review? We have harness related to review? We have special we could have special special we could have special special we could have special instructions for review itself and yeah instructions for review itself and yeah instructions for review itself and yeah so there a lot whole layer of problems so there a lot whole layer of problems so there a lot whole layer of problems just because of accountability is not just because of accountability is not just because of accountability is not clear these days clear these days clear these days >> and of course this is also feedback that >> and of course this is also feedback that >> and of course this is also feedback that you can use to improve your you can use to improve your you can use to improve your organizational harness layer or whatever organizational harness layer or whatever organizational harness layer or whatever you you you >> absolutely yes by the way you you you >> absolutely yes by the way you you you >> absolutely yes by the way you you you mentioned uh something that I really mentioned uh something that I really mentioned uh something that I really want to wanted to communicate uh and if want to wanted to communicate uh and if want to wanted to communicate uh and if you look at this um adoption curve right you look at this um adoption curve right you look at this um adoption curve right what I see as a pattern across multiple what I see as a pattern across multiple what I see as a pattern across multiple teams this is classic J curve teams this is classic J curve teams this is classic J curve Initially if it compare like the Initially if it compare like the Initially if it compare like the performance and timeline right if we uh

performance and timeline right if we uh performance and timeline right if we uh initial excitement right and it's over initial excitement right and it's over initial excitement right and it's over baseline performance yeah your first baseline performance yeah your first baseline performance yeah your first prompt u created some new feature you prompt u created some new feature you prompt u created some new feature you created lots of tests and uh kind of you created lots of tests and uh kind of you created lots of tests and uh kind of you feel productivity boost your developer feel productivity boost your developer feel productivity boost your developer >> you hate writing tests so you don't have >> you hate writing tests so you don't have >> you hate writing tests so you don't have to anyway so perfect uh let's let's like to anyway so perfect uh let's let's like to anyway so perfect uh let's let's like full green light to that and this full green light to that and this full green light to that and this initial like excitement stage could last initial like excitement stage could last initial like excitement stage could last for uh several weeks but it's all for uh several weeks but it's all for uh several weeks but it's all compressing these days right and of compressing these days right and of compressing these days right and of course depending on the team it could course depending on the team it could course depending on the team it could last for for only only several days but last for for only only several days but last for for only only several days but then you hit kind of well and this is then you hit kind of well and this is then you hit kind of well and this is where your AI agent hits your real code where your AI agent hits your real code where your AI agent hits your real code base and there are so many things that base and there are so many things that base and there are so many things that could go wrong just because uh you don't could go wrong just because uh you don't could go wrong just because uh you don't have this context engineering layer like have this context engineering layer like have this context engineering layer like things get messy at that point things get messy at that point things get messy at that point mature right um again uh some implicit mature right um again uh some implicit mature right um again uh some implicit conventions some uh all their APIs that conventions some uh all their APIs that conventions some uh all their APIs that it's somewhere in the project but no one it's somewhere in the project but no one it's somewhere in the project but no one uses them uh plenty of things and then uses them uh plenty of things and then uses them uh plenty of things and then well it's kind of uh inflection point well it's kind of uh inflection point well it's kind of uh inflection point right at the very bottom of course if right at the very bottom of course if right at the very bottom of course if you stop here and revert back to either you stop here and revert back to either you stop here and revert back to either I don't know code suggestions or chat or I don't know code suggestions or chat or I don't know code suggestions or chat or just remove AI completely just because just remove AI completely just because just remove AI completely just because it failed but it's not AI agent failed it failed but it's not AI agent failed it failed but it's not AI agent failed it's not model failed it's your context it's not model failed it's your context it's not model failed it's your context failed exactly failed exactly failed exactly >> failed so how to how to how to get uh >> failed so how to how to how to get uh >> failed so how to how to how to get uh this to to this uptrend again is to not this to to this uptrend again is to not this to to this uptrend again is to not only fix the code that AI agent only fix the code that AI agent only fix the code that AI agent generated but fix the hardness. So I I

generated but fix the hardness. So I I generated but fix the hardness. So I I kind of develop my personal pattern if I kind of develop my personal pattern if I kind of develop my personal pattern if I see that AI agent drifts from what I see that AI agent drifts from what I see that AI agent drifts from what I expect it to do. I do not ask I do not expect it to do. I do not ask I do not expect it to do. I do not ask I do not only ask to regenerate the code. I ask only ask to regenerate the code. I ask only ask to regenerate the code. I ask regenerate instruction and sometimes uh regenerate instruction and sometimes uh regenerate instruction and sometimes uh if I'm not lazy I ask like hey what is if I'm not lazy I ask like hey what is if I'm not lazy I ask like hey what is the root cause? But in many cases I the root cause? But in many cases I the root cause? But in many cases I suggest hey just improve this context suggest hey just improve this context suggest hey just improve this context layer. So next time you will not hit layer. So next time you will not hit layer. So next time you will not hit that and with this approach so it's kind that and with this approach so it's kind that and with this approach so it's kind of failure for diagnostics approach uh of failure for diagnostics approach uh of failure for diagnostics approach uh it again it will not uh you will not go it again it will not uh you will not go it again it will not uh you will not go up immediately after just two days but up immediately after just two days but up immediately after just two days but at some time point you will evolve to at some time point you will evolve to at some time point you will evolve to critical mass of these improvements and critical mass of these improvements and critical mass of these improvements and then kind of hockey stick and then kind of hockey stick and then kind of hockey stick and exponential. So given this new exponential. So given this new exponential. So given this new environment where you know developers environment where you know developers environment where you know developers are not just uh engineering the software are not just uh engineering the software are not just uh engineering the software but they're engineering the system that but they're engineering the system that but they're engineering the system that makes the software including the context makes the software including the context makes the software including the context layer like how do you see the profession layer like how do you see the profession layer like how do you see the profession changing in the next year or two? changing in the next year or two? changing in the next year or two? >> It's a big question. I I I believe um >> It's a big question. I I I believe um >> It's a big question. I I I believe um 100% of developers already feel right 100% of developers already feel right 100% of developers already feel right and um already started to to adapt to to and um already started to to adapt to to and um already started to to adapt to to that and again uh flexibility is 100% uh that and again uh flexibility is 100% uh that and again uh flexibility is 100% uh important in that sense and uh to some important in that sense and uh to some important in that sense and uh to some extent rewiring brain and uh being open extent rewiring brain and uh being open extent rewiring brain and uh being open after all right so uh I I I mean I I've after all right so uh I I I mean I I've after all right so uh I I I mean I I've seen lots of skeptics right and my my seen lots of skeptics right and my my seen lots of skeptics right and my my full respect to people who who still say full respect to people who who still say full respect to people who who still say that uh AI is a slope And I mean they that uh AI is a slope And I mean they that uh AI is a slope And I mean they they don't want to use that. My only ask

they don't want to use that. My only ask they don't want to use that. My only ask to these people is try it reconsider to these people is try it reconsider to these people is try it reconsider because uh what what I hear also very because uh what what I hear also very because uh what what I hear also very often that there was quality leap in the often that there was quality leap in the often that there was quality leap in the model development right not not even model development right not not even model development right not not even saying about context layer but just the saying about context layer but just the saying about context layer but just the models uh somewhere around uh OPUS 4.5 models uh somewhere around uh OPUS 4.5 models uh somewhere around uh OPUS 4.5 release right this is where lots of release right this is where lots of release right this is where lots of skeptics those who reconsider it yes skeptics those who reconsider it yes skeptics those who reconsider it yes started to finally understand benefits started to finally understand benefits started to finally understand benefits but um but how how The role is u but um but how how The role is u but um but how how The role is u evolving. I think I already mentioned evolving. I think I already mentioned evolving. I think I already mentioned few things that uh few things that uh few things that uh >> these new roles you mentioned three new >> these new roles you mentioned three new >> these new roles you mentioned three new roles. roles. roles. >> Yeah. Not obligatory roles but >> Yeah. Not obligatory roles but >> Yeah. Not obligatory roles but responsibilities for every software responsibilities for every software responsibilities for every software developer. developer. developer. >> Technical writing is becoming crucial >> Technical writing is becoming crucial >> Technical writing is becoming crucial skill. Okay. But again yeah again not skill. Okay. But again yeah again not skill. Okay. But again yeah again not everyone is comfortable with that. everyone is comfortable with that. everyone is comfortable with that. >> Many developers hate doing any kind of >> Many developers hate doing any kind of >> Many developers hate doing any kind of documentation. documentation. documentation. >> I again yes I'm I'm not a big fan of of >> I again yes I'm I'm not a big fan of of >> I again yes I'm I'm not a big fan of of that myself but that's becomes that myself but that's becomes that myself but that's becomes inevitable. inevitable. inevitable. >> But it's essential for agents. Yes. >> But it's essential for agents. Yes. >> But it's essential for agents. Yes. >> Yes. Yes. Yes. And also again this is >> Yes. Yes. Yes. And also again this is >> Yes. Yes. Yes. And also again this is very specific technical writing right very specific technical writing right very specific technical writing right basically not obligatory for us but but basically not obligatory for us but but basically not obligatory for us but but for agents uh and again this is a skill for agents uh and again this is a skill for agents uh and again this is a skill uh you you you do not get your first uh uh you you you do not get your first uh uh you you you do not get your first uh specification um in optimal shape from specification um in optimal shape from specification um in optimal shape from the first try. most likely your first, the first try. most likely your first, the first try. most likely your first, second, third will just be let's say second, third will just be let's say second, third will just be let's say suboptimal to be gentle. Um and yeah suboptimal to be gentle. Um and yeah suboptimal to be gentle. Um and yeah also this um agent orchestration is also this um agent orchestration is also this um agent orchestration is becoming uh very important maybe um uh becoming uh very important maybe um uh becoming uh very important maybe um uh we can call this as a part of context

we can call this as a part of context we can call this as a part of context engineering uh or as a as a engineering uh or as a as a engineering uh or as a as a self-standing discipline uh because uh self-standing discipline uh because uh self-standing discipline uh because uh with evolving of large language models with evolving of large language models with evolving of large language models now we kind of uh we try to build whole now we kind of uh we try to build whole now we kind of uh we try to build whole agentic loop right not just uh hey uh agentic loop right not just uh hey uh agentic loop right not just uh hey uh this is my implement this feature and this is my implement this feature and this is my implement this feature and then I I will review it. Uh we want to then I I will review it. Uh we want to then I I will review it. Uh we want to have everything in in this loop. So have everything in in this loop. So have everything in in this loop. So basically again back to we are becoming basically again back to we are becoming basically again back to we are becoming system designers of not the software but system designers of not the software but system designers of not the software but the software factory that will be this the software factory that will be this the software factory that will be this software. So yeah this is uh super software. So yeah this is uh super software. So yeah this is uh super interesting skill interesting skill interesting skill >> and I'm not saying that uh tool link is >> and I'm not saying that uh tool link is >> and I'm not saying that uh tool link is perfect these days but again might be perfect these days but again might be perfect these days but again might be wide space for lots of startups. Uh so wide space for lots of startups. Uh so wide space for lots of startups. Uh so we we we suddenly got this Ralph loop we we we suddenly got this Ralph loop we we we suddenly got this Ralph loop taking uh you know our planet's uh taking uh you know our planet's uh taking uh you know our planet's uh message by storm message by storm message by storm >> by storm. Yes. Uh and there is no kind >> by storm. Yes. Uh and there is no kind >> by storm. Yes. Uh and there is no kind of uh kind of commercial product that is of uh kind of commercial product that is of uh kind of commercial product that is Ralph loop. So they're all kind of Ralph loop. So they're all kind of Ralph loop. So they're all kind of plugins for uh for this one. So maybe plugins for uh for this one. So maybe plugins for uh for this one. So maybe >> it's more like a design pattern as we >> it's more like a design pattern as we >> it's more like a design pattern as we discussed.

discussed. discussed. >> Yeah. Yeah. Yeah. So but uh may maybe >> Yeah. Yeah. Yeah. So but uh may maybe >> Yeah. Yeah. Yeah. So but uh may maybe again could be interesting option for again could be interesting option for again could be interesting option for someone's next startup next uh venture. someone's next startup next uh venture. someone's next startup next uh venture. Um yes and uh writing specifications Um yes and uh writing specifications Um yes and uh writing specifications working orchestrating patterns are are working orchestrating patterns are are working orchestrating patterns are are becoming inevitable skills and also as I becoming inevitable skills and also as I becoming inevitable skills and also as I mentioned um in some larger mentioned um in some larger mentioned um in some larger organizations there could be separate organizations there could be separate organizations there could be separate job titles like context engineer uh or job titles like context engineer uh or job titles like context engineer uh or agent operator because uh again there is agent operator because uh again there is agent operator because uh again there is so many new things new new totally new so many new things new new totally new so many new things new new totally new layers of abstraction of on top of that layers of abstraction of on top of that layers of abstraction of on top of that appears that not uh obligatory should be appears that not uh obligatory should be appears that not uh obligatory should be uh on the shoulders of uh of developers, uh on the shoulders of uh of developers, uh on the shoulders of uh of developers, right? right? right? >> Or it becomes, you know, uh just a >> Or it becomes, you know, uh just a >> Or it becomes, you know, uh just a normal part of the role of a staff normal part of the role of a staff normal part of the role of a staff engineer engineer engineer >> also also might be the case especially >> also also might be the case especially >> also also might be the case especially for senior ones. for senior ones. for senior ones. >> So from working with all of these like >> So from working with all of these like >> So from working with all of these like hundred different large enterprise hundred different large enterprise hundred different large enterprise customers like is what's the one thing customers like is what's the one thing customers like is what's the one thing you wish every team lead who is trying you wish every team lead who is trying you wish every team lead who is trying to go agent native new that you can only to go agent native new that you can only to go agent native new that you can only really learn from that kind of really learn from that kind of really learn from that kind of experience that you have?

experience that you have? experience that you have? >> Uh I I think here we go back to to >> Uh I I think here we go back to to >> Uh I I think here we go back to to context engineering. So to perceive this context engineering. So to perceive this context engineering. So to perceive this as uh one of the main activities you as uh one of the main activities you as uh one of the main activities you need to drive here now because if you need to drive here now because if you need to drive here now because if you just uh sit and wait um it's it's not just uh sit and wait um it's it's not just uh sit and wait um it's it's not neutral it's uh it's negative because neutral it's uh it's negative because neutral it's uh it's negative because your competitors are actively building your competitors are actively building your competitors are actively building this layer and this is what this layer and this is what this layer and this is what differentiates their team their product differentiates their team their product differentiates their team their product from uh from your from uh from your from uh from your >> okay I think that's a great point in >> okay I think that's a great point in >> okay I think that's a great point in which to finish. So thank you very much which to finish. So thank you very much which to finish. So thank you very much Maxim and thank you to everyone for Maxim and thank you to everyone for Maxim and thank you to everyone for joining us. joining us. joining us. >> My pleasure.