How to Get Your Org to Adopt Coding Agents (Without Shipping Garbage) — Eyal Blum, Figma

>> Good afternoon. My name is Alon Blum. I >> Good afternoon. My name is Alon Blum. I am a software engineer at Figma. am a software engineer at Figma. am a software engineer at Figma. And in my talk today, we're going to And in my talk today, we're going to And in my talk today, we're going to talk about how we've adopted or are talk about how we've adopted or are talk about how we've adopted or are adopting adopting adopting agent into our workflow at Figma agent into our workflow at Figma agent into our workflow at Figma while maintaining while maintaining while maintaining high quality for our code base. high quality for our code base. high quality for our code base. So, as you may know, Figma is the So, as you may know, Figma is the So, as you may know, Figma is the browser-based browser-based browser-based editor where design and engineering and editor where design and engineering and editor where design and engineering and now AI agent collaborate together to now AI agent collaborate together to now AI agent collaborate together to ship code. ship code. ship code. Uh this Uh this Uh this Figma has Figma has Figma has pivoted very strongly from being a pivoted very strongly from being a pivoted very strongly from being a traditional tool to an AI-first tool. traditional tool to an AI-first tool. traditional tool to an AI-first tool. But in this talk, I'm not going to talk But in this talk, I'm not going to talk But in this talk, I'm not going to talk about our product. I'm going to talk about our product. I'm going to talk about our product. I'm going to talk more about our internal organization and more about our internal organization and more about our internal organization and how our engineering org has been how our engineering org has been how our engineering org has been adopting AI agents. Um what we we found internally is both Um what we we found internally is both organizations, companies, and individual organizations, companies, and individual organizations, companies, and individual there's kind of a three-act there's kind of a three-act there's kind of a three-act process of AI adoption.

process of AI adoption. process of AI adoption. You start with picking up something, You start with picking up something, You start with picking up something, whether it was a lot of the people in whether it was a lot of the people in whether it was a lot of the people in this room who have been doing using our this room who have been doing using our this room who have been doing using our AI pal and have been using AI for a AI pal and have been using AI for a AI pal and have been using AI for a while and they picked up something and while and they picked up something and while and they picked up something and got some simple things to work very got some simple things to work very got some simple things to work very well. well. well. 10x faster. 10x faster. 10x faster. Then you start applying those same Then you start applying those same Then you start applying those same practices to bigger problems and AI practices to bigger problems and AI practices to bigger problems and AI fails pretty badly at that, gives you fails pretty badly at that, gives you fails pretty badly at that, gives you bad stuff, bad stuff, bad stuff, lots of bugs, lots of bugs, lots of bugs, and the trust that you build breaks and the trust that you build breaks and the trust that you build breaks down. down. down. And then And then And then from that point, you start building the from that point, you start building the from that point, you start building the real skill, which is learning how to use real skill, which is learning how to use real skill, which is learning how to use AI correctly and put the right AI correctly and put the right AI correctly and put the right guardrails and the right prompting and guardrails and the right prompting and guardrails and the right prompting and the right context and all the stuff that the right context and all the stuff that the right context and all the stuff that we've been talking all day about here in we've been talking all day about here in we've been talking all day about here in all the talks in order to actually build all the talks in order to actually build all the talks in order to actually build a real scale. a real scale. a real scale. And one thing that And one thing that And one thing that is happening internally as we we adopted is happening internally as we we adopted is happening internally as we we adopted whether teams or individuals whether teams or individuals whether teams or individuals the adoption is uneven. We have teams the adoption is uneven. We have teams the adoption is uneven. We have teams that are very AI forward and have that are very AI forward and have that are very AI forward and have already transformed their entire already transformed their entire already transformed their entire workflows and then we have teams that workflows and then we have teams that workflows and then we have teams that are still experimenting in the earlier are still experimenting in the earlier are still experimenting in the earlier act and or have lost confidence and they act and or have lost confidence and they act and or have lost confidence and they all need to work together in order to all need to work together in order to all need to work together in order to ship our product. Um ship our product. Um ship our product. Um So, they need to coexist So, they need to coexist So, they need to coexist in the organization and we need to find in the organization and we need to find in the organization and we need to find a way to support them and while bringing a way to support them and while bringing a way to support them and while bringing on everybody along for the journey and on everybody along for the journey and on everybody along for the journey and getting everybody to the third act of getting everybody to the third act of getting everybody to the third act of the story.

Aside from that main friction point, we Aside from that main friction point, we have also noticed other friction points have also noticed other friction points have also noticed other friction points that happened that happened that happened as we adopt AI. One thing that we've heard a lot from One thing that we've heard a lot from developers and managers have have been developers and managers have have been developers and managers have have been noticing is that reduced developer noticing is that reduced developer noticing is that reduced developer agency causes agency causes agency causes um um um engineers to lose some of their job engineers to lose some of their job engineers to lose some of their job satisfaction. So, if a lot of people satisfaction. So, if a lot of people satisfaction. So, if a lot of people used to take a lot of pride and used to take a lot of pride and used to take a lot of pride and enjoyment in writing code and getting enjoyment in writing code and getting enjoyment in writing code and getting into the flow into the flow into the flow and a lot of people feel like that's and a lot of people feel like that's and a lot of people feel like that's been lost or they're losing a lot of been lost or they're losing a lot of been lost or they're losing a lot of that that that element and getting into more of a element and getting into more of a element and getting into more of a prompt cycle where they just wait on prompt cycle where they just wait on prompt cycle where they just wait on output from AI and then speak to the AI output from AI and then speak to the AI output from AI and then speak to the AI that like not as much fun as they used that like not as much fun as they used that like not as much fun as they used to have and they're getting burned out. to have and they're getting burned out. to have and they're getting burned out. Um we've noticed another interesting Um we've noticed another interesting Um we've noticed another interesting thing. It's actually our best engineer, thing. It's actually our best engineer, thing. It's actually our best engineer, the one that hold all their contacts in the one that hold all their contacts in the one that hold all their contacts in their brain. Um they end up getting out their brain. Um they end up getting out their brain. Um they end up getting out of the burden. What ends up happening is of the burden. What ends up happening is of the burden. What ends up happening is they they know where all the pitfalls they they know where all the pitfalls they they know where all the pitfalls are. They are like holding together with are. They are like holding together with are. They are like holding together with with like their mental duct tape all the with like their mental duct tape all the with like their mental duct tape all the places places places that agents are not working well and that agents are not working well and that agents are not working well and they're preventing all the really bad they're preventing all the really bad they're preventing all the really bad stuff from coming in or all they they stuff from coming in or all they they stuff from coming in or all they they have all the institutional contact that have all the institutional contact that have all the institutional contact that have never written down in their head have never written down in their head have never written down in their head and they get so much burden and and and they get so much burden and and and they get so much burden and and become bottlenecks and gets really become bottlenecks and gets really become bottlenecks and gets really frustrated. So, they actually end up frustrated. So, they actually end up frustrated. So, they actually end up being slowest to adopt because they see being slowest to adopt because they see being slowest to adopt because they see all the problem all the problem all the problem uh first hand.

uh first hand. uh first hand. That's another big big issue that we've That's another big big issue that we've That's another big big issue that we've seen. seen. seen. Um and this one I'm sure everybody can Um and this one I'm sure everybody can Um and this one I'm sure everybody can resonate or in Sorry, I'm sure everybody resonate or in Sorry, I'm sure everybody resonate or in Sorry, I'm sure everybody here will resonate. here will resonate. here will resonate. Um that all of a sudden all the design Um that all of a sudden all the design Um that all of a sudden all the design docs and all the Slack messages you docs and all the Slack messages you docs and all the Slack messages you know, this the emails have gotten three know, this the emails have gotten three know, this the emails have gotten three or four times as long and we've gotten or four times as long and we've gotten or four times as long and we've gotten two or three times as many emails two or three times as many emails two or three times as many emails and they say basically as much as they and they say basically as much as they and they say basically as much as they did before. So, communication has gotten did before. So, communication has gotten did before. So, communication has gotten quite inefficient and some of the quite inefficient and some of the quite inefficient and some of the markers of like what is high quality and markers of like what is high quality and markers of like what is high quality and important things versus not so much high important things versus not so much high important things versus not so much high quality quality quality um has become challenging to navigate. um has become challenging to navigate. um has become challenging to navigate. Um so, I'm going to spend uh the next Um so, I'm going to spend uh the next Um so, I'm going to spend uh the next few minutes talking about some of the few minutes talking about some of the few minutes talking about some of the lessons that we've learned and how we've lessons that we've learned and how we've lessons that we've learned and how we've been trying to apply this. This is a been trying to apply this. This is a been trying to apply this. This is a journey. We have not come out through journey. We have not come out through journey. We have not come out through the other end, but we've seen some the other end, but we've seen some the other end, but we've seen some really interesting progress really interesting progress really interesting progress along a lot of these lines. Um I think this Um I think this uh a lot of the speakers here have uh a lot of the speakers here have uh a lot of the speakers here have touched upon this, but investing in touched upon this, but investing in touched upon this, but investing in verification is probably the highest verification is probably the highest verification is probably the highest value thing we can do in our code base.

value thing we can do in our code base. value thing we can do in our code base. Um anytime that we can lift a Um anytime that we can lift a Um anytime that we can lift a left shift anything in our workflow from left shift anything in our workflow from left shift anything in our workflow from a human needing to do it to an agent a human needing to do it to an agent a human needing to do it to an agent being able to verify it. being able to verify it. being able to verify it. So, for example, when uh Playwright and So, for example, when uh Playwright and So, for example, when uh Playwright and MCP came out, instead of having humans MCP came out, instead of having humans MCP came out, instead of having humans navigate the code, now the agent can navigate the code, now the agent can navigate the code, now the agent can explore the code. That was a big win explore the code. That was a big win explore the code. That was a big win unlock for productivity in a lot of our unlock for productivity in a lot of our unlock for productivity in a lot of our team. That's really That's always a team. That's really That's always a team. That's really That's always a a big win for us. a big win for us. a big win for us. The other thing is The other thing is The other thing is um um um it's even better if when you find it's even better if when you find it's even better if when you find something that the agent has found to be something that the agent has found to be something that the agent has found to be useful, useful, useful, take the time to take that and encode take the time to take that and encode take the time to take that and encode into a deterministic flow. into a deterministic flow. into a deterministic flow. A deterministic flow that can be easily A deterministic flow that can be easily A deterministic flow that can be easily repeated is saved on tokens, save on repeated is saved on tokens, save on repeated is saved on tokens, save on time for the and then it also you also time for the and then it also you also time for the and then it also you also know that you're using the the LLM when know that you're using the the LLM when know that you're using the the LLM when it needs to reason, but when you have it needs to reason, but when you have it needs to reason, but when you have something that is already something that is already something that is already known and basically can be encoded into known and basically can be encoded into known and basically can be encoded into a test, spending that time always always a test, spending that time always always a test, spending that time always always pays dividends.

pays dividends. pays dividends. Um Um Um And another tip, if you tell your scale And another tip, if you tell your scale And another tip, if you tell your scale your agent to write the code that you're your agent to write the code that you're your agent to write the code that you're writing writing writing um like at the red to green to red to um like at the red to green to red to um like at the red to green to red to green at the TDD style, green at the TDD style, green at the TDD style, it almost always gives you better it almost always gives you better it almost always gives you better results because you set a goal, then you results because you set a goal, then you results because you set a goal, then you tell the agent to strive toward that tell the agent to strive toward that tell the agent to strive toward that goal, it will almost always give you goal, it will almost always give you goal, it will almost always give you better results than writing the code and better results than writing the code and better results than writing the code and then writing the test afterward because then writing the test afterward because then writing the test afterward because then it will fit the test to the code then it will fit the test to the code then it will fit the test to the code rather than fit the code to pass the rather than fit the code to pass the rather than fit the code to pass the verification criteria. Um this is the testing pyramid that uh Um this is the testing pyramid that uh can't the classic testing pyramid from can't the classic testing pyramid from can't the classic testing pyramid from the the the previous previous previous uh just when you think about the testing uh just when you think about the testing uh just when you think about the testing themselves, which you had the end-to-end themselves, which you had the end-to-end themselves, which you had the end-to-end test and the integration test and the test and the integration test and the test and the integration test and the unit test. This is very similar. unit test. This is very similar. unit test. This is very similar. Move as much as you can down to the Move as much as you can down to the Move as much as you can down to the deterministic analysis where that's deterministic analysis where that's deterministic analysis where that's linting, the compiler, um the unit test linting, the compiler, um the unit test linting, the compiler, um the unit test themselves. themselves. themselves. Whatever that can come be covered Whatever that can come be covered Whatever that can come be covered easily, you can have engine agent do easily, you can have engine agent do easily, you can have engine agent do reviews on it based on on criteria, so reviews on it based on on criteria, so reviews on it based on on criteria, so um um um architectural standards that that have architectural standards that that have architectural standards that that have been easily encoded into the code base, been easily encoded into the code base, been easily encoded into the code base, you can move into the agent. And then you can move into the agent. And then you can move into the agent. And then only at the very top you need to have only at the very top you need to have only at the very top you need to have some sort of human review, which is some sort of human review, which is some sort of human review, which is usually around the functionality and usually around the functionality and usually around the functionality and this is the right thing to build.

this is the right thing to build. this is the right thing to build. That like only leave the human to do That like only leave the human to do That like only leave the human to do what the humans need to actually be what the humans need to actually be what the humans need to actually be involved in. Um Another really important thing is the Um Another really important thing is the planning planning planning versus prompting. This is really tied versus prompting. This is really tied versus prompting. This is really tied into the giving agency back to into the giving agency back to into the giving agency back to developers and finding a replacement to developers and finding a replacement to developers and finding a replacement to the craft of writing code. the craft of writing code. the craft of writing code. Um Um Um spending a lot of time writing the plan spending a lot of time writing the plan spending a lot of time writing the plan and then and then and then sending enough to the agent basically as sending enough to the agent basically as sending enough to the agent basically as a a a as an implementation that can be done as an implementation that can be done as an implementation that can be done automatically is something that we find automatically is something that we find automatically is something that we find to really to really to really kind of kind of kind of reintroduce the joy of of building back reintroduce the joy of of building back reintroduce the joy of of building back into the process. into the process. into the process. And so it's not uncommon to spend a week And so it's not uncommon to spend a week And so it's not uncommon to spend a week writing a very detailed plan, making all writing a very detailed plan, making all writing a very detailed plan, making all the decisions, flushing it out, the decisions, flushing it out, the decisions, flushing it out, iterating, sending it out to teammates iterating, sending it out to teammates iterating, sending it out to teammates to review. to review. to review. And then only when it's ready and you've And then only when it's ready and you've And then only when it's ready and you've flushed out all the decision, you can flushed out all the decision, you can flushed out all the decision, you can send it to the agent. The agent will send it to the agent. The agent will send it to the agent. The agent will um send it back to you when it's um send it back to you when it's um send it back to you when it's implemented. implemented. implemented. And that that has been really successful And that that has been really successful And that that has been really successful also in accelerating and also also in accelerating and also also in accelerating and also really restoring some of the joy into really restoring some of the joy into really restoring some of the joy into the development process.

Uh so what makes a good plan? Um Uh so what makes a good plan? Um really important to start with a why at really important to start with a why at really important to start with a why at the top. It really helps preventing the top. It really helps preventing the top. It really helps preventing agent drift. If you have like a bold big agent drift. If you have like a bold big agent drift. If you have like a bold big section of kind of like it when you section of kind of like it when you section of kind of like it when you write a design doc, you want to have the write a design doc, you want to have the write a design doc, you want to have the executive summary. Put that in there for executive summary. Put that in there for executive summary. Put that in there for the agent. Otherwise, they'll start the agent. Otherwise, they'll start the agent. Otherwise, they'll start drifting over time and make sure that drifting over time and make sure that drifting over time and make sure that the agent don't go back and change that the agent don't go back and change that the agent don't go back and change that because they feel like it. because they feel like it. because they feel like it. Uh so we start with a why. Uh so we start with a why. Uh so we start with a why. Make sure that the plan can be broken Make sure that the plan can be broken Make sure that the plan can be broken down down down into small parts that can each be into small parts that can each be into small parts that can each be verified independently. verified independently. verified independently. And my personal way of knowing what is a And my personal way of knowing what is a And my personal way of knowing what is a good size would I want to review that good size would I want to review that good size would I want to review that the PR that will correspond to that the PR that will correspond to that the PR that will correspond to that part? If it's going to be too big for me part? If it's going to be too big for me part? If it's going to be too big for me to want to review in one sitting, it's to want to review in one sitting, it's to want to review in one sitting, it's kind of like the test is kind of like the test is kind of like the test is I'm going to get need to get a cup of I'm going to get need to get a cup of I'm going to get need to get a cup of coffee before I read this. coffee before I read this. coffee before I read this. That means it's too big and I'm going to That means it's too big and I'm going to That means it's too big and I'm going to want to have it broken down into pieces. want to have it broken down into pieces. want to have it broken down into pieces. And then And then And then I make sure that each part can be I make sure that each part can be I make sure that each part can be validated independently cuz what I don't validated independently cuz what I don't validated independently cuz what I don't want to have is want to have is want to have is have five stages and then the first one have five stages and then the first one have five stages and then the first one is written but not validated, and then is written but not validated, and then is written but not validated, and then everything else is is built on top of everything else is is built on top of everything else is is built on top of all the assumptions. So, having kind of all the assumptions. So, having kind of all the assumptions. So, having kind of a validation gate or an exception a validation gate or an exception a validation gate or an exception criteria for each phase really helps and criteria for each phase really helps and criteria for each phase really helps and make the plan uh resilient to drift. And make the plan uh resilient to drift. And make the plan uh resilient to drift. And all and and there's all kind of all and and there's all kind of all and and there's all kind of technique on how to manage the contacts technique on how to manage the contacts technique on how to manage the contacts and and and doing a a software factory on top of doing a a software factory on top of doing a a software factory on top of that. But once you have the plan, you that. But once you have the plan, you that. But once you have the plan, you can use whatever loop uh you want or can use whatever loop uh you want or can use whatever loop uh you want or whatever workflow you want whatever workflow you want whatever workflow you want in order to implement it.

in order to implement it. in order to implement it. Uh Uh Uh this is a screenshot that I randomly this is a screenshot that I randomly this is a screenshot that I randomly picked up a plan, but this is what I picked up a plan, but this is what I picked up a plan, but this is what I usually look for. The executive summary usually look for. The executive summary usually look for. The executive summary at the top, the phases break it down, at the top, the phases break it down, at the top, the phases break it down, and then each one of them I would go and then each one of them I would go and then each one of them I would go into lots of details so that I can just into lots of details so that I can just into lots of details so that I can just fit it into a sub agent, and the sub fit it into a sub agent, and the sub fit it into a sub agent, and the sub agent can independently work on that and agent can independently work on that and agent can independently work on that and not have to worry about it. Um that's not have to worry about it. Um that's not have to worry about it. Um that's that's it. There are other workflows that's it. There are other workflows that's it. There are other workflows that would work or other structure to that would work or other structure to that would work or other structure to the plan. I find that the plan. I find that the plan. I find that part of the things that great about uh part of the things that great about uh part of the things that great about uh AI workflows is that everybody can set AI workflows is that everybody can set AI workflows is that everybody can set up the thing that works best for them. up the thing that works best for them. up the thing that works best for them. Oh-oh. No, thank you. No, thank you. Everybody can very easily set up the Everybody can very easily set up the Everybody can very easily set up the workflow that work exactly for them for workflow that work exactly for them for workflow that work exactly for them for them. So, there's them. So, there's them. So, there's diminishing return in trying to diminishing return in trying to diminishing return in trying to centralize everybody on one thing, but centralize everybody on one thing, but centralize everybody on one thing, but as long as it works for their flow and as long as it works for their flow and as long as it works for their flow and other people can iterate with them, I other people can iterate with them, I other people can iterate with them, I find that it generally works very well. find that it generally works very well. find that it generally works very well. And this is just an example kind of a And this is just an example kind of a And this is just an example kind of a brag of like this is uh could be a brag of like this is uh could be a brag of like this is uh could be a result from a plan. Um there are result from a plan. Um there are result from a plan. Um there are probably 20 PRs here. Some of them would probably 20 PRs here. Some of them would probably 20 PRs here. Some of them would be maybe 10 lines, and some of them be maybe 10 lines, and some of them be maybe 10 lines, and some of them would be 100 lines. There's probably would be 100 lines. There's probably would be 100 lines. There's probably nothing bigger than that, and that nothing bigger than that, and that nothing bigger than that, and that allows us to allows us to allows us to This is in the pre-AI world, this plan This is in the pre-AI world, this plan This is in the pre-AI world, this plan probably worked in that for a week. I probably worked in that for a week. I probably worked in that for a week. I aligned with the other with three other aligned with the other with three other aligned with the other with three other teams for another week on that, and then teams for another week on that, and then teams for another week on that, and then I just sent it to an agent to implement I just sent it to an agent to implement I just sent it to an agent to implement overnight, and it came back. This is overnight, and it came back. This is overnight, and it came back. This is probably from two plans, not one, but probably from two plans, not one, but probably from two plans, not one, but it's it's basically six weeks of of it's it's basically six weeks of of it's it's basically six weeks of of coding work just It's coding work just It's coding work just It's um um um only took 1 week, so that's where

only took 1 week, so that's where only took 1 week, so that's where I got the 5x speed up. If I include the I got the 5x speed up. If I include the I got the 5x speed up. If I include the review cycle at the end that we always review cycle at the end that we always review cycle at the end that we always have to remember. Um moving on from planning Um moving on from planning back to the issue that we had with the back to the issue that we had with the back to the issue that we had with the skeptics and the people who are burdened skeptics and the people who are burdened skeptics and the people who are burdened with the most work, make sure that with the most work, make sure that with the most work, make sure that you you you bring them in and take their feedback bring them in and take their feedback bring them in and take their feedback really seriously. really seriously. really seriously. They're skeptic because they're seeing They're skeptic because they're seeing They're skeptic because they're seeing the the way you are lacking validation, the the way you are lacking validation, the the way you are lacking validation, where your tools fail. So, and their where your tools fail. So, and their where your tools fail. So, and their feedback is basically the road map of feedback is basically the road map of feedback is basically the road map of how to improve your agent how to improve your agent how to improve your agent interacting with the code base. interacting with the code base. interacting with the code base. So, just make sure to bring them in So, just make sure to bring them in So, just make sure to bring them in rather than trying to rather than trying to rather than trying to um figure out how to make them use the um figure out how to make them use the um figure out how to make them use the AI. Just let's have them be in charge of AI. Just let's have them be in charge of AI. Just let's have them be in charge of the road map to the road map to the road map to make AI safe your organization, and they make AI safe your organization, and they make AI safe your organization, and they will come along once they see that will come along once they see that will come along once they see that that the improvement that they're making that the improvement that they're making that the improvement that they're making actually making their life better. actually making their life better. actually making their life better. Um and as you can see, they'll not be Um and as you can see, they'll not be Um and as you can see, they'll not be shy about telling you what you need to shy about telling you what you need to shy about telling you what you need to fix. This is Latin hour sitting with a fix. This is Latin hour sitting with a fix. This is Latin hour sitting with a bunch of people, and bunch of people, and bunch of people, and this is the result of brainstorms.

Um another thing that's Um another thing that's been really helpful with my team been really helpful with my team been really helpful with my team specifically, and we're working to adopt specifically, and we're working to adopt specifically, and we're working to adopt it it it in the broader organization as well, is in the broader organization as well, is in the broader organization as well, is to make sure that you have an to make sure that you have an to make sure that you have an attention-aware communication. attention-aware communication. attention-aware communication. In the age of AI, human attention is a In the age of AI, human attention is a In the age of AI, human attention is a scarce resource. I think I've heard it scarce resource. I think I've heard it scarce resource. I think I've heard it for multiple talks, and a lot of people for multiple talks, and a lot of people for multiple talks, and a lot of people have have have have come to the same conclusion. You have come to the same conclusion. You have come to the same conclusion. You can't get more human attention. So, can't get more human attention. So, can't get more human attention. So, where you spend your time and what where you spend your time and what where you spend your time and what you're reading is really becomes really you're reading is really becomes really you're reading is really becomes really important. important. important. Um so, since it's such a scarce Um so, since it's such a scarce Um so, since it's such a scarce resource, resource, resource, marking what was generated by AI versus marking what was generated by AI versus marking what was generated by AI versus what was written by human is really what was written by human is really what was written by human is really helpful to know how much time you need helpful to know how much time you need helpful to know how much time you need to spend reading this, and how much slop to spend reading this, and how much slop to spend reading this, and how much slop can you expect in this part of the can you expect in this part of the can you expect in this part of the communication? communication? communication? Um Um Um and that can building a and that can building a and that can building a new culture around that new culture around that new culture around that self-communication that really helps. Um self-communication that really helps. Um self-communication that really helps. Um So, for example, So, for example, So, for example, um the team that team that I work with, um the team that team that I work with, um the team that team that I work with, we've decided we always every PR we've decided we always every PR we've decided we always every PR description will start with something description will start with something description will start with something like that, something that I wrote by like that, something that I wrote by like that, something that I wrote by hand. It could be very short that I hand. It could be very short that I hand. It could be very short that I describe what this is in code and what describe what this is in code and what describe what this is in code and what this is doing. And then the AI this is doing. And then the AI this is doing. And then the AI description is going to come after that, description is going to come after that, description is going to come after that, which is I will probably read it. I will which is I will probably read it. I will which is I will probably read it. I will probably edit it to remove uh some wrong probably edit it to remove uh some wrong probably edit it to remove uh some wrong things, but they didn't write every line things, but they didn't write every line things, but they didn't write every line here, so they should be more suspicious here, so they should be more suspicious here, so they should be more suspicious and they should pay more attention to and they should pay more attention to and they should pay more attention to what I wrote in the top and they should what I wrote in the top and they should what I wrote in the top and they should override it. Things like that in Slack, override it. Things like that in Slack, override it. Things like that in Slack, in email, it's like leaning into the in email, it's like leaning into the in email, it's like leaning into the fact that everybody knows that you're fact that everybody knows that you're fact that everybody knows that you're using AI to to craft your communication, using AI to to craft your communication, using AI to to craft your communication, but just let them share about it, tell

but just let them share about it, tell but just let them share about it, tell them what they should read and what you them what they should read and what you them what they should read and what you they should pay less attention to. they should pay less attention to. they should pay less attention to. And I remember early on, maybe And I remember early on, maybe And I remember early on, maybe like earlier in this year, I tried to I like earlier in this year, I tried to I like earlier in this year, I tried to I had some senior engineers in our org had some senior engineers in our org had some senior engineers in our org that had kind of were very much AI that had kind of were very much AI that had kind of were very much AI skeptic and I tried to reach out to them skeptic and I tried to reach out to them skeptic and I tried to reach out to them to see what was the problem, what was to see what was the problem, what was to see what was the problem, what was going on. I said, "I tried to run an going on. I said, "I tried to run an going on. I said, "I tried to run an analysis on some of the PR comments that analysis on some of the PR comments that analysis on some of the PR comments that you've run." And obviously you've run." And obviously you've run." And obviously I used the AI to do that. I used the AI to do that. I used the AI to do that. And then I didn't distinguish very And then I didn't distinguish very And then I didn't distinguish very clearly what I wrote versus what they clearly what I wrote versus what they clearly what I wrote versus what they what AI generated. And they got very what AI generated. And they got very what AI generated. And they got very upset. They're like, "Why is sending I upset. They're like, "Why is sending I upset. They're like, "Why is sending I did not expect somebody um that I did not expect somebody um that I did not expect somebody um that I respect this much to send me respect this much to send me respect this much to send me something that's clearly this sloppy." something that's clearly this sloppy." something that's clearly this sloppy." And then like I I took immediately like And then like I I took immediately like And then like I I took immediately like I apologize. I realize I should have I apologize. I realize I should have I apologize. I realize I should have marked it clearly and marked my marked it clearly and marked my marked it clearly and marked my intention like this is what I wrote. intention like this is what I wrote. intention like this is what I wrote. This is what the AI wrote and I need This is what the AI wrote and I need This is what the AI wrote and I need your feedback on that because I don't your feedback on that because I don't your feedback on that because I don't have the context to know if it is sloppy have the context to know if it is sloppy have the context to know if it is sloppy or not and that's what I'm asking you or not and that's what I'm asking you or not and that's what I'm asking you for, so lesson like that and change the for, so lesson like that and change the for, so lesson like that and change the culture is just as important as some the culture is just as important as some the culture is just as important as some the engineering challenges that we've been engineering challenges that we've been engineering challenges that we've been facing.

Um another thing that's really helpful Um another thing that's really helpful around the adoption is around the adoption is around the adoption is um as you progress through adoption, um as you progress through adoption, um as you progress through adoption, there's a lot of very fancy tools and a there's a lot of very fancy tools and a there's a lot of very fancy tools and a lot of very fancy workflow that we've lot of very fancy workflow that we've lot of very fancy workflow that we've we've been implementing, but one of the we've been implementing, but one of the we've been implementing, but one of the really effective thing is just letting really effective thing is just letting really effective thing is just letting people use the AI where they're at. So, people use the AI where they're at. So, people use the AI where they're at. So, uh it help it really helps normalize uh it help it really helps normalize uh it help it really helps normalize uh the use of AI for everyday tasks and uh the use of AI for everyday tasks and uh the use of AI for everyday tasks and it helps reduce the friction. it helps reduce the friction. it helps reduce the friction. And really one of the most powerful And really one of the most powerful And really one of the most powerful thing is being able to tag an agent in thing is being able to tag an agent in thing is being able to tag an agent in the Slack message with somebody and they the Slack message with somebody and they the Slack message with somebody and they can you just do this for me? can you just do this for me? can you just do this for me? And have the agents to close the loop in And have the agents to close the loop in And have the agents to close the loop in the thread. the thread. the thread. Um that that's kind of thing is really Um that that's kind of thing is really Um that that's kind of thing is really powerful. And then you can go on top of powerful. And then you can go on top of powerful. And then you can go on top of that and have all this thing automated that and have all this thing automated that and have all this thing automated and do all kind of fancy things, but if and do all kind of fancy things, but if and do all kind of fancy things, but if you have a new conversation with you have a new conversation with you have a new conversation with somebody who's not fully bought in and somebody who's not fully bought in and somebody who's not fully bought in and then you can tag it in a non like then you can tag it in a non like then you can tag it in a non like non-passive-aggressive way. You can tag non-passive-aggressive way. You can tag non-passive-aggressive way. You can tag it and say, "Let's try to see if the it and say, "Let's try to see if the it and say, "Let's try to see if the agent can get it this time." agent can get it this time." agent can get it this time." And they close the loop and if it's a And they close the loop and if it's a And they close the loop and if it's a good experience, that really helps good experience, that really helps good experience, that really helps people try it out on their own people try it out on their own people try it out on their own in other cases.

And our journey continues. We're still And our journey continues. We're still learning even though we're shipping AI learning even though we're shipping AI learning even though we're shipping AI externally, our AI adoption externally, our AI adoption externally, our AI adoption um um um we're experimenting with with so many we're experimenting with with so many we're experimenting with with so many things all the time. Our automation things all the time. Our automation things all the time. Our automation story is not story is not story is not uh fully there yet. We're still trying uh fully there yet. We're still trying uh fully there yet. We're still trying trying to figure out when we should use trying to figure out when we should use trying to figure out when we should use how we can use cloud agent effectively how we can use cloud agent effectively how we can use cloud agent effectively given all the dependencies we have for given all the dependencies we have for given all the dependencies we have for some of our bell system. some of our bell system. some of our bell system. And so we are continuing to learn. It's And so we are continuing to learn. It's And so we are continuing to learn. It's a culture shift, it's an engineering a culture shift, it's an engineering a culture shift, it's an engineering shift and I don't know about you, but shift and I don't know about you, but shift and I don't know about you, but for I've been I've been working in the for I've been I've been working in the for I've been I've been working in the valley for the last 15 years and this is valley for the last 15 years and this is valley for the last 15 years and this is the biggest change by orders of the biggest change by orders of the biggest change by orders of magnitude of everything that I've seen magnitude of everything that I've seen magnitude of everything that I've seen in term culture and technology. in term culture and technology. in term culture and technology. So, um we're all here together and we're So, um we're all here together and we're So, um we're all here together and we're all figuring it out and that's that's all figuring it out and that's that's all figuring it out and that's that's what I wanted to talk to you today. what I wanted to talk to you today. what I wanted to talk to you today. Thank you. Thank you. Thank you. >> [applause]