How Forward Deployed Engineering is done at Cognition — Jia Wu

>> It's nice to be here. I appreciate you >> It's nice to be here. I appreciate you all. My name is Gia. I'm a deployed all. My name is Gia. I'm a deployed all. My name is Gia. I'm a deployed engineering lead at Cognition. And engineering lead at Cognition. And engineering lead at Cognition. And today, hopefully, what you'll take away today, hopefully, what you'll take away today, hopefully, what you'll take away from this is that how we deploy Devin in from this is that how we deploy Devin in from this is that how we deploy Devin in the field is the field is the field is very much a function of how we view very much a function of how we view very much a function of how we view deployed engineering at Cognition. So, deployed engineering at Cognition. So, deployed engineering at Cognition. So, how the four deployed motion makes AI how the four deployed motion makes AI how the four deployed motion makes AI engineering actually real. engineering actually real. engineering actually real. Before I start, how many people like Before I start, how many people like Before I start, how many people like have heard of Devin or like know of have heard of Devin or like know of have heard of Devin or like know of Devin? Devin? Devin? Oh, cool. And I'm not talking about like Oh, cool. And I'm not talking about like Oh, cool. And I'm not talking about like the Devin of today. Like, I'm talking the Devin of today. Like, I'm talking the Devin of today. Like, I'm talking about the Devin back in 2024 when we about the Devin back in 2024 when we about the Devin back in 2024 when we first released and it was like, "Oh, first released and it was like, "Oh, first released and it was like, "Oh, SweepBench 13%. We're so We're so back." SweepBench 13%. We're so We're so back." SweepBench 13%. We're so We're so back." And as engineers, we were like, "We're And as engineers, we were like, "We're And as engineers, we were like, "We're so cooked." But, I mean, after a week, so cooked." But, I mean, after a week, so cooked." But, I mean, after a week, everyone's like, "Oh, this is actually everyone's like, "Oh, this is actually everyone's like, "Oh, this is actually like not that useful." like not that useful." like not that useful." Um Um Um "I would only use this if I was "I would only use this if I was "I would only use this if I was desperate and out of ideas." desperate and out of ideas." desperate and out of ideas." That's 2024. That's 2024. That's 2024. Uh Uh Uh and let it be said that we have a sense and let it be said that we have a sense and let it be said that we have a sense of humor because we took this and we ran of humor because we took this and we ran of humor because we took this and we ran with it. with it. with it. I'm sure you've seen all these ads I'm sure you've seen all these ads I'm sure you've seen all these ads around SF. around SF. around SF. We're actually good now. We're actually good now. We're actually good now. So, the reason why we're good is So, the reason why we're good is So, the reason why we're good is I'll talk a little bit about the product I'll talk a little bit about the product I'll talk a little bit about the product surface area a little bit just to give surface area a little bit just to give surface area a little bit just to give you folks a little bit of context for you folks a little bit of context for you folks a little bit of context for who might not have used Devin before, uh who might not have used Devin before, uh who might not have used Devin before, uh who might not have exposure to something who might not have exposure to something who might not have exposure to something like Cognition. So, if you've used Flood like Cognition. So, if you've used Flood like Cognition. So, if you've used Flood Code, we also expose a CLI. If you've Code, we also expose a CLI. If you've Code, we also expose a CLI. If you've used something like, you know, Cursor, used something like, you know, Cursor, used something like, you know, Cursor, Windsurf, we also expose an interface Windsurf, we also expose an interface Windsurf, we also expose an interface such as an IDE. Uh actually, how many such as an IDE. Uh actually, how many such as an IDE. Uh actually, how many people know of Windsurf or have used people know of Windsurf or have used people know of Windsurf or have used Windsurf in the past?

Windsurf in the past? Windsurf in the past? Sweet. Sweet. Sweet. So, I come over from the Windsurf side So, I come over from the Windsurf side So, I come over from the Windsurf side after the Windsurf acquisition. Um after the Windsurf acquisition. Um after the Windsurf acquisition. Um great times. great times. great times. And then, what we're actually known for, And then, what we're actually known for, And then, what we're actually known for, specifically, is for Devin Cloud or the specifically, is for Devin Cloud or the specifically, is for Devin Cloud or the Devin Cloud agent. And I'm not going to Devin Cloud agent. And I'm not going to Devin Cloud agent. And I'm not going to bore you to death talking about like all bore you to death talking about like all bore you to death talking about like all the components and features and the components and features and the components and features and everything that comprise of the actual everything that comprise of the actual everything that comprise of the actual product. I'm not here to sell you on product. I'm not here to sell you on product. I'm not here to sell you on that. What I am here to sell you on is that. What I am here to sell you on is that. What I am here to sell you on is we are one of the premier software we are one of the premier software we are one of the premier software engineering functions across the engineering functions across the engineering functions across the enterprise, and we're actually able to enterprise, and we're actually able to enterprise, and we're actually able to deliver impact on a global scale. What deliver impact on a global scale. What deliver impact on a global scale. What do I mean by that? do I mean by that? do I mean by that? We can take a pause and take a look at We can take a pause and take a look at We can take a pause and take a look at this figure. this figure. this figure. So, internally at Cognition, over the So, internally at Cognition, over the So, internally at Cognition, over the last 6 months, for better or worse, we last 6 months, for better or worse, we last 6 months, for better or worse, we might have been behind on hiring, but might have been behind on hiring, but might have been behind on hiring, but using our agent, we were able to ship using our agent, we were able to ship using our agent, we were able to ship almost an order of magnitude more almost an order of magnitude more almost an order of magnitude more good quality robust PRs across the good quality robust PRs across the good quality robust PRs across the organization. So, it's a step function organization. So, it's a step function organization. So, it's a step function increase in the amount of engineering increase in the amount of engineering increase in the amount of engineering leverage that we can have by deploying leverage that we can have by deploying leverage that we can have by deploying our own agent. You don't have to take our own agent. You don't have to take our own agent. You don't have to take our word for it. If we take a look If we our word for it. If we take a look If we our word for it. If we take a look If we can take a look at the specifics of how can take a look at the specifics of how can take a look at the specifics of how we're actually being consumed, how we're we're actually being consumed, how we're we're actually being consumed, how we're being utilized across the enterprise, being utilized across the enterprise, being utilized across the enterprise, it's a parabolic growth of how companies it's a parabolic growth of how companies it's a parabolic growth of how companies are adopting our agent, deploying our are adopting our agent, deploying our are adopting our agent, deploying our agent, and using it in multiple agent, and using it in multiple agent, and using it in multiple different use cases and multiple different use cases and multiple different use cases and multiple different scenarios.

different scenarios. different scenarios. So, what does it mean, right? Like, how So, what does it mean, right? Like, how So, what does it mean, right? Like, how does this actually happen? Well, it can does this actually happen? Well, it can does this actually happen? Well, it can only happen with the forward-deployed only happen with the forward-deployed only happen with the forward-deployed engineers at Cognition. engineers at Cognition. engineers at Cognition. And I'm going to frame up the problem And I'm going to frame up the problem And I'm going to frame up the problem from like a couple of like buckets, from like a couple of like buckets, from like a couple of like buckets, right? So, there's two circles in front right? So, there's two circles in front right? So, there's two circles in front of you on the screen. of you on the screen. of you on the screen. One of them can represent the domain of One of them can represent the domain of One of them can represent the domain of a product, right? As a business, as a a product, right? As a business, as a a product, right? As a business, as a software engineering organization, software engineering organization, software engineering organization, obviously, you have a product. obviously, you have a product. obviously, you have a product. You obviously also, on the other side, You obviously also, on the other side, You obviously also, on the other side, on the left-hand side or right-hand side on the left-hand side or right-hand side on the left-hand side or right-hand side for you guys, you specifically have like for you guys, you specifically have like for you guys, you specifically have like a bucket of problems that you're looking a bucket of problems that you're looking a bucket of problems that you're looking to solve, right? You have a product, you to solve, right? You have a product, you to solve, right? You have a product, you have problems that you're trying to have problems that you're trying to have problems that you're trying to solve, and the intersection of these solve, and the intersection of these solve, and the intersection of these two, or whatever you would call it, is two, or whatever you would call it, is two, or whatever you would call it, is the product-market fit. Right? the product-market fit. Right? the product-market fit. Right? Hopefully, you have a pretty good Hopefully, you have a pretty good Hopefully, you have a pretty good overlap in the sense that whatever it is overlap in the sense that whatever it is overlap in the sense that whatever it is that your company does, you can actually that your company does, you can actually that your company does, you can actually bring value to customers. And hopefully, bring value to customers. And hopefully, bring value to customers. And hopefully, whatever problems the customer has, you whatever problems the customer has, you whatever problems the customer has, you can solve with your company's stuff. can solve with your company's stuff. can solve with your company's stuff. So, the forward-deployed motion at So, the forward-deployed motion at So, the forward-deployed motion at Cognition essentially aims to maximize Cognition essentially aims to maximize Cognition essentially aims to maximize the overlap between the products that we the overlap between the products that we the overlap between the products that we typically build and the problems that typically build and the problems that typically build and the problems that we're experiencing across the we're experiencing across the we're experiencing across the enterprise.

enterprise. enterprise. So, what does that mean, right? The So, what does that mean, right? The So, what does that mean, right? The first fundamental concept that I would first fundamental concept that I would first fundamental concept that I would like to convey is that forward-deployed like to convey is that forward-deployed like to convey is that forward-deployed engineers at Cog deeply understand the engineers at Cog deeply understand the engineers at Cog deeply understand the problem space at hand. problem space at hand. problem space at hand. And specifically, right? Like if we And specifically, right? Like if we And specifically, right? Like if we think of the problem of software think of the problem of software think of the problem of software engineering, and I'm just going to like engineering, and I'm just going to like engineering, and I'm just going to like mask the features at the bottom, we mask the features at the bottom, we mask the features at the bottom, we don't really care about those, but if we don't really care about those, but if we don't really care about those, but if we think about when you go ahead to take think about when you go ahead to take think about when you go ahead to take some sort of codebase, some sort of some sort of codebase, some sort of some sort of codebase, some sort of implementation, and you need to like implementation, and you need to like implementation, and you need to like build features, you need to maintain build features, you need to maintain build features, you need to maintain that software, uh you need to like that software, uh you need to like that software, uh you need to like review, deploy, maintain that software. review, deploy, maintain that software. review, deploy, maintain that software. All of these steps, all of these All of these steps, all of these All of these steps, all of these functions have a lot of business value functions have a lot of business value functions have a lot of business value behind them, right? You can only do like behind them, right? You can only do like behind them, right? You can only do like it would be great if we could build from it would be great if we could build from it would be great if we could build from zero to one and just like prompt stuff zero to one and just like prompt stuff zero to one and just like prompt stuff and not worry about like legacy code, and not worry about like legacy code, and not worry about like legacy code, but that's not the reality of the but that's not the reality of the but that's not the reality of the situation. It would be great if our situation. It would be great if our situation. It would be great if our product engineers or our product product engineers or our product product engineers or our product managers could just take user stories managers could just take user stories managers could just take user stories and say like these are the things that and say like these are the things that and say like these are the things that we need to build we need to build we need to build in order to get value and revenue. in order to get value and revenue. in order to get value and revenue. And then we come to coding. And then we come to coding. And then we come to coding. Coding itself, at least from our Coding itself, at least from our Coding itself, at least from our perspective, is a mostly solved problem, perspective, is a mostly solved problem, perspective, is a mostly solved problem, right? These models are so good now that right? These models are so good now that right? These models are so good now that like with any type of context, with like with any type of context, with like with any type of context, with enough context engineering, you can get enough context engineering, you can get enough context engineering, you can get the code blocks that you really care the code blocks that you really care the code blocks that you really care about. But the problem isn't like about. But the problem isn't like about. But the problem isn't like writing code faster, that's usually only writing code faster, that's usually only writing code faster, that's usually only 20% of the problem. The problem really 20% of the problem. The problem really 20% of the problem. The problem really just becomes like how do you test this just becomes like how do you test this just becomes like how do you test this code? How do you review and deploy this code? How do you review and deploy this code? How do you review and deploy this code? And how do you maintain this code code? And how do you maintain this code code? And how do you maintain this code across the enterprise? So, that's the across the enterprise? So, that's the across the enterprise? So, that's the premise of the problem. Now, I will premise of the problem. Now, I will premise of the problem. Now, I will predicate that by saying that like we predicate that by saying that like we predicate that by saying that like we also have the solution, I'm not going to also have the solution, I'm not going to also have the solution, I'm not going to bore you to death about talking to you bore you to death about talking to you bore you to death about talking to you about the solution, but for deployed about the solution, but for deployed about the solution, but for deployed engineers at Cognition, we map Devin's engineers at Cognition, we map Devin's engineers at Cognition, we map Devin's capabilities specifically to the capabilities specifically to the capabilities specifically to the customer problem. So, if the software

customer problem. So, if the software customer problem. So, if the software development life cycle is extremely development life cycle is extremely development life cycle is extremely complex, deploying the agents for with complex, deploying the agents for with complex, deploying the agents for with like no specific direction, you're like no specific direction, you're like no specific direction, you're straight up just token maxing. Right? straight up just token maxing. Right? straight up just token maxing. Right? Like you were wasting tokens, you were Like you were wasting tokens, you were Like you were wasting tokens, you were burning you were burning spend, burning you were burning spend, burning you were burning spend, you're not getting any tangible you're not getting any tangible you're not getting any tangible outcomes. outcomes. outcomes. So, we try to identify, when we partner So, we try to identify, when we partner So, we try to identify, when we partner with customers, when we take meetings, with customers, when we take meetings, with customers, when we take meetings, and for example, right? Like my day and for example, right? Like my day and for example, right? Like my day might look like four or five hours of might look like four or five hours of might look like four or five hours of customer calls, and then four or five customer calls, and then four or five customer calls, and then four or five hours of like actual hands-on keyboard hours of like actual hands-on keyboard hours of like actual hands-on keyboard work. work. work. Those four or five hours of calls Those four or five hours of calls Those four or five hours of calls actually allow us to understand very actually allow us to understand very actually allow us to understand very deeply what strategic initiatives are deeply what strategic initiatives are deeply what strategic initiatives are the highest leverage for the business. the highest leverage for the business. the highest leverage for the business. Right? Once Once identified that, right? Right? Once Once identified that, right? Right? Once Once identified that, right? How can we automate ourselves out of the How can we automate ourselves out of the How can we automate ourselves out of the job in the sense that we set up the job in the sense that we set up the job in the sense that we set up the agent in a way that it runs all of the agent in a way that it runs all of the agent in a way that it runs all of the automations for us, right? We don't have automations for us, right? We don't have automations for us, right? We don't have to be there manually triggering the to be there manually triggering the to be there manually triggering the agents. It can respond to like specific agents. It can respond to like specific agents. It can respond to like specific alerts, specific events. alerts, specific events. alerts, specific events. But most importantly, as forward But most importantly, as forward But most importantly, as forward deployed engineer, how do you measure deployed engineer, how do you measure deployed engineer, how do you measure the return on investment? the return on investment? the return on investment? And it's very ambiguous. And it's very ambiguous. And it's very ambiguous. And it's an unsolved problem because the And it's an unsolved problem because the And it's an unsolved problem because the company that will solve this will be um company that will solve this will be um company that will solve this will be um you know, $5 trillion market cap. you know, $5 trillion market cap. you know, $5 trillion market cap. So, specifically, our forward deployed So, specifically, our forward deployed So, specifically, our forward deployed engineers will embed in the customers engineers will embed in the customers engineers will embed in the customers ecosystems, right? We take a look at, ecosystems, right? We take a look at, ecosystems, right? We take a look at, you know, the backlog of stuff that you know, the backlog of stuff that you know, the backlog of stuff that needs to be built. We take a look at the needs to be built. We take a look at the needs to be built. We take a look at the remediations that need to be done. We remediations that need to be done. We remediations that need to be done. We take a look at all of like the delayed take a look at all of like the delayed take a look at all of like the delayed code that never ships or the test that code that never ships or the test that code that never ships or the test that nobody writes uh or the automatic triage nobody writes uh or the automatic triage nobody writes uh or the automatic triage of specific alerts.

of specific alerts. of specific alerts. And we map our product capabilities into And we map our product capabilities into And we map our product capabilities into those problems. That being said, That being said, if we all do our job and we solve the if we all do our job and we solve the if we all do our job and we solve the customers problems 100% and the customer customers problems 100% and the customer customers problems 100% and the customer is very happy, that is only half of the is very happy, that is only half of the is very happy, that is only half of the equation, right? equation, right? equation, right? Not only do we have to solve the Not only do we have to solve the Not only do we have to solve the customer's problem, customer's problem, customer's problem, we also need to solve for our product. we also need to solve for our product. we also need to solve for our product. What do I mean by that, right? If we are What do I mean by that, right? If we are What do I mean by that, right? If we are taking it union of like the problems taking it union of like the problems taking it union of like the problems that the customers have and the products that the customers have and the products that the customers have and the products that we are building, right? Solving the that we are building, right? Solving the that we are building, right? Solving the problem only shifts like part of the problem only shifts like part of the problem only shifts like part of the Venn diagram. But in order to get that Venn diagram. But in order to get that Venn diagram. But in order to get that true feedback loop where we unify like true feedback loop where we unify like true feedback loop where we unify like the maximal overlap between what we're the maximal overlap between what we're the maximal overlap between what we're doing and what the customers need, that doing and what the customers need, that doing and what the customers need, that is forward deployed engineering at is forward deployed engineering at is forward deployed engineering at Cockroach Labs. So, the second part of Cockroach Labs. So, the second part of Cockroach Labs. So, the second part of this that I want to emphasize is that at this that I want to emphasize is that at this that I want to emphasize is that at our company, our company, our company, we map the customer problems back to the we map the customer problems back to the we map the customer problems back to the capabilities at hand. Now, how do we do capabilities at hand. Now, how do we do capabilities at hand. Now, how do we do that, right? that, right? that, right? A lot of engineering challenges A lot of engineering challenges A lot of engineering challenges come in similar shapes um from the come in similar shapes um from the come in similar shapes um from the field, right? We have the highest field, right? We have the highest field, right? We have the highest fidelity evaluation set that comes back fidelity evaluation set that comes back fidelity evaluation set that comes back from our customers, right? We are in the from our customers, right? We are in the from our customers, right? We are in the field every single day. We are hearing field every single day. We are hearing field every single day. We are hearing about the problems. We are hearing about about the problems. We are hearing about about the problems. We are hearing about whatever the customers are doing.

whatever the customers are doing. whatever the customers are doing. And we have to take that context and And we have to take that context and And we have to take that context and bring it back to product in a sensible bring it back to product in a sensible bring it back to product in a sensible way. So, what are the ways in which way. So, what are the ways in which way. So, what are the ways in which these enterprise challenges, you know, these enterprise challenges, you know, these enterprise challenges, you know, manifest? are they common across the manifest? are they common across the manifest? are they common across the entire enterprise or are they unique to entire enterprise or are they unique to entire enterprise or are they unique to a specific user? a specific user? a specific user? Should like workarounds or like hacks or Should like workarounds or like hacks or Should like workarounds or like hacks or bugs in bugs in bugs in in in what we're building become in in what we're building become in in what we're building become features, right? Because ultimately what features, right? Because ultimately what features, right? Because ultimately what we want to do we want to do we want to do as a company is we want to de-risk our as a company is we want to de-risk our as a company is we want to de-risk our road map. road map. road map. At the end of the day, At the end of the day, At the end of the day, it would be great if like I as an it would be great if like I as an it would be great if like I as an engineer knew exactly what to build to engineer knew exactly what to build to engineer knew exactly what to build to get Y percentage of revenue from get Y percentage of revenue from get Y percentage of revenue from particular customer. And that's exactly particular customer. And that's exactly particular customer. And that's exactly what the problem is that we're trying to what the problem is that we're trying to what the problem is that we're trying to solve for because we are ultimately at solve for because we are ultimately at solve for because we are ultimately at the end of the day the heralds of the the end of the day the heralds of the the end of the day the heralds of the change, right? We are the bridge between change, right? We are the bridge between change, right? We are the bridge between products. We are the bridge between products. We are the bridge between products. We are the bridge between problems and the feedback is actually problems and the feedback is actually problems and the feedback is actually like half of the loop that makes the like half of the loop that makes the like half of the loop that makes the next deployment better than the previous next deployment better than the previous next deployment better than the previous deployment. deployment. deployment. So, So, So, if we think about the T-shape or if we think about the T-shape or if we think about the T-shape or personas of the folks that we hire, and for a deployed engineer, it's just and for a deployed engineer, it's just so so so it's so flexible in terms of how you it's so flexible in terms of how you it's so flexible in terms of how you actually define it. Like, are you a actually define it. Like, are you a actually define it. Like, are you a sales engineer? Are you a solutions sales engineer? Are you a solutions sales engineer? Are you a solutions architect?

architect? architect? What are you, right? So, if you think What are you, right? So, if you think What are you, right? So, if you think about all of the skills that FDEs about all of the skills that FDEs about all of the skills that FDEs typically are expected to have, right? typically are expected to have, right? typically are expected to have, right? You probably want to go wide. You You probably want to go wide. You You probably want to go wide. You probably want to have, you know, good probably want to have, you know, good probably want to have, you know, good people skill, like business skill, good people skill, like business skill, good people skill, like business skill, good process, customer skill, or even process, customer skill, or even process, customer skill, or even technology, right? In order to be technology, right? In order to be technology, right? In order to be deployed in the space, you need to know deployed in the space, you need to know deployed in the space, you need to know like what whatever the tech is. like what whatever the tech is. like what whatever the tech is. So, we also look for very deep spikes So, we also look for very deep spikes So, we also look for very deep spikes across people, right? across people, right? across people, right? Folks at Cognition deployed engineers at Folks at Cognition deployed engineers at Folks at Cognition deployed engineers at Cognition, we will hire them from like Cognition, we will hire them from like Cognition, we will hire them from like product management backgrounds if they product management backgrounds if they product management backgrounds if they have a really good sense of like how have a really good sense of like how have a really good sense of like how products are supposed to fit into each products are supposed to fit into each products are supposed to fit into each other, right? Because if the cost of other, right? Because if the cost of other, right? Because if the cost of software engineering is going to zero, software engineering is going to zero, software engineering is going to zero, you actually need to know how to like you actually need to know how to like you actually need to know how to like design a product that makes sense. design a product that makes sense. design a product that makes sense. Cuz you can just prompt it. Cuz you can just prompt it. Cuz you can just prompt it. But we also hire folks that are like But we also hire folks that are like But we also hire folks that are like founders, software engineers, but founders, software engineers, but founders, software engineers, but specifically like very spiky in the specifically like very spiky in the specifically like very spiky in the technology domains, right? technology domains, right? technology domains, right? It's fine if you don't have like the It's fine if you don't have like the It's fine if you don't have like the strongest business sense, that can be strongest business sense, that can be strongest business sense, that can be learned, but it's also very hard to learned, but it's also very hard to learned, but it's also very hard to teach like technicality and being the teach like technicality and being the teach like technicality and being the expert in the room while you're on the expert in the room while you're on the expert in the room while you're on the job. So, there's a couple of personas job. So, there's a couple of personas job. So, there's a couple of personas that we specifically hire for. that we specifically hire for. that we specifically hire for. Good customer engineers or deployed Good customer engineers or deployed Good customer engineers or deployed engineers engineers engineers do everything, right? Like they can do everything, right? Like they can do everything, right? Like they can map the product back to road map, like map the product back to road map, like map the product back to road map, like they can solve the customer's problems.

they can solve the customer's problems. they can solve the customer's problems. Great customer engineers are able to Great customer engineers are able to Great customer engineers are able to actually have that relentless curiosity actually have that relentless curiosity actually have that relentless curiosity for why. for why. for why. So, at Cognition, we always ask So, at Cognition, we always ask So, at Cognition, we always ask ourselves, ourselves, ourselves, "Why are we solving this problem? "Why are we solving this problem? "Why are we solving this problem? Does this problem matter to the to the Does this problem matter to the to the Does this problem matter to the to the business as a whole? business as a whole? business as a whole? And can I communicate this back to the And can I communicate this back to the And can I communicate this back to the road map in a way that like improves the road map in a way that like improves the road map in a way that like improves the problem for everybody else?" problem for everybody else?" problem for everybody else?" But we also are The second mantra that But we also are The second mantra that But we also are The second mantra that we subscribe to is that you have to be we subscribe to is that you have to be we subscribe to is that you have to be relentlessly relentlessly relentlessly tied in to the customer, right? They are tied in to the customer, right? They are tied in to the customer, right? They are our lifeblood at the end of the day. our lifeblood at the end of the day. our lifeblood at the end of the day. Making them successful is the only way Making them successful is the only way Making them successful is the only way that you can survive as a business and that you can survive as a business and that you can survive as a business and become the obvious choice for an become the obvious choice for an become the obvious choice for an enterprise partnership. So, these are enterprise partnership. So, these are enterprise partnership. So, these are the two mantras that we specifically the two mantras that we specifically the two mantras that we specifically subscribe to as uh deployed engineers at subscribe to as uh deployed engineers at subscribe to as uh deployed engineers at Cognition. Cognition. Cognition. So, So, So, previously, um previously, um previously, um the first approach to deployed the first approach to deployed the first approach to deployed engineering was just, you know, token engineering was just, you know, token engineering was just, you know, token maxing, right? Uh the next era that I'll maxing, right? Uh the next era that I'll maxing, right? Uh the next era that I'll talk about is talk about is talk about is intelligent orchestration, but with like intelligent orchestration, but with like intelligent orchestration, but with like outcomes that you can actually measure, outcomes that you can actually measure, outcomes that you can actually measure, and I'll give you guys some examples. and I'll give you guys some examples. and I'll give you guys some examples. So, So, So, previously, like a year, maybe a year previously, like a year, maybe a year previously, like a year, maybe a year and a half, two years ago, uh the target and a half, two years ago, uh the target and a half, two years ago, uh the target or KPI for whatever deployed engineers or KPI for whatever deployed engineers or KPI for whatever deployed engineers were trying to do is maximize token were trying to do is maximize token were trying to do is maximize token usage, right? It was It was like the usage, right? It was It was like the usage, right? It was It was like the perfect time. Didn't have to worry about perfect time. Didn't have to worry about perfect time. Didn't have to worry about budgets. Everything was subsidized. You budgets. Everything was subsidized. You budgets. Everything was subsidized. You could just like run anything you wanted.

could just like run anything you wanted. could just like run anything you wanted. But now, the problem has really shifted But now, the problem has really shifted But now, the problem has really shifted into into into the delivery space, right? the delivery space, right? the delivery space, right? A lot of organizations that we work A lot of organizations that we work A lot of organizations that we work with, some of the largest and most with, some of the largest and most with, some of the largest and most regulated enterprises in the world, they regulated enterprises in the world, they regulated enterprises in the world, they really care about, "Are we getting true really care about, "Are we getting true really care about, "Are we getting true value out of this solution, or are we value out of this solution, or are we value out of this solution, or are we just burning tokens for no reason, just burning tokens for no reason, just burning tokens for no reason, right?" right?" right?" And that is one core differentiator that And that is one core differentiator that And that is one core differentiator that I need to call out between us and some I need to call out between us and some I need to call out between us and some of the other platforms. You can make of the other platforms. You can make of the other platforms. You can make engineers like 10x faster. That's fine. engineers like 10x faster. That's fine. engineers like 10x faster. That's fine. That's still valuable. But can you make That's still valuable. But can you make That's still valuable. But can you make an organization 10x faster, including an organization 10x faster, including an organization 10x faster, including every single person that might be every single person that might be every single person that might be technical or non-technical uh across the technical or non-technical uh across the technical or non-technical uh across the company? company? company? That's when you unlock the true value of That's when you unlock the true value of That's when you unlock the true value of being the partnership. And that's why being the partnership. And that's why being the partnership. And that's why like single point tools that are just like single point tools that are just like single point tools that are just like CLIs or just IDEs, they fail to do like CLIs or just IDEs, they fail to do like CLIs or just IDEs, they fail to do that. that. that. So, So, So, let me just give you some proof points let me just give you some proof points let me just give you some proof points of how we've operated across the of how we've operated across the of how we've operated across the enterprise. So, at Cognition, um So, at Cognition, um when we run the agent and the agent has when we run the agent and the agent has when we run the agent and the agent has like a specific trace or trajectory or like a specific trace or trajectory or like a specific trace or trajectory or like the agent does something, we call like the agent does something, we call like the agent does something, we call that a session.

that a session. that a session. A session itself, right? We have metrics A session itself, right? We have metrics A session itself, right? We have metrics that allow you to derive how many that allow you to derive how many that allow you to derive how many engineering hours you can actually engineering hours you can actually engineering hours you can actually generate and how much how many generate and how much how many generate and how much how many engineering hours are actually engineering hours are actually engineering hours are actually productive that users are running. productive that users are running. productive that users are running. Right? So, one of these examples is a Right? So, one of these examples is a Right? So, one of these examples is a case study where we embedded ourselves case study where we embedded ourselves case study where we embedded ourselves within a customer for 3 months. We within a customer for 3 months. We within a customer for 3 months. We brought them on board. And functionally, brought them on board. And functionally, brought them on board. And functionally, over the course of those 3 months, we over the course of those 3 months, we over the course of those 3 months, we delivered about 150% like plus delivered about 150% like plus delivered about 150% like plus headcount. So, if you thought about like headcount. So, if you thought about like headcount. So, if you thought about like a project that you're trying to ship or a project that you're trying to ship or a project that you're trying to ship or something that you're trying to develop something that you're trying to develop something that you're trying to develop or a migration that you're trying to go or a migration that you're trying to go or a migration that you're trying to go through, imagine having 150 extra through, imagine having 150 extra through, imagine having 150 extra coworkers doing that with you by your coworkers doing that with you by your coworkers doing that with you by your side. You might just say, "Hey, this actually You might just say, "Hey, this actually just kind of looks like token maxing, just kind of looks like token maxing, just kind of looks like token maxing, right? Like you're just giving me a right? Like you're just giving me a right? Like you're just giving me a metric that says metric that says metric that says it's just, you know, engineering hours. it's just, you know, engineering hours. it's just, you know, engineering hours. You're like running a bunch of different You're like running a bunch of different You're like running a bunch of different sessions. Like, how do we know that sessions. Like, how do we know that sessions. Like, how do we know that these sessions are true, meaningful, and these sessions are true, meaningful, and these sessions are true, meaningful, and valuable?" So, the second part of that valuable?" So, the second part of that valuable?" So, the second part of that is, okay, is, okay, is, okay, we can think about how we've reduced we can think about how we've reduced we can think about how we've reduced timelines for delivery projects timelines for delivery projects timelines for delivery projects on an order of magnitude. So, about like on an order of magnitude. So, about like on an order of magnitude. So, about like 82% reduction across like delivery. So, 82% reduction across like delivery. So, 82% reduction across like delivery. So, if you subscribe to the agile deliver if you subscribe to the agile deliver if you subscribe to the agile deliver agile development methodology, obviously agile development methodology, obviously agile development methodology, obviously like you have tickets, you have sprints, like you have tickets, you have sprints, like you have tickets, you have sprints, like you have things that need to be like you have things that need to be like you have things that need to be built. If you look at every single built. If you look at every single built. If you look at every single metric that you measure before you bring metric that you measure before you bring metric that you measure before you bring in Devin and after you bring in Devin, in Devin and after you bring in Devin, in Devin and after you bring in Devin, you can take a look at that. We can you can take a look at that. We can you can take a look at that. We can compress this timeline compress this timeline compress this timeline by a factor of like 82%. So, across the by a factor of like 82%. So, across the by a factor of like 82%. So, across the board, whenever we get developed, board, whenever we get developed, board, whenever we get developed, whenever we get deployed, and fully whenever we get deployed, and fully whenever we get deployed, and fully activated within the customer activated within the customer activated within the customer environment, not only do we deliver environment, not only do we deliver environment, not only do we deliver a massive scale in terms of like a massive scale in terms of like a massive scale in terms of like engineering capacity, but we also reduce engineering capacity, but we also reduce engineering capacity, but we also reduce the time to value in terms of bringing

the time to value in terms of bringing the time to value in terms of bringing things to market. things to market. things to market. Now, the third part of that is, hey, but Now, the third part of that is, hey, but Now, the third part of that is, hey, but I actually really care about the I actually really care about the I actually really care about the numbers, right? I actually really want numbers, right? I actually really want numbers, right? I actually really want to see how many PRs are you actually to see how many PRs are you actually to see how many PRs are you actually shipping? Like, is this meaningful? Does shipping? Like, is this meaningful? Does shipping? Like, is this meaningful? Does this actually make sense? So, if you this actually make sense? So, if you this actually make sense? So, if you think about think about think about dissecting the numbers a little like one dissecting the numbers a little like one dissecting the numbers a little like one dimension further, and you want to just dimension further, and you want to just dimension further, and you want to just look at like the raw PRs that people are look at like the raw PRs that people are look at like the raw PRs that people are ripping across the enterprise, we ripping across the enterprise, we ripping across the enterprise, we deliver almost double the amount of PRs deliver almost double the amount of PRs deliver almost double the amount of PRs that engineers were able to do with that engineers were able to do with that engineers were able to do with single-point tools and before you single-point tools and before you single-point tools and before you brought in an agent harness like Devin. brought in an agent harness like Devin. brought in an agent harness like Devin. So, there's three proof points of So, there's three proof points of So, there's three proof points of anonymized case studies in which we are anonymized case studies in which we are anonymized case studies in which we are able to deliver value at scale able to deliver value at scale able to deliver value at scale and across like various different and across like various different and across like various different problem domains. What I'll say is What I'll say is these aren't like, you know, private these aren't like, you know, private these aren't like, you know, private case studies. We have a bunch of case studies. We have a bunch of case studies. We have a bunch of different public case studies as well. different public case studies as well. different public case studies as well. So, we partner with companies like So, we partner with companies like So, we partner with companies like Nubank, right? If you folks have ever Nubank, right? If you folks have ever Nubank, right? If you folks have ever gone to Latin America, gone to Latin America, gone to Latin America, you can understand that, you know, the you can understand that, you know, the you can understand that, you know, the there's a lot of developers there. there's a lot of developers there. there's a lot of developers there. There's a lot of projects that are There's a lot of projects that are There's a lot of projects that are tangentially related. So, specifically, tangentially related. So, specifically, tangentially related. So, specifically, like we can say that there was an ETL like we can say that there was an ETL like we can say that there was an ETL migration. They had 50 engineers migration. They had 50 engineers migration. They had 50 engineers staffing this migration. We were able to staffing this migration. We were able to staffing this migration. We were able to deliver this within, um, I think like deliver this within, um, I think like deliver this within, um, I think like 1/3 of the timeline.

1/3 of the timeline. 1/3 of the timeline. Just with Devin autonomously. Just with Devin autonomously. Just with Devin autonomously. We have another bank, right? We have We have another bank, right? We have We have another bank, right? We have another use case where we work with one another use case where we work with one another use case where we work with one of the largest banks in Latin America. of the largest banks in Latin America. of the largest banks in Latin America. They were trying to migrate like the tax They were trying to migrate like the tax They were trying to migrate like the tax identification system. I know, like identification system. I know, like identification system. I know, like rocket science, right? Um, rocket science, right? Um, rocket science, right? Um, but the idea is that we were able to but the idea is that we were able to but the idea is that we were able to deliver this with half of the amount of deliver this with half of the amount of deliver this with half of the amount of effort actually required. effort actually required. effort actually required. So, if you think about like legacy So, if you think about like legacy So, if you think about like legacy languages like COBOL, if you think about languages like COBOL, if you think about languages like COBOL, if you think about things like JCLs, you think about things things like JCLs, you think about things things like JCLs, you think about things that like people don't learn anymore that like people don't learn anymore that like people don't learn anymore just because it's like not fun and not just because it's like not fun and not just because it's like not fun and not interesting, we're able to operate interesting, we're able to operate interesting, we're able to operate across some of the most complicated across some of the most complicated across some of the most complicated codebases in the world and deliver codebases in the world and deliver codebases in the world and deliver results that actually matter. results that actually matter. results that actually matter. And last but not least, obviously like And last but not least, obviously like And last but not least, obviously like if we think about the built card, um, if we think about the built card, um, if we think about the built card, um, specifically, we're able to actually specifically, we're able to actually specifically, we're able to actually merge like an order of magnitude more, merge like an order of magnitude more, merge like an order of magnitude more, um, in terms of like PR acceptance rate. um, in terms of like PR acceptance rate. um, in terms of like PR acceptance rate. We deliver like 10x per sub like worth We deliver like 10x per sub like worth We deliver like 10x per sub like worth of engineering talent like every single of engineering talent like every single of engineering talent like every single week. And then we're actually able to, week. And then we're actually able to, week. And then we're actually able to, you know, generate the weekly output of you know, generate the weekly output of you know, generate the weekly output of like over 10 engineers at the like over 10 engineers at the like over 10 engineers at the organization. Built has great engineers, organization. Built has great engineers, organization. Built has great engineers, by the way, right? These guys are so by the way, right? These guys are so by the way, right? These guys are so cracked. cracked. cracked. So, if you take one of these engineers So, if you take one of these engineers So, if you take one of these engineers and multiply them by 10, and multiply them by 10, and multiply them by 10, you just imagine the amount of returns.

you just imagine the amount of returns. you just imagine the amount of returns. So, what I'll say is I'll I'll probably So, what I'll say is I'll I'll probably So, what I'll say is I'll I'll probably like round off this talk by just saying like round off this talk by just saying like round off this talk by just saying that at Cognition, we don't just, you that at Cognition, we don't just, you that at Cognition, we don't just, you know, embed ourselves in the customers. know, embed ourselves in the customers. know, embed ourselves in the customers. We don't just, you know, propagate We don't just, you know, propagate We don't just, you know, propagate feedback back to everybody else. feedback back to everybody else. feedback back to everybody else. But, the core values of the company But, the core values of the company But, the core values of the company are things and principles that we are things and principles that we are things and principles that we subscribe to not internally to the subscribe to not internally to the subscribe to not internally to the company, but external to the company as company, but external to the company as company, but external to the company as well, right? It's really fun being on well, right? It's really fun being on well, right? It's really fun being on the winning team. It's really fun when the winning team. It's really fun when the winning team. It's really fun when you come into an organization and say, you come into an organization and say, you come into an organization and say, "We can actually deliver so much cool "We can actually deliver so much cool "We can actually deliver so much cool stuff stuff stuff and like make people our champions, and like make people our champions, and like make people our champions, right? Whoever deploys Devin within the right? Whoever deploys Devin within the right? Whoever deploys Devin within the organization, they can show results that organization, they can show results that organization, they can show results that are essentially unmatched across the are essentially unmatched across the are essentially unmatched across the board." board." board." And we go for it all, right? We leave And we go for it all, right? We leave And we go for it all, right? We leave nothing on the table. We've deployed nothing on the table. We've deployed nothing on the table. We've deployed somebody in Brazil for like 10 months. somebody in Brazil for like 10 months. somebody in Brazil for like 10 months. >> [laughter] >> [laughter] >> [laughter] [snorts] [snorts] [snorts] >> To to live next to one of the customers >> To to live next to one of the customers >> To to live next to one of the customers to just make them successful. to just make them successful. to just make them successful. So, we're down for the mission. So, we're down for the mission. So, we're down for the mission. And it's it's more about like And it's it's more about like And it's it's more about like correctness, right? Like if if there are correctness, right? Like if if there are correctness, right? Like if if there are engineering practices that we want to engineering practices that we want to engineering practices that we want to fix, if there are things that we want to fix, if there are things that we want to fix, if there are things that we want to flag and raise, like these are all flag and raise, like these are all flag and raise, like these are all things that we take back to product and things that we take back to product and things that we take back to product and there's no ego involved. At the end of there's no ego involved. At the end of there's no ego involved. At the end of the day, we are all in the same boat. the day, we are all in the same boat. the day, we are all in the same boat. We're on the same mission and we're just We're on the same mission and we're just We're on the same mission and we're just shipping.

shipping. shipping. And everybody essentially is And everybody essentially is And everybody essentially is go-to-market. I know forward deployed go-to-market. I know forward deployed go-to-market. I know forward deployed engineering is kind of like this fuzzy engineering is kind of like this fuzzy engineering is kind of like this fuzzy thing where it's like, "Am I part of thing where it's like, "Am I part of thing where it's like, "Am I part of sales? Am I part of post-sales? Like sales? Am I part of post-sales? Like sales? Am I part of post-sales? Like what do I actually do as an FDE?" But, what do I actually do as an FDE?" But, what do I actually do as an FDE?" But, everybody is go-to-market because the everybody is go-to-market because the everybody is go-to-market because the target is to make the customer target is to make the customer target is to make the customer successful at all costs. successful at all costs. successful at all costs. And at the end of the day, we just do And at the end of the day, we just do And at the end of the day, we just do things things things because every second counts. because every second counts. because every second counts. So, if you're interested in, you know, So, if you're interested in, you know, So, if you're interested in, you know, forward deployed engineering at forward deployed engineering at forward deployed engineering at Cognition, being the intersection Cognition, being the intersection Cognition, being the intersection between some of the hardest problems in between some of the hardest problems in between some of the hardest problems in this world, being part of like all of this world, being part of like all of this world, being part of like all of the software disruption at scale, and the software disruption at scale, and the software disruption at scale, and then being on the other side then being on the other side then being on the other side of these problems, of these problems, of these problems, we should talk. we should talk. we should talk. Thank you.