Always-on agents run production without the on-call tax — Justin Smith, Resolve AI

>> Hello. Hello. >> Hello. Hello. Everybody, welcome to Everybody, welcome to Everybody, welcome to this talk always on agents run this talk always on agents run this talk always on agents run production without the on-call tax. production without the on-call tax. production without the on-call tax. My name is Justin Smith. One of the My name is Justin Smith. One of the My name is Justin Smith. One of the founding product engineers at Resolve founding product engineers at Resolve founding product engineers at Resolve AI. Been in the space for about 15 plus AI. Been in the space for about 15 plus AI. Been in the space for about 15 plus years in the sort of monitoring, years in the sort of monitoring, years in the sort of monitoring, observability, how do you kind of observability, how do you kind of observability, how do you kind of operate production systems space. operate production systems space. operate production systems space. Was at Splunk for a while. Was one of Was at Splunk for a while. Was one of Was at Splunk for a while. Was one of the architects on the observability the architects on the observability the architects on the observability suite there. Spent a good 10 year at suite there. Spent a good 10 year at suite there. Spent a good 10 year at VMware. VMware. VMware. Really really enjoy like product design Really really enjoy like product design Really really enjoy like product design and front-end architecture. How do you and front-end architecture. How do you and front-end architecture. How do you How do you How do people experience How do you How do people experience How do you How do people experience a product or use case or something like a product or use case or something like a product or use case or something like that? That's the stuff I like to that? That's the stuff I like to that? That's the stuff I like to dabble in. dabble in. dabble in. Um Um Um but I want to talk a little bit about but I want to talk a little bit about but I want to talk a little bit about the first wave of the first wave of the first wave of AI and it's it's been a fun one. I think AI and it's it's been a fun one. I think AI and it's it's been a fun one. I think the first big wave and I'm sure we've the first big wave and I'm sure we've the first big wave and I'm sure we've all experienced this is just how we all experienced this is just how we all experienced this is just how we build software. Um build software. Um build software. Um but there's some sort of net effects of but there's some sort of net effects of but there's some sort of net effects of that. It's a lot of bigger PRs that are that. It's a lot of bigger PRs that are that. It's a lot of bigger PRs that are coming through.

coming through. coming through. We definitely see a lot of this a lot We definitely see a lot of this a lot We definitely see a lot of this a lot more frequently. So people are shipping more frequently. So people are shipping more frequently. So people are shipping code at a much, you know, faster rate code at a much, you know, faster rate code at a much, you know, faster rate from developers and we're beginning to from developers and we're beginning to from developers and we're beginning to see maybe from even non-developers see maybe from even non-developers see maybe from even non-developers that maybe don't actually know the code that maybe don't actually know the code that maybe don't actually know the code or what it's doing or sort of like or what it's doing or sort of like or what it's doing or sort of like operating principles behind it. operating principles behind it. operating principles behind it. But we're getting But we're getting But we're getting developer productivity. developer productivity. developer productivity. And that's good, right? That's that's a And that's good, right? That's that's a And that's good, right? That's that's a good thing that we're all able to sort good thing that we're all able to sort good thing that we're all able to sort of produce more and faster. of produce more and faster. of produce more and faster. Kind of sort of what we actually found Kind of sort of what we actually found Kind of sort of what we actually found out and this was a survey study done is out and this was a survey study done is out and this was a survey study done is that 70% of the time from an engineer is that 70% of the time from an engineer is that 70% of the time from an engineer is actually not just like is not focused actually not just like is not focused actually not just like is not focused just on writing code. It's actually just on writing code. It's actually just on writing code. It's actually spent on actually running the code that spent on actually running the code that spent on actually running the code that is actually shipped into production. is actually shipped into production. is actually shipped into production. Um maintaining all the the platforms, Um maintaining all the the platforms, Um maintaining all the the platforms, scaling the infrastructure, debugging scaling the infrastructure, debugging scaling the infrastructure, debugging all the incidents, and being on call, um all the incidents, and being on call, um all the incidents, and being on call, um shipping hot fixes, right? Dealing with shipping hot fixes, right? Dealing with shipping hot fixes, right? Dealing with alerts, um alerts, um alerts, um updating all the sort of run books and updating all the sort of run books and updating all the sort of run books and operating procedures, restoring operating procedures, restoring operating procedures, restoring services, dealing with escalations, services, dealing with escalations, services, dealing with escalations, dealing with sort of like um questions dealing with sort of like um questions dealing with sort of like um questions from other, you know, teams and things from other, you know, teams and things from other, you know, teams and things like that. So, really coding was never like that. So, really coding was never like that. So, really coding was never the the the big bottleneck, right? Um a the the the big bottleneck, right? Um a the the the big bottleneck, right? Um a lot of it was really around, uh thank lot of it was really around, uh thank lot of it was really around, uh thank you, granola. Um a lot of it was really you, granola. Um a lot of it was really you, granola. Um a lot of it was really around like how do we actually run these around like how do we actually run these around like how do we actually run these things sort of in production.

And that's getting harder and harder and And that's getting harder and harder and harder. Um harder. Um harder. Um AI is creating a lot more issues in AI is creating a lot more issues in AI is creating a lot more issues in production as, you know, AI code sort of production as, you know, AI code sort of production as, you know, AI code sort of goes through. Um it's not clear we have goes through. Um it's not clear we have goes through. Um it's not clear we have the right sort of um structures in place the right sort of um structures in place the right sort of um structures in place to deal with the amount of kind of to deal with the amount of kind of to deal with the amount of kind of changes that are coming through. Um changes that are coming through. Um changes that are coming through. Um unlimited tokens is is sort of coming to unlimited tokens is is sort of coming to unlimited tokens is is sort of coming to an end, the the token max, right? an end, the the token max, right? an end, the the token max, right? They're starting to clamp down. Prices They're starting to clamp down. Prices They're starting to clamp down. Prices are going up. Companies are getting a are going up. Companies are getting a are going up. Companies are getting a lot more stringent on, you know, what's lot more stringent on, you know, what's lot more stringent on, you know, what's being used um for AI. Um you know, we being used um for AI. Um you know, we being used um for AI. Um you know, we need full stack AI. It's not just about need full stack AI. It's not just about need full stack AI. It's not just about the models anymore, it's about the the models anymore, it's about the the models anymore, it's about the context around the models and what the context around the models and what the context around the models and what the models can do inside of a specific models can do inside of a specific models can do inside of a specific domain. These become the the problem domain. These become the the problem domain. These become the the problem areas that we need to sort of uh focus areas that we need to sort of uh focus areas that we need to sort of uh focus in and tackle on. in and tackle on. in and tackle on. And this is true today, right? So, it's And this is true today, right? So, it's And this is true today, right? So, it's it's creating more sort of uh complexity it's creating more sort of uh complexity it's creating more sort of uh complexity inside of our environment. But, I mean inside of our environment. But, I mean inside of our environment. But, I mean the the reality is that systems have the the reality is that systems have the the reality is that systems have always been complex. That's why we have, always been complex. That's why we have, always been complex. That's why we have, you know, these big tools that can, you you know, these big tools that can, you you know, these big tools that can, you know, try to give us insights into these know, try to give us insights into these know, try to give us insights into these systems. Um systems. Um systems. Um there are multiple teams, there's there are multiple teams, there's there are multiple teams, there's multiple systems that are all having to multiple systems that are all having to multiple systems that are all having to work together, and they all have their work together, and they all have their work together, and they all have their own, you know, goals that they're trying own, you know, goals that they're trying own, you know, goals that they're trying to deliver towards, but you have to deliver towards, but you have to deliver towards, but you have organizational goals. And how do you organizational goals. And how do you organizational goals. And how do you keep all of this sort of uh you know, um keep all of this sort of uh you know, um keep all of this sort of uh you know, um in in in Get rid of this. Um how do you keep all Get rid of this. Um how do you keep all Get rid of this. Um how do you keep all of this in balance, right? Um how do of this in balance, right? Um how do of this in balance, right? Um how do you, you, you, you know, pull all of this stuff you know, pull all of this stuff you know, pull all of this stuff together in a way that actually uh together in a way that actually uh together in a way that actually uh actually helps you and and facilitates actually helps you and and facilitates actually helps you and and facilitates your uh your uh your uh your organization. Um and the answer is, your organization. Um and the answer is, your organization. Um and the answer is, well, you got to use AI inside of well, you got to use AI inside of well, you got to use AI inside of production to deal with um sort of the production to deal with um sort of the production to deal with um sort of the the amount of increase of complexity the amount of increase of complexity the amount of increase of complexity that AI is kind of putting into your that AI is kind of putting into your that AI is kind of putting into your product or into your system.

product or into your system. product or into your system. Um and so that's where Resolve, you Um and so that's where Resolve, you Um and so that's where Resolve, you know, this was kind of our sort of know, this was kind of our sort of know, this was kind of our sort of hypothesis from the beginning was um you hypothesis from the beginning was um you hypothesis from the beginning was um you know, we're going to see an influx in um know, we're going to see an influx in um know, we're going to see an influx in um you know, issues uh coming out of coding you know, issues uh coming out of coding you know, issues uh coming out of coding um just the increase in coding uh um just the increase in coding uh um just the increase in coding uh velocity. Um there's going to be more velocity. Um there's going to be more velocity. Um there's going to be more need for kind of AI to actually operate need for kind of AI to actually operate need for kind of AI to actually operate and run these run these systems. Um and run these run these systems. Um and run these run these systems. Um we're, you know, lucky to work with, you we're, you know, lucky to work with, you we're, you know, lucky to work with, you know, some uh world-class engineering know, some uh world-class engineering know, some uh world-class engineering teams that are solving like really teams that are solving like really teams that are solving like really difficult problems at, you know, crazy difficult problems at, you know, crazy difficult problems at, you know, crazy scale. Um and you know, that that gives scale. Um and you know, that that gives scale. Um and you know, that that gives us insight into a lot of how bigger us insight into a lot of how bigger us insight into a lot of how bigger organizations are having to deal with organizations are having to deal with organizations are having to deal with the influx of AI, etc. the influx of AI, etc. the influx of AI, etc. Um Resolve itself uh hosts a bunch of Um Resolve itself uh hosts a bunch of Um Resolve itself uh hosts a bunch of different sort of capabilities. Um we different sort of capabilities. Um we different sort of capabilities. Um we have a number of agents that sort of you have a number of agents that sort of you have a number of agents that sort of you know, you get to kind of experience. One know, you get to kind of experience. One know, you get to kind of experience. One of them is just an on-call agent. This of them is just an on-call agent. This of them is just an on-call agent. This is kind of where we started, right? Um is kind of where we started, right? Um is kind of where we started, right? Um so for every alert that comes in, um we so for every alert that comes in, um we so for every alert that comes in, um we can do a triage of that alert. We can do can do a triage of that alert. We can do can do a triage of that alert. We can do kind of a full root cause investigation kind of a full root cause investigation kind of a full root cause investigation of that alert. Um and you know, this is of that alert. Um and you know, this is of that alert. Um and you know, this is for anybody that's had to be on call for anybody that's had to be on call for anybody that's had to be on call before, um you know, on call is a it's a before, um you know, on call is a it's a before, um you know, on call is a it's a nightmare, right? Um you're often uh nightmare, right? Um you're often uh nightmare, right? Um you're often uh only going on call every few weeks. Uh only going on call every few weeks. Uh only going on call every few weeks. Uh you you maybe don't fully understand all you you maybe don't fully understand all you you maybe don't fully understand all the changes that have come in. You don't the changes that have come in. You don't the changes that have come in. You don't fully understand maybe all the different fully understand maybe all the different fully understand maybe all the different systems that you're having to interact systems that you're having to interact systems that you're having to interact with. Um and so, you know, the with. Um and so, you know, the with. Um and so, you know, the complexity is already there and having complexity is already there and having complexity is already there and having an AI agent that's able to come support an AI agent that's able to come support an AI agent that's able to come support you and pull that context together is you and pull that context together is you and pull that context together is incredibly incredibly valuable. Um and incredibly incredibly valuable. Um and incredibly incredibly valuable. Um and so, uh that can often grow from just so, uh that can often grow from just so, uh that can often grow from just sort of getting a single page into a sort of getting a single page into a sort of getting a single page into a much larger incident across many much larger incident across many much larger incident across many different teams um across an

different teams um across an different teams um across an organization. Um and we have agents organization. Um and we have agents organization. Um and we have agents there to come support that much larger there to come support that much larger there to come support that much larger activity of, you know, all all kind of activity of, you know, all all kind of activity of, you know, all all kind of cross-collaboration, etc. Keeping cross-collaboration, etc. Keeping cross-collaboration, etc. Keeping everybody uh in sync and aligned on uh everybody uh in sync and aligned on uh everybody uh in sync and aligned on uh um where the where the incident is um where the where the incident is um where the where the incident is happening, what the impact of that is, happening, what the impact of that is, happening, what the impact of that is, um etc. um etc. um etc. And then we also focus a lot on And then we also focus a lot on And then we also focus a lot on background agents. And this is sort of background agents. And this is sort of background agents. And this is sort of covering the long tail of, you know, covering the long tail of, you know, covering the long tail of, you know, what happens when there's not a fire what happens when there's not a fire what happens when there's not a fire brewing um or or going on at any one, brewing um or or going on at any one, brewing um or or going on at any one, you know, point. Um there's still lots you know, point. Um there's still lots you know, point. Um there's still lots of operational work that you as an of operational work that you as an of operational work that you as an engineer or an engineering team have to engineer or an engineering team have to engineer or an engineering team have to do and a lot of ceremonies of, you know, do and a lot of ceremonies of, you know, do and a lot of ceremonies of, you know, passing contact context off or dealing passing contact context off or dealing passing contact context off or dealing with kind of one-off issues or kind of with kind of one-off issues or kind of with kind of one-off issues or kind of having to scratch that itch in the back having to scratch that itch in the back having to scratch that itch in the back of your head of like, is that part of of your head of like, is that part of of your head of like, is that part of the system okay or not okay? Um and the system okay or not okay? Um and the system okay or not okay? Um and you're constantly having to sort of you're constantly having to sort of you're constantly having to sort of balance across all the these different balance across all the these different balance across all the these different things. Um underneath all of that, you things. Um underneath all of that, you things. Um underneath all of that, you know, the we have an agent uh know, the we have an agent uh know, the we have an agent uh architecture um deals with models and architecture um deals with models and architecture um deals with models and context and reasoning and and actions. context and reasoning and and actions. context and reasoning and and actions. Um Um Um learning is a a I'll sort of like half learning is a a I'll sort of like half learning is a a I'll sort of like half pause on that one. I think, you know, pause on that one. I think, you know, pause on that one. I think, you know, some of the biggest issues that we've some of the biggest issues that we've some of the biggest issues that we've seen, it's it's not that a a model by seen, it's it's not that a a model by seen, it's it's not that a a model by itself is is not smart or whatever. I itself is is not smart or whatever. I itself is is not smart or whatever. I mean, models have gotten incredi- mean, models have gotten incredi- mean, models have gotten incredi- incredibly capable over the last um incredibly capable over the last um incredibly capable over the last um year, let's say, right? But especially year, let's say, right? But especially year, let's say, right? But especially over the last like 6 months or so. Um over the last like 6 months or so. Um over the last like 6 months or so. Um but the idea of understanding, like but the idea of understanding, like but the idea of understanding, like truly understanding your environment um truly understanding your environment um truly understanding your environment um and the way that your services interact and the way that your services interact and the way that your services interact and where the hotspots are, keeping and where the hotspots are, keeping and where the hotspots are, keeping track of all of that sort of track of all of that sort of track of all of that sort of understanding is incredibly difficult.

understanding is incredibly difficult. understanding is incredibly difficult. But it's incredibly important for any But it's incredibly important for any But it's incredibly important for any model to be successful at, you know, the model to be successful at, you know, the model to be successful at, you know, the task that it needs to do. It has to have task that it needs to do. It has to have task that it needs to do. It has to have an underlying sort of learning system to an underlying sort of learning system to an underlying sort of learning system to be able to capture that knowledge um and be able to capture that knowledge um and be able to capture that knowledge um and that sort of understanding of how your that sort of understanding of how your that sort of understanding of how your system operates. Um so we spend a lot of system operates. Um so we spend a lot of system operates. Um so we spend a lot of time thinking about how do we have time thinking about how do we have time thinking about how do we have systems that not just can understand systems that not just can understand systems that not just can understand your environment at any one point, but your environment at any one point, but your environment at any one point, but grow as as your system evolves? Because grow as as your system evolves? Because grow as as your system evolves? Because again, your system is evolving faster again, your system is evolving faster again, your system is evolving faster and faster. We need to keep up with and faster. We need to keep up with and faster. We need to keep up with learning about what's the current state, learning about what's the current state, learning about what's the current state, um what's the current sort of causal um what's the current sort of causal um what's the current sort of causal chains that we need to be sort of chains that we need to be sort of chains that we need to be sort of keeping keeping an eye on. Um and then keeping keeping an eye on. Um and then keeping keeping an eye on. Um and then of course all the enterprise of course all the enterprise of course all the enterprise enterprise stuff sort of underneath. enterprise stuff sort of underneath. enterprise stuff sort of underneath. Um and so this is that same view kind of Um and so this is that same view kind of Um and so this is that same view kind of uh packed out. Um uh packed out. Um uh packed out. Um Come on. Okay. Um, but so today so we do Come on. Okay. Um, but so today so we do Come on. Okay. Um, but so today so we do a lot at Resolve um, the on-call and the a lot at Resolve um, the on-call and the a lot at Resolve um, the on-call and the incident stuff. I'm going to focus a lot incident stuff. I'm going to focus a lot incident stuff. I'm going to focus a lot uh, more just on the background agent uh, more just on the background agent uh, more just on the background agent um, stuff. So how do we deal with the um, stuff. So how do we deal with the um, stuff. So how do we deal with the things that maybe aren't sort of things that maybe aren't sort of things that maybe aren't sort of immediate fires. Um, if you have immediate fires. Um, if you have immediate fires. Um, if you have questions about the immediate fire questions about the immediate fire questions about the immediate fire stuff, um, we have a booth down in the stuff, um, we have a booth down in the stuff, um, we have a booth down in the expo. Please come check it out. Uh, our expo. Please come check it out. Uh, our expo. Please come check it out. Uh, our team would love to demo to you etc. Uh, team would love to demo to you etc. Uh, team would love to demo to you etc. Uh, but today we're going to focus on the but today we're going to focus on the but today we're going to focus on the background agent.

background agent. background agent. So kind of a little pop quiz. Um, feel So kind of a little pop quiz. Um, feel So kind of a little pop quiz. Um, feel free to raise your hands. Is anybody free to raise your hands. Is anybody free to raise your hands. Is anybody using agents in part as part of your using agents in part as part of your using agents in part as part of your daily workflow? Maybe outside of the daily workflow? Maybe outside of the daily workflow? Maybe outside of the coding. I'm assuming everybody's doing coding. I'm assuming everybody's doing coding. I'm assuming everybody's doing coding agents these days. Is anybody coding agents these days. Is anybody coding agents these days. Is anybody doing like actually running sort of um, doing like actually running sort of um, doing like actually running sort of um, agents that are sort of helping in other agents that are sort of helping in other agents that are sort of helping in other ways? Okay. Uh, decent amount. Any good ways? Okay. Uh, decent amount. Any good ways? Okay. Uh, decent amount. Any good examples? Any fun stuff that anybody examples? Any fun stuff that anybody examples? Any fun stuff that anybody has? You can just yell it out. has? You can just yell it out. has? You can just yell it out. >> Meeting review. >> Meeting review. >> Meeting review. >> Meeting reviews? Yeah, meeting reviews. >> Meeting reviews? Yeah, meeting reviews. >> Meeting reviews? Yeah, meeting reviews. I just had my granola show up and I just had my granola show up and I just had my granola show up and >> Market research. >> Market research. >> Market research. >> Market research. I do a lot of. >> Market research. I do a lot of. >> Market research. I do a lot of. >> [laughter] >> [laughter] >> [laughter] >> So let's let's have a good conversation >> So let's let's have a good conversation >> So let's let's have a good conversation about. Yeah, yeah, yeah, I do that all about. Yeah, yeah, yeah, I do that all about. Yeah, yeah, yeah, I do that all the time. Any other ones? Maybe one the time. Any other ones? Maybe one the time. Any other ones? Maybe one more? more? more? >> Therapy. >> Therapy. >> Therapy. >> What's the one? >> What's the one? >> What's the one? >> Therapy. >> Therapy. >> Therapy. >> I can't hear it. >> I can't hear it. >> I can't hear it. >> Therapy. >> Therapy. >> Therapy. >> Therapy. That's a fantastic one >> Therapy. That's a fantastic one >> Therapy. That's a fantastic one actually. We're we we are humans here actually. We're we we are humans here actually. We're we we are humans here today. This is very important. Um, today. This is very important. Um, today. This is very important. Um, that's actually that's a very good one. that's actually that's a very good one. that's actually that's a very good one. Um, okay. So so people are having some Um, okay. So so people are having some Um, okay. So so people are having some stuff uh, going on. Um, stuff uh, going on. Um, stuff uh, going on. Um, So, you know, and this is kind of recaps So, you know, and this is kind of recaps So, you know, and this is kind of recaps a little bit uh, again. A lot of a little bit uh, again. A lot of a little bit uh, again. A lot of production work is not about is not production work is not about is not production work is not about is not there's not a sort of big ceremony that there's not a sort of big ceremony that there's not a sort of big ceremony that everyone is focused on for the the type everyone is focused on for the the type everyone is focused on for the the type of work that we have to do. Um, on-call of work that we have to do. Um, on-call of work that we have to do. Um, on-call you you've got a page that goes off. You you you've got a page that goes off. You you you've got a page that goes off. You know somebody's going to receive that.

know somebody's going to receive that. know somebody's going to receive that. Incidents you create a bridge, you Incidents you create a bridge, you Incidents you create a bridge, you invite people in. That's great. Um, but invite people in. That's great. Um, but invite people in. That's great. Um, but there's just a long tail of other things there's just a long tail of other things there's just a long tail of other things that we are accountable for that doesn't that we are accountable for that doesn't that we are accountable for that doesn't have sort of a thing that's going to have sort of a thing that's going to have sort of a thing that's going to show up in your sort of job description show up in your sort of job description show up in your sort of job description of like this is what you're going to be, of like this is what you're going to be, of like this is what you're going to be, you know, responsible for. Um, watching you know, responsible for. Um, watching you know, responsible for. Um, watching deploys that go out and make sure that deploys that go out and make sure that deploys that go out and make sure that they're actually getting out um, they're actually getting out um, they're actually getting out um, healthy. Um, a morning report uh, healthy. Um, a morning report uh, healthy. Um, a morning report uh, incident digest of just like what's the incident digest of just like what's the incident digest of just like what's the state of my system today so that we're state of my system today so that we're state of my system today so that we're all on the same page. Um, hey that P99 all on the same page. Um, hey that P99 all on the same page. Um, hey that P99 drift kind of came back. Is somebody drift kind of came back. Is somebody drift kind of came back. Is somebody looking at that or not? looking at that or not? looking at that or not? And you know, this is pulling people in And you know, this is pulling people in And you know, this is pulling people in to to try to like figure out what's to to try to like figure out what's to to try to like figure out what's going on. This may not be paging, right? going on. This may not be paging, right? going on. This may not be paging, right? Because we don't we're not going to Because we don't we're not going to Because we don't we're not going to alert on everything. alert on everything. alert on everything. Produce the capacity report, right? Like Produce the capacity report, right? Like Produce the capacity report, right? Like are we tracking okay, right? This is are we tracking okay, right? This is are we tracking okay, right? This is maybe a company goal this this quarter. maybe a company goal this this quarter. maybe a company goal this this quarter. Are we tracking against that? Somebody's Are we tracking against that? Somebody's Are we tracking against that? Somebody's going to have to be responsible for going to have to be responsible for going to have to be responsible for doing that. The recurring health check doing that. The recurring health check doing that. The recurring health check and just kind of checking and making and just kind of checking and making and just kind of checking and making sure things are kind of running okay and sure things are kind of running okay and sure things are kind of running okay and not waiting for a customer to come not waiting for a customer to come not waiting for a customer to come complain first. So this work doesn't complain first. So this work doesn't complain first. So this work doesn't have like an obvious like, oh, this you have like an obvious like, oh, this you have like an obvious like, oh, this you know, this now needs to go be done.

know, this now needs to go be done. know, this now needs to go be done. But it's work that we end up having to But it's work that we end up having to But it's work that we end up having to do. do. do. So what is a task? A task is just So what is a task? A task is just So what is a task? A task is just execution and the context to understand execution and the context to understand execution and the context to understand how to actually execute the task. how to actually execute the task. how to actually execute the task. Execution Execution Execution is very very important. It's is very very important. It's is very very important. It's understanding what to do and being able understanding what to do and being able understanding what to do and being able to execute that. Maybe having access to to execute that. Maybe having access to to execute that. Maybe having access to the tools, etc., right? the tools, etc., right? the tools, etc., right? Obviously very important to do. Obviously very important to do. Obviously very important to do. But we think the production context is But we think the production context is But we think the production context is just way more important because it's one just way more important because it's one just way more important because it's one thing to go check a dashboard. It's thing to go check a dashboard. It's thing to go check a dashboard. It's another thing to say that metric smells another thing to say that metric smells another thing to say that metric smells off. off. off. And the execution is can load the And the execution is can load the And the execution is can load the dashboard. It's the production context dashboard. It's the production context dashboard. It's the production context that's going to say, this feels wrong. that's going to say, this feels wrong. that's going to say, this feels wrong. And I don't know if I can even explain And I don't know if I can even explain And I don't know if I can even explain why it feels wrong. It just feels wrong why it feels wrong. It just feels wrong why it feels wrong. It just feels wrong and I want to dig into the next layer of and I want to dig into the next layer of and I want to dig into the next layer of sort of understanding of that. sort of understanding of that. sort of understanding of that. And so really if if we start talking And so really if if we start talking And so really if if we start talking about background agents and being able about background agents and being able about background agents and being able to kind of perform task, you need both to kind of perform task, you need both to kind of perform task, you need both of these. You need the execution engine, of these. You need the execution engine, of these. You need the execution engine, that's great, but you really need that that's great, but you really need that that's great, but you really need that production context that tells you is production context that tells you is production context that tells you is this important or not important.

So every background agent, So every background agent, you know, there's a a few different you know, there's a a few different you know, there's a a few different principles that we like to think about principles that we like to think about principles that we like to think about with our background agents. When does it with our background agents. When does it with our background agents. When does it work? How does it work? How does it know work? How does it work? How does it know work? How does it work? How does it know what to go do? what to go do? what to go do? When does the agent work? When does the agent work? When does the agent work? It can work in a bunch of different It can work in a bunch of different It can work in a bunch of different ways. It can just do it on a schedule. ways. It can just do it on a schedule. ways. It can just do it on a schedule. Maybe this is the the report, etc. Just Maybe this is the the report, etc. Just Maybe this is the the report, etc. Just kind of do some summarization for me kind of do some summarization for me kind of do some summarization for me kind of on an ongoing basis. Maybe it's kind of on an ongoing basis. Maybe it's kind of on an ongoing basis. Maybe it's a weekly event, right? We do an on-call a weekly event, right? We do an on-call a weekly event, right? We do an on-call handover uh every Thursday, and so that handover uh every Thursday, and so that handover uh every Thursday, and so that a lot of um the work that our agent does a lot of um the work that our agent does a lot of um the work that our agent does is sort of prepare like what what are is sort of prepare like what what are is sort of prepare like what what are the kind of interesting trends from the the kind of interesting trends from the the kind of interesting trends from the last week that the next on-caller needs last week that the next on-caller needs last week that the next on-caller needs to sort of understand as they pick up to sort of understand as they pick up to sort of understand as they pick up the rotation. Um, the rotation. Um, the rotation. Um, event streams, so you know, there's lots event streams, so you know, there's lots event streams, so you know, there's lots of systems that will sort of push events of systems that will sort of push events of systems that will sort of push events as kind of key things happen. Um, so as kind of key things happen. Um, so as kind of key things happen. Um, so deployments go through a CI ICD deployments go through a CI ICD deployments go through a CI ICD pipeline. pipeline. pipeline. Um, there's other sort of uh Um, there's other sort of uh Um, there's other sort of uh Slack-based, right? We get a lot of uh Slack-based, right? We get a lot of uh Slack-based, right? We get a lot of uh Slack things messages coming through, um Slack things messages coming through, um Slack things messages coming through, um etc. Um, and these are things that we etc. Um, and these are things that we etc. Um, and these are things that we can sort of pick up and trigger and say, can sort of pick up and trigger and say, can sort of pick up and trigger and say, "Oh, if this event happens, um let me "Oh, if this event happens, um let me "Oh, if this event happens, um let me sort of understand what that event is sort of understand what that event is sort of understand what that event is and go do some work." Um, and then and go do some work." Um, and then and go do some work." Um, and then message-based, so I can just tell it, message-based, so I can just tell it, message-based, so I can just tell it, "Hey, go do some work." And it will go "Hey, go do some work." And it will go "Hey, go do some work." And it will go do some work. That's fantastic.

do some work. That's fantastic. do some work. That's fantastic. Um, how does it run? Always runs. It's Um, how does it run? Always runs. It's Um, how does it run? Always runs. It's in the cloud. Um, so if you close your in the cloud. Um, so if you close your in the cloud. Um, so if you close your laptop, it's okay. Um, runs inside of a laptop, it's okay. Um, runs inside of a laptop, it's okay. Um, runs inside of a sandbox, so it has kind of a file system sandbox, so it has kind of a file system sandbox, so it has kind of a file system underneath it. Um, this allows it to underneath it. Um, this allows it to underneath it. Um, this allows it to sort of self-organize a lot of its work, sort of self-organize a lot of its work, sort of self-organize a lot of its work, etc. as it's doing uh doing things. Um, etc. as it's doing uh doing things. Um, etc. as it's doing uh doing things. Um, and then obviously back to the learning and then obviously back to the learning and then obviously back to the learning loop, right? So, that idea of knowledge loop, right? So, that idea of knowledge loop, right? So, that idea of knowledge and sort of a memory system underneath and sort of a memory system underneath and sort of a memory system underneath that um to really understand your that um to really understand your that um to really understand your systems and as it's doing a task, able systems and as it's doing a task, able systems and as it's doing a task, able to sort of reflect on that task and uh to sort of reflect on that task and uh to sort of reflect on that task and uh you know, do a better job next time. Or you know, do a better job next time. Or you know, do a better job next time. Or the things that it learned from one the things that it learned from one the things that it learned from one task, it can sort of apply into a task, it can sort of apply into a task, it can sort of apply into a different task. Um, because again, this different task. Um, because again, this different task. Um, because again, this this sort of shared uh sort of knowledge this sort of shared uh sort of knowledge this sort of shared uh sort of knowledge system um works across all the different system um works across all the different system um works across all the different tasks that we have. tasks that we have. tasks that we have. Um, Um, Um, so how does the agent know what to do? so how does the agent know what to do? so how does the agent know what to do? Um, it has a task system. It can pull in Um, it has a task system. It can pull in Um, it has a task system. It can pull in all the skills that you have in other all the skills that you have in other all the skills that you have in other systems, that's fine. You can connect systems, that's fine. You can connect systems, that's fine. You can connect those. Um, and it's got obviously the those. Um, and it's got obviously the those. Um, and it's got obviously the integrations that it's going to plug integrations that it's going to plug integrations that it's going to plug into. Um, so let's talk a little bit into. Um, so let's talk a little bit into. Um, so let's talk a little bit about what types of things you uh can about what types of things you uh can about what types of things you uh can hand over. Um, we've got four sort of hand over. Um, we've got four sort of hand over. Um, we've got four sort of workloads that we're going to talk workloads that we're going to talk workloads that we're going to talk about, but if you think about the about, but if you think about the about, but if you think about the previous couple slides, these are sort previous couple slides, these are sort previous couple slides, these are sort of very basic primitives that we've of very basic primitives that we've of very basic primitives that we've built into the system. You can get very built into the system. You can get very built into the system. You can get very creative. We've We have a number of creative. We've We have a number of creative. We've We have a number of people inside of Resolve that have people inside of Resolve that have people inside of Resolve that have gotten very creative with the type of gotten very creative with the type of gotten very creative with the type of sort of background activities that sort of background activities that sort of background activities that things that that they that they have.

things that that they that they have. things that that they that they have. So, I want you to use these as kind of So, I want you to use these as kind of So, I want you to use these as kind of These are things we've seen be very These are things we've seen be very These are things we've seen be very successful successful successful inside of Resolve, but also with you inside of Resolve, but also with you inside of Resolve, but also with you know, a number of our customers. know, a number of our customers. know, a number of our customers. But, you know, sky's the limit and and But, you know, sky's the limit and and But, you know, sky's the limit and and you can get really creative. So, you can get really creative. So, you can get really creative. So, deployment monitoring. So, this is a big deployment monitoring. So, this is a big deployment monitoring. So, this is a big one, you know, any change inside of your one, you know, any change inside of your one, you know, any change inside of your environment is an opportunity for environment is an opportunity for environment is an opportunity for something to go wrong. Um and so, you something to go wrong. Um and so, you something to go wrong. Um and so, you know, having an agent that's able to know, having an agent that's able to know, having an agent that's able to watch as all these change events come in watch as all these change events come in watch as all these change events come in just to do a sanity check of is just to do a sanity check of is just to do a sanity check of is everything stable is incredibly everything stable is incredibly everything stable is incredibly incredibly important. And, you know, a incredibly important. And, you know, a incredibly important. And, you know, a lot of people have decent CICD system. I lot of people have decent CICD system. I lot of people have decent CICD system. I mean, this is like tried and true stuff mean, this is like tried and true stuff mean, this is like tried and true stuff that we've had as an industry for quite that we've had as an industry for quite that we've had as an industry for quite a while. a while. a while. But, we we noticed a couple gaps, But, we we noticed a couple gaps, But, we we noticed a couple gaps, you know, from in most of our customers. you know, from in most of our customers. you know, from in most of our customers. You know, typically the checks that it You know, typically the checks that it You know, typically the checks that it does are good. They're good baselines, does are good. They're good baselines, does are good. They're good baselines, but it's not exhaustive but it's not exhaustive but it's not exhaustive based on the type of changes that are based on the type of changes that are based on the type of changes that are going in etc. There's certain signals going in etc. There's certain signals going in etc. There's certain signals you'd want to watch or not want to you'd want to watch or not want to you'd want to watch or not want to watch. watch. watch. And so, every rollout is a bit unique. And so, every rollout is a bit unique. And so, every rollout is a bit unique. Often times you have change systems that Often times you have change systems that Often times you have change systems that you're not piping through a CICD system you're not piping through a CICD system you're not piping through a CICD system like a feature flag or maybe some infra like a feature flag or maybe some infra like a feature flag or maybe some infra changes that might happen which maybe changes that might happen which maybe changes that might happen which maybe don't get any monitoring at all. And don't get any monitoring at all. And don't get any monitoring at all. And you're sort of just trusting that an you're sort of just trusting that an you're sort of just trusting that an alert might fire and an on-caller will alert might fire and an on-caller will alert might fire and an on-caller will wake up and say, "Who changed what?"

wake up and say, "Who changed what?" wake up and say, "Who changed what?" right? right? right? Deployment monitoring is is actually a Deployment monitoring is is actually a Deployment monitoring is is actually a really big use case that we suggest really big use case that we suggest really big use case that we suggest people sort of go through and I'll show people sort of go through and I'll show people sort of go through and I'll show some examples of that in a second. some examples of that in a second. some examples of that in a second. Schedule health and anomaly checks. So, Schedule health and anomaly checks. So, Schedule health and anomaly checks. So, this is just sort of the ongoing this is just sort of the ongoing this is just sort of the ongoing periodic checking of some of your periodic checking of some of your periodic checking of some of your systems. systems. systems. And, you know, this is maybe something And, you know, this is maybe something And, you know, this is maybe something where it's like go check, you know, sort where it's like go check, you know, sort where it's like go check, you know, sort of my general dashboards on a routine of my general dashboards on a routine of my general dashboards on a routine basis maybe every morning just kind of basis maybe every morning just kind of basis maybe every morning just kind of do a casual check just to make sure do a casual check just to make sure do a casual check just to make sure there's nothing kind of weird from last there's nothing kind of weird from last there's nothing kind of weird from last night that I might need to be be of. night that I might need to be be of. night that I might need to be be of. But, uh, this can also just be sort of a But, uh, this can also just be sort of a But, uh, this can also just be sort of a time-based thing. Like I made a change time-based thing. Like I made a change time-based thing. Like I made a change in part of our system. I'm worried about in part of our system. I'm worried about in part of our system. I'm worried about this, you know, uh, you know, this, you know, uh, you know, this, you know, uh, you know, third-party service that I'm kind of third-party service that I'm kind of third-party service that I'm kind of interacting with. Let me just kind of interacting with. Let me just kind of interacting with. Let me just kind of set an agent to kind of watch that maybe set an agent to kind of watch that maybe set an agent to kind of watch that maybe for the next week just to make sure for the next week just to make sure for the next week just to make sure everything is kind of stable and then everything is kind of stable and then everything is kind of stable and then that agent can sort of, um, you know, that agent can sort of, um, you know, that agent can sort of, um, you know, stop his job. stop his job. stop his job. Uh, operational reports and handoffs, I Uh, operational reports and handoffs, I Uh, operational reports and handoffs, I talked a little bit about this. Uh, talked a little bit about this. Uh, talked a little bit about this. Uh, these are the sort of ceremonial things these are the sort of ceremonial things these are the sort of ceremonial things that we might want to do just to, you that we might want to do just to, you that we might want to do just to, you know, spread information, summarize know, spread information, summarize know, spread information, summarize things, kind of bring things to the things, kind of bring things to the things, kind of bring things to the fore. fore. fore. Um, and then a first responder to Um, and then a first responder to Um, and then a first responder to engineering questions. And this one's engineering questions. And this one's engineering questions. And this one's kind of fun a little bit, um, because kind of fun a little bit, um, because kind of fun a little bit, um, because the trigger for this is actually just a the trigger for this is actually just a the trigger for this is actually just a Slack message and I will say, um, Slack message and I will say, um, Slack message and I will say, um, one of my biggest, like, let's call it one of my biggest, like, let's call it one of my biggest, like, let's call it responsibilities, uh, as an engineer is responsibilities, uh, as an engineer is responsibilities, uh, as an engineer is watching all my Slack channels and watching all my Slack channels and watching all my Slack channels and trying to make sure everyone's kind of trying to make sure everyone's kind of trying to make sure everyone's kind of happy. Um, and like that nobody has any happy. Um, and like that nobody has any happy. Um, and like that nobody has any burning questions or anything like that.

burning questions or anything like that. burning questions or anything like that. Um, and so I can be sort of heads down Um, and so I can be sort of heads down Um, and so I can be sort of heads down trying to build something, um, and then, trying to build something, um, and then, trying to build something, um, and then, you know, the, you know, uh, eventual you know, the, you know, uh, eventual you know, the, you know, uh, eventual sort of Slack notification comes in that sort of Slack notification comes in that sort of Slack notification comes in that like this channel somebody asked this like this channel somebody asked this like this channel somebody asked this sort of kind of important question, um, sort of kind of important question, um, sort of kind of important question, um, and I just need to jump in there and and and I just need to jump in there and and and I just need to jump in there and and try to provide context, etc. It's not try to provide context, etc. It's not try to provide context, etc. It's not hard work. It's not hard for me to go hard work. It's not hard for me to go hard work. It's not hard for me to go answer questions, but it's disrupting me answer questions, but it's disrupting me answer questions, but it's disrupting me and if I don't go answer it, um, they and if I don't go answer it, um, they and if I don't go answer it, um, they won't get an answer for a while. And won't get an answer for a while. And won't get an answer for a while. And what we found is like our agent actually what we found is like our agent actually what we found is like our agent actually has access to a lot of information that has access to a lot of information that has access to a lot of information that people ask questions about. At least people ask questions about. At least people ask questions about. At least this is true internally. Um, so we this is true internally. Um, so we this is true internally. Um, so we actually have an agent that can watch actually have an agent that can watch actually have an agent that can watch all of these sort of critical channels, all of these sort of critical channels, all of these sort of critical channels, um, and determine whether it has enough um, and determine whether it has enough um, and determine whether it has enough sort of confidence to answer the sort of confidence to answer the sort of confidence to answer the question or not. Um, and one of the fun question or not. Um, and one of the fun question or not. Um, and one of the fun things is like it, uh, the our agents things is like it, uh, the our agents things is like it, uh, the our agents have access to like Slack DMs and things have access to like Slack DMs and things have access to like Slack DMs and things like that. Um, and so you can have an like that. Um, and so you can have an like that. Um, and so you can have an agent that basically will DM you to say, agent that basically will DM you to say, agent that basically will DM you to say, "I think I know the answer to this, but "I think I know the answer to this, but "I think I know the answer to this, but I'm not sure. Can you confirm this for I'm not sure. Can you confirm this for I'm not sure. Can you confirm this for me before I, you know, respond back?" me before I, you know, respond back?" me before I, you know, respond back?" Um, so this kind of emergent behaviors Um, so this kind of emergent behaviors Um, so this kind of emergent behaviors gets kind of fun and interesting as you gets kind of fun and interesting as you gets kind of fun and interesting as you just kind of build these things out. just kind of build these things out. just kind of build these things out. Okay, so I'm going to flip over and hope Okay, so I'm going to flip over and hope Okay, so I'm going to flip over and hope all of this works.

all of this works. all of this works. Um, Cool. Cool. Um Um Um So, uh So, uh So, uh let's see if I can find the one that I let's see if I can find the one that I let's see if I can find the one that I wanted to show. So, this is our uh sort wanted to show. So, this is our uh sort wanted to show. So, this is our uh sort of demo application running in our sort of demo application running in our sort of demo application running in our sort of demo sort of Slack environment. Um of demo sort of Slack environment. Um of demo sort of Slack environment. Um and uh what I wanted to show off was and uh what I wanted to show off was and uh what I wanted to show off was some of our deployment stuff and talk a some of our deployment stuff and talk a some of our deployment stuff and talk a little bit more about um what's kind of little bit more about um what's kind of little bit more about um what's kind of going on under the hood. Um so, this is, going on under the hood. Um so, this is, going on under the hood. Um so, this is, you know, sort of fake environment um you know, sort of fake environment um you know, sort of fake environment um just to kind of showcase some things. Um just to kind of showcase some things. Um just to kind of showcase some things. Um so, here um anytime somebody posts a so, here um anytime somebody posts a so, here um anytime somebody posts a sort of GitHub tag, um our agent's going sort of GitHub tag, um our agent's going sort of GitHub tag, um our agent's going to sort of see that and say, "Oh, I I to sort of see that and say, "Oh, I I to sort of see that and say, "Oh, I I should That's a release. Is that a should That's a release. Is that a should That's a release. Is that a release? Yes, that is a release. Um let release? Yes, that is a release. Um let release? Yes, that is a release. Um let me go watch that." Um but, it's not just me go watch that." Um but, it's not just me go watch that." Um but, it's not just going to watch it. Um it's going to do going to watch it. Um it's going to do going to watch it. Um it's going to do something that's slightly more something that's slightly more something that's slightly more intelligent. Uh cuz like I said before, intelligent. Uh cuz like I said before, intelligent. Uh cuz like I said before, everybody kind of has a CI/CD system. It everybody kind of has a CI/CD system. It everybody kind of has a CI/CD system. It will do the sort of standard checks on will do the sort of standard checks on will do the sort of standard checks on on, you know, certain KPIs. Um but, what on, you know, certain KPIs. Um but, what on, you know, certain KPIs. Um but, what the agent is able to do is actually look the agent is able to do is actually look the agent is able to do is actually look at the changes that are going in, at the changes that are going in, at the changes that are going in, understand what telemetry might help us understand what telemetry might help us understand what telemetry might help us evaluate whether those changes are, you evaluate whether those changes are, you evaluate whether those changes are, you know, good or not good or like are know, good or not good or like are know, good or not good or like are putting the system in an abnormal state, putting the system in an abnormal state, putting the system in an abnormal state, um and build a sort of customized plan um and build a sort of customized plan um and build a sort of customized plan that it's going to check uh just for that it's going to check uh just for that it's going to check uh just for this specific release. And this is why I this specific release. And this is why I this specific release. And this is why I kind of go back to like, you know, our kind of go back to like, you know, our kind of go back to like, you know, our goal is not to sit here and say, "We're goal is not to sit here and say, "We're goal is not to sit here and say, "We're going to replace an entire CI/CD going to replace an entire CI/CD going to replace an entire CI/CD pipeline. You've spent time organizing pipeline. You've spent time organizing pipeline. You've spent time organizing that." But, this can sort of patch a lot that." But, this can sort of patch a lot that." But, this can sort of patch a lot of, you know, uh parts of your system of, you know, uh parts of your system of, you know, uh parts of your system that may not be as robust as they should that may not be as robust as they should that may not be as robust as they should be. And it would be great if you had a be. And it would be great if you had a be. And it would be great if you had a single engineer just focused on like single engineer just focused on like single engineer just focused on like watching all the things on every watching all the things on every watching all the things on every release, but that's really expensive.

release, but that's really expensive. release, but that's really expensive. There's a lot of cognitive load. You'd There's a lot of cognitive load. You'd There's a lot of cognitive load. You'd rather have them doing other things. Um rather have them doing other things. Um rather have them doing other things. Um so, now the agent can come and actually so, now the agent can come and actually so, now the agent can come and actually do a lot of that sort of dynamic do a lot of that sort of dynamic do a lot of that sort of dynamic understanding of this is the change, um understanding of this is the change, um understanding of this is the change, um so, I'm going to sort of check for these so, I'm going to sort of check for these so, I'm going to sort of check for these things. And so, here, um the checkout things. And so, here, um the checkout things. And so, here, um the checkout replaces currency service, um replaces currency service, um replaces currency service, um you know, we're monitoring the checkout you know, we're monitoring the checkout you know, we're monitoring the checkout latency and the error rates. Well, let's latency and the error rates. Well, let's latency and the error rates. Well, let's take a look at the Kafka pipeline cuz take a look at the Kafka pipeline cuz take a look at the Kafka pipeline cuz that's sort of involved. This is the that's sort of involved. This is the that's sort of involved. This is the sort of causal chain I want to sort of sort of causal chain I want to sort of sort of causal chain I want to sort of say, I want to make sure is is healthy. say, I want to make sure is is healthy. say, I want to make sure is is healthy. And it'll check that, and it can check And it'll check that, and it can check And it'll check that, and it can check it not just once, um but sort of on an it not just once, um but sort of on an it not just once, um but sort of on an ongoing basis. And you know, none of ongoing basis. And you know, none of ongoing basis. And you know, none of this is hard-coded in. It's not like, this is hard-coded in. It's not like, this is hard-coded in. It's not like, oh, let's just wait for 15 minutes and oh, let's just wait for 15 minutes and oh, let's just wait for 15 minutes and then try this again, and then we'll be then try this again, and then we'll be then try this again, and then we'll be done. done. done. Again, the agent has a bit more Again, the agent has a bit more Again, the agent has a bit more autonomy, and and you get to guide it a autonomy, and and you get to guide it a autonomy, and and you get to guide it a bit on how you how much autonomy you bit on how you how much autonomy you bit on how you how much autonomy you want it to have, but it could decide, I want it to have, but it could decide, I want it to have, but it could decide, I want to wait for another hour cuz this want to wait for another hour cuz this want to wait for another hour cuz this type of issue might only hit every you type of issue might only hit every you type of issue might only hit every you know, every so often, so I really want know, every so often, so I really want know, every so often, so I really want to spend a little bit more time focused to spend a little bit more time focused to spend a little bit more time focused on this. Maybe I'll come back in 3 days on this. Maybe I'll come back in 3 days on this. Maybe I'll come back in 3 days and say, is this deploy still kind of and say, is this deploy still kind of and say, is this deploy still kind of healthy? Are we seeing the the change in healthy? Are we seeing the the change in healthy? Are we seeing the the change in the effect that I expected to see out of the effect that I expected to see out of the effect that I expected to see out of this? this? this? So this is a type of thing that we can So this is a type of thing that we can So this is a type of thing that we can bring, and and this again works for bring, and and this again works for bring, and and this again works for feature flags, infra changes, sort of feature flags, infra changes, sort of feature flags, infra changes, sort of any sort of eventing system that you can any sort of eventing system that you can any sort of eventing system that you can think of.

think of. think of. The sort of on-call handoff reports, The sort of on-call handoff reports, The sort of on-call handoff reports, let's scroll up just a little bit. let's scroll up just a little bit. let's scroll up just a little bit. Um Um Um So this is just a summarization of all So this is just a summarization of all So this is just a summarization of all of the work that was done, you know, of the work that was done, you know, of the work that was done, you know, over the last day over the last day over the last day that the agent is kind of just that the agent is kind of just that the agent is kind of just summarizing up. And this one's a little summarizing up. And this one's a little summarizing up. And this one's a little bit verbose, bit verbose, bit verbose, but but but you can see a bunch of different sort of you can see a bunch of different sort of you can see a bunch of different sort of investigation summaries that we did, investigation summaries that we did, investigation summaries that we did, some notable changes, etc. Work work some notable changes, etc. Work work some notable changes, etc. Work work completed. completed. completed. Um Um Um So there's a critical open. You know, I So there's a critical open. You know, I So there's a critical open. You know, I guess that's the guess that's the guess that's the on-call handoff. on-call handoff. on-call handoff. But one of the nice things But one of the nice things But one of the nice things I don't know if I'll be able to watch it I don't know if I'll be able to watch it I don't know if I'll be able to watch it go all the way. But go all the way. But go all the way. But you can always just come back in this you can always just come back in this you can always just come back in this thread and, you know, this is too thread and, you know, this is too thread and, you know, this is too verbose. verbose. verbose. Verbose. Verbose. Verbose. Make it shorter. Make it shorter. Make it shorter. Um and I'm not going to be able to Um and I'm not going to be able to Um and I'm not going to be able to unfortunately no time to watch this unfortunately no time to watch this unfortunately no time to watch this actually go, but this works. The the actually go, but this works. The the actually go, but this works. The the agent is able to update its task agent is able to update its task agent is able to update its task underneath, and able to sort of give you underneath, and able to sort of give you underneath, and able to sort of give you the answer like update so that the next the answer like update so that the next the answer like update so that the next time it fires, time it fires, time it fires, it's not going to uh it's not going to uh it's not going to uh be as verbose. And I can tell it be as verbose. And I can tell it be as verbose. And I can tell it explicitly what I want, etc. I was just explicitly what I want, etc. I was just explicitly what I want, etc. I was just kind of giving an example.

kind of giving an example. kind of giving an example. This is kind of the more fun one. This is kind of the more fun one. This is kind of the more fun one. So, you know, here I'm just like posting So, you know, here I'm just like posting So, you know, here I'm just like posting different problems. Um, I'm not having different problems. Um, I'm not having different problems. Um, I'm not having to know that resolve exists. Um, I don't to know that resolve exists. Um, I don't to know that resolve exists. Um, I don't have to like at mention resolve, have to like at mention resolve, have to like at mention resolve, whatever. Um, I've set up this agent to whatever. Um, I've set up this agent to whatever. Um, I've set up this agent to sort of passively watch this channel. sort of passively watch this channel. sort of passively watch this channel. Um, if you see something that you think Um, if you see something that you think Um, if you see something that you think you have an answer for that somebody's you have an answer for that somebody's you have an answer for that somebody's kind of, you know, digging into, um, go kind of, you know, digging into, um, go kind of, you know, digging into, um, go ahead and ahead and ahead and respond. Um, otherwise don't. So, you respond. Um, otherwise don't. So, you respond. Um, otherwise don't. So, you know, here's a message that I posted um, know, here's a message that I posted um, know, here's a message that I posted um, that it's decided I don't need to that it's decided I don't need to that it's decided I don't need to respond to this. Um, so again, very kind respond to this. Um, so again, very kind respond to this. Um, so again, very kind of flexible system um, that can kind of of flexible system um, that can kind of of flexible system um, that can kind of adapt to a bunch of different things. We have uh, in the UI there's a bunch of We have uh, in the UI there's a bunch of stuff that you can do. You know, you can stuff that you can do. You know, you can stuff that you can do. You know, you can always go down and inspect all the always go down and inspect all the always go down and inspect all the different tasks that you have and and different tasks that you have and and different tasks that you have and and view their reports, view previous runs. view their reports, view previous runs. view their reports, view previous runs. You can see all the work that the that You can see all the work that the that You can see all the work that the that the agent has done underneath um, to the agent has done underneath um, to the agent has done underneath um, to sort of accomplish that task. So, you sort of accomplish that task. So, you sort of accomplish that task. So, you get a lot of visibility into what the get a lot of visibility into what the get a lot of visibility into what the agent is doing, but we think the surface agent is doing, but we think the surface agent is doing, but we think the surface area being where you live, right? So, area being where you live, right? So, area being where you live, right? So, Slack is this like kind of um, or or MS Slack is this like kind of um, or or MS Slack is this like kind of um, or or MS Teams if you're on MS Teams, as this Teams if you're on MS Teams, as this Teams if you're on MS Teams, as this kind of first party experience to sort kind of first party experience to sort kind of first party experience to sort of integrate the the agent into um, is of integrate the the agent into um, is of integrate the the agent into um, is incredibly important. Um, and so, you incredibly important. Um, and so, you incredibly important. Um, and so, you know, how would you get this stuff uh, know, how would you get this stuff uh, know, how would you get this stuff uh, sort of set up? Um, sort of set up? Um, sort of set up? Um, it's really just through talking with it's really just through talking with it's really just through talking with the agent. Um, and so, here this is me the agent. Um, and so, here this is me the agent. Um, and so, here this is me sort of saying, "Hey, I want to do a new sort of saying, "Hey, I want to do a new sort of saying, "Hey, I want to do a new recurring health summary for my team."

recurring health summary for my team." recurring health summary for my team." Um, so the agent's going to uh, take a Um, so the agent's going to uh, take a Um, so the agent's going to uh, take a look at my environment, it's going to look at my environment, it's going to look at my environment, it's going to explore my environment a little bit um, explore my environment a little bit um, explore my environment a little bit um, and eventually likely come back and ask and eventually likely come back and ask and eventually likely come back and ask me a couple questions about um, what I me a couple questions about um, what I me a couple questions about um, what I want to see, what kind of reports do I want to see, what kind of reports do I want to see, what kind of reports do I want, how how verbose do I want it, um, want, how how verbose do I want it, um, want, how how verbose do I want it, um, etc. Uh, the agent's going to go ahead etc. Uh, the agent's going to go ahead etc. Uh, the agent's going to go ahead and do all that and it's going to set up and do all that and it's going to set up and do all that and it's going to set up that sort of initial thing for me so that sort of initial thing for me so that sort of initial thing for me so that I can test it out, make sure it's that I can test it out, make sure it's that I can test it out, make sure it's working, um, and then share it with the working, um, and then share it with the working, um, and then share it with the rest of uh, with the rest of my team. In the interest of time, I don't think In the interest of time, I don't think we'll get to this, but come by the booth we'll get to this, but come by the booth we'll get to this, but come by the booth and you can see more. and you can see more. and you can see more. Um, Um, Um, cool. Um, so that that's background agents. Um, so that that's background agents. And again, I I sort of lean back on be And again, I I sort of lean back on be And again, I I sort of lean back on be creative, right? Like everyone has creative, right? Like everyone has creative, right? Like everyone has unique work, I mean as as a company we unique work, I mean as as a company we unique work, I mean as as a company we believe that every, you know, every believe that every, you know, every believe that every, you know, every company is a unique place. That's why we company is a unique place. That's why we company is a unique place. That's why we spend so much time on our knowledge spend so much time on our knowledge spend so much time on our knowledge system, etc. Um, truly understanding system, etc. Um, truly understanding system, etc. Um, truly understanding your, you know, what your environment your, you know, what your environment your, you know, what your environment looks like, what your needs are, etc. looks like, what your needs are, etc. looks like, what your needs are, etc. Um, that begins to kind of tell you Um, that begins to kind of tell you Um, that begins to kind of tell you where the where the biggest benefit from where the where the biggest benefit from where the where the biggest benefit from having these agents begin to pick up having these agents begin to pick up having these agents begin to pick up work um, would be. Um, also just to call work um, would be. Um, also just to call work um, would be. Um, also just to call out, if you have an agent harness, if out, if you have an agent harness, if out, if you have an agent harness, if you, you know, internally if you're you, you know, internally if you're you, you know, internally if you're building your own, um, everything that I building your own, um, everything that I building your own, um, everything that I showed is accessible uh, through kind of showed is accessible uh, through kind of showed is accessible uh, through kind of MCP servers, etc. Um, so you can really MCP servers, etc. Um, so you can really MCP servers, etc. Um, so you can really graft resolve into kind of any um, graft resolve into kind of any um, graft resolve into kind of any um, system that you have um, as just kind of system that you have um, as just kind of system that you have um, as just kind of an extension of learning to kind of do an extension of learning to kind of do an extension of learning to kind of do deeper work or to sort of pull deeper work or to sort of pull deeper work or to sort of pull production context uh, a bit more production context uh, a bit more production context uh, a bit more efficiently or you know, even augmented efficiently or you know, even augmented efficiently or you know, even augmented with all that learning stuff that we've with all that learning stuff that we've with all that learning stuff that we've done. Um, and then obviously bring your done. Um, and then obviously bring your done. Um, and then obviously bring your own skills um, along for the ride. Um,

own skills um, along for the ride. Um, own skills um, along for the ride. Um, don't go duplicate a bunch of stuff. Um, don't go duplicate a bunch of stuff. Um, don't go duplicate a bunch of stuff. Um, so really the biggest things to take so really the biggest things to take so really the biggest things to take away, um, cost of operational work, it's away, um, cost of operational work, it's away, um, cost of operational work, it's not navigating uh, you know, it's it's not navigating uh, you know, it's it's not navigating uh, you know, it's it's it's not just in the task execution, it's not just in the task execution, it's not just in the task execution, it's in the environment complexity, it's in the environment complexity, it's in the environment complexity, right? Um, that's where the biggest right? Um, that's where the biggest right? Um, that's where the biggest issue is going to happen. Um, issue is going to happen. Um, issue is going to happen. Um, background agents, they run on background agents, they run on background agents, they run on schedules, they run on triggers, they're schedules, they run on triggers, they're schedules, they run on triggers, they're very composable, you can sort of graft very composable, you can sort of graft very composable, you can sort of graft them into lots of different use cases. them into lots of different use cases. them into lots of different use cases. Um, it's fun to see people explore that. Um, it's fun to see people explore that. Um, it's fun to see people explore that. Um, Um, Um, and uh, and uh, and uh, yeah. Um, you open resolve just to kind yeah. Um, you open resolve just to kind yeah. Um, you open resolve just to kind of see what the top findings are, etc. of see what the top findings are, etc. of see what the top findings are, etc. Um, but you know, ideally a lot of your Um, but you know, ideally a lot of your Um, but you know, ideally a lot of your interaction is kind of in the places interaction is kind of in the places interaction is kind of in the places that you're already kind of doing work. that you're already kind of doing work. that you're already kind of doing work. Um, so um, if you have any questions, Um, so um, if you have any questions, Um, so um, if you have any questions, uh, you can find me down at the booth or uh, you can find me down at the booth or uh, you can find me down at the booth or you know, just meet me out in the hall. you know, just meet me out in the hall. you know, just meet me out in the hall. Um, but thanks for coming. Appreciate Um, but thanks for coming. Appreciate Um, but thanks for coming. Appreciate it. it. it. >> [applause]