Build a Research Agent with Claude Managed Agents (Part 2 of 4) — Anthropic

Thank you for the presentation. Out of curiosity about the Out of curiosity about the session design, what session design, what session design, what changed in the first and changed in the first and changed in the first and second iterations? second iterations? Good question. Gabe, Good question. Gabe, do you want to ask do you want to ask do you want to ask this question about this question about this question about what changed in the what changed in the what changed in the session store between the session store between the session store between the first and second first and second first and second iterations? iterations? Of course. The main thing that Of course. The main thing that changed was that changed was that changed was that we wanted to we wanted to we wanted to abstract as abstract as abstract as many things as possible and many things as possible and many things as possible and separate them into different separate them into different separate them into different components so that we components so that we components so that we could extend the could extend the could extend the storage mechanisms storage mechanisms storage mechanisms for future events for future events for future events that we wanted to that we wanted to that we wanted to store without store without store without tying them so tying them so tying them so tightly to the tightly to the tightly to the session mechanics. So, we session mechanics. So, we session mechanics. So, we applied the applied the applied the same principle of same principle of same principle of building and building and building and virtualizing virtualizing virtualizing interfaces all the way down interfaces all the way down interfaces all the way down , not only across , not only across , not only across service boundaries, but also service boundaries, but also service boundaries, but also at the event level. at the event level. Okay, thank you. Okay, thank you. It seems like maybe It seems like maybe It seems like maybe not everyone will be able to not everyone will be able to not everyone will be able to follow along, but follow along, but follow along, but hopefully we hopefully we hopefully we can find a way to can find a way to can find a way to share these share these share these slides and you slides and you slides and you can try can try can try it at home. Or, if it at home. Or, if it at home. Or, if you have a phone, you have a phone, you have a phone, maybe you can maybe you can maybe you can try try try connecting it to connecting it to connecting it to your hotspot your hotspot your hotspot and see if and see if and see if you can you can you can track. Okay, track. Okay, track. Okay, so let's try again.

so let's try again. so let's try again. We'll say, "Create We'll say, "Create We'll say, "Create me an agent that me an agent that me an agent that will tell me will tell me will tell me jokes whenever a jokes whenever a jokes whenever a topic is given." And then topic is given." And then topic is given." And then you just press you just press you just press Enter. And you see that Enter. And you see that Enter. And you see that Claude issues a system Claude issues a system Claude issues a system request. He came up with a request. He came up with a request. He came up with a name, a description, and chose a name, a description, and chose a name, a description, and chose a model. You can model. You can model. You can modify this model modify this model modify this model if you want, and then if you want, and then if you want, and then give the agent access give the agent access give the agent access to the to the to the agent toolkit. agent toolkit. agent toolkit. And then you click And then you click And then you click "create agent." This "create agent." This "create agent." This should trigger the should trigger the V1 agent endpoint to create an V1 agent endpoint to create an agent. Okay, it's agent. Okay, it's agent. Okay, it's done. Next, we done. Next, we done. Next, we will configure will configure will configure the environment. Uh, which is a the environment. Uh, which is a the environment. Uh, which is a template for a template for a template for a sandbox. Er, we'll sandbox. Er, we'll sandbox. Er, we'll just create a new one. just create a new one. And then we'll just And then we'll just give it give it give it unrestricted access unrestricted access unrestricted access to do that, but you to do that, but you to do that, but you can obviously, can obviously, can obviously, when you create a when you create a when you create a production agent, you production agent, you production agent, you want to tighten want to tighten want to tighten access, control access, control access, control what domains it what domains it what domains it can access, or can access, or can access, or certain MCP servers.

certain MCP servers. certain MCP servers. So, we'll make it So, we'll make it So, we'll make it unlimited. And then unlimited. And then unlimited. And then here you can choose the here you can choose the here you can choose the packages you want to packages you want to packages you want to install, etc. And install, etc. And install, etc. And then we'll just then we'll just then we'll just start the session. start the session. Good. Now we Good. Now we will move on to will move on to will move on to test running test running test running our session. Emm. our session. Emm. our session. Emm. Tell me Tell me Tell me jokes about cats. jokes about cats. You will see that You will see that the model has started to think. the model has started to think. And she made a joke And she made a joke about the cat. Yeah, it was kind of like, it's about the cat. Yeah, it was kind of like, it's so easy for you so easy for you to create your own to create your own to create your own agent. This agent. This agent. This agent is now up and agent is now up and agent is now up and running in the cloud. All running in the cloud. All running in the cloud. All you need to you need to you need to do is start a do is start a do is start a session with the agent, session with the agent, session with the agent, send events, send events, send events, receive events receive events receive events back. Everything else is back. Everything else is back. Everything else is controlled by us. So, controlled by us. So, controlled by us. So, we manage all the we manage all the we manage all the infrastructure, infrastructure, infrastructure, hosting, hosting, hosting, hardware, hardware, sandbox management, sandbox management, scaling, scaling, scaling, storing your storing your storing your event log, and event log, and event log, and then you can then you can then you can focus on focus on focus on building your building your building your product around product around product around that. Um, thank you.

that. Um, thank you. that. Um, thank you. I'll now hand it over to I'll now hand it over to I'll now hand it over to Gabe to do the Gabe to do the Gabe to do the more hands-on more hands-on more hands-on part of the workshop, and part of the workshop, and part of the workshop, and hopefully we'll have some hopefully we'll have some hopefully we'll have some luck with the Wi-Fi. Thank you luck with the Wi-Fi. Thank you . Can . Can you hear me? Perfectly. you hear me? Perfectly. I see a bunch of I see a bunch of I see a bunch of people pressing in from behind. There's room here and here people pressing in from behind. There's room here and here people pressing in from behind. There's room here and here if you want to if you want to if you want to come in, so we can come in, so we can come in, so we can fit everyone in. Before fit everyone in. Before fit everyone in. Before we begin, here is we begin, here is we begin, here is the repository we the repository we the repository we will be will be will be using. So using. So using. So if you want to if you want to if you want to take a picture of the QR code, take a picture of the QR code, take a picture of the QR code, clone this clone this clone this repository, repository, repository, download it from the CD download it from the CD download it from the CD to the to the to the research research research center directory, and then center directory, and then center directory, and then run NPM install or run NPM install or run NPM install or do this. I'll do this. I'll do this. I'll let you do let you do let you do that, and then I'll tell that, and then I'll tell that, and then I'll tell you what we're you what we're you what we're going to create going to create going to create together today. I see some together today. I see some together today. I see some phones are still not phones are still not phones are still not working. I know working. I know working. I know you're having problems with your Wi-Fi. you're having problems with your Wi-Fi. you're having problems with your Wi-Fi. I hope you I hope you I hope you can get to the can get to the can get to the repository. OK.

repository. OK. repository. OK. Oh, I see you. OK. Oh, I see you. OK. Steeply. OK. So what are Steeply. OK. So what are we going to create we going to create we going to create today? I'll show today? I'll show today? I'll show you how you can you how you can you how you can use use use cloud-managed cloud-managed cloud-managed agents to agents to agents to build a build a build a variety of variety of variety of applications. We'll start applications. We'll start applications. We'll start by creating by creating by creating our own our own our own chat-like interface so you chat-like interface so you chat-like interface so you can communicate with an can communicate with an can communicate with an agent. We could say agent. We could say agent. We could say hello. Then we hello. Then we hello. Then we are going to extend are going to extend are going to extend this agent to a this agent to a this agent to a multi-agent multi-agent multi-agent ecosystem so that it ecosystem so that it ecosystem so that it can delegate can delegate can delegate tasks to sub-agents. tasks to sub-agents. tasks to sub-agents. We're going to We're going to We're going to give him give him give him skills, specifically skills, specifically skills, specifically the skill to be the skill to be the skill to be able to go to the able to go to the able to go to the SEC website and SEC website and SEC website and get get get SEC filings for individual SEC filings for individual SEC filings for individual tickers and tickers and tickers and analyze certain analyze certain analyze certain forms. We can then forms. We can then use these use these new improved new improved new improved agents to perform agents to perform agents to perform more asynchronous more asynchronous more asynchronous agent tasks agent tasks agent tasks that can be that can be that can be time-consuming. And then time-consuming. And then time-consuming. And then we're going to we're going to we're going to bring all of this bring all of this bring all of this together by enabling a together by enabling a map reduction interface where we can map reduction interface where we can talk to an talk to an talk to an agent, and that agent agent, and that agent agent, and that agent can then can then can then delegate tasks to delegate tasks to delegate tasks to many agents and many agents and many agents and do something like do something like do something like map reduction; map reduction; map reduction; showing you how you showing you how you showing you how you can build your can build your can build your own interface own interface own interface on top of CMA to on top of CMA to on top of CMA to orchestrate, orchestrate, orchestrate, you know, your own you know, your own you know, your own work with work with work with custom custom custom tools, tools, tools, manage manage manage multi-session, ahem, multi-session, ahem, multi-session, ahem, interactions. Uh, and interactions. Uh, and interactions. Uh, and then we'll wrap up, I'll then we'll wrap up, I'll then we'll wrap up, I'll show you, uh, our show you, uh, our show you, uh, our deployment API, which deployment API, which deployment API, which allows you allows you allows you to set up a to set up a to set up a recurring job recurring job recurring job on a schedule so you on a schedule so you on a schedule so you can run, uh, can run, uh, can run, uh, sessions, uh, at a given sessions, uh, at a given sessions, uh, at a given frequency to frequency to frequency to do do do long-term long-term long-term analysis that's

analysis that's analysis that's constantly updated constantly updated . Does that sound good? Quietly. . Does that sound good? Quietly. . Does that sound good? Quietly. Okay, great. Okay, great. Okay, so let's Okay, so let's move on to the move on to the move on to the coding interface coding interface . I will use . I will use . I will use Cloud Code for this. You Cloud Code for this. You Cloud Code for this. You can can can use use any encoding agent any encoding agent you you you like. I don't think like. I don't think anyone writes code anyone writes code anyone writes code by hand, so I wo by hand, so I wo by hand, so I wo n't do it either. n't do it either. We have uploaded We have uploaded a skill to your repository a skill to your repository a skill to your repository that will that will that will guide you through the guide you through the guide you through the workshop step by step. So if you workshop step by step. So if you workshop step by step. So if you want to repeat want to repeat want to repeat the workshop after the workshop after the workshop after I finish, later I finish, later I finish, later when you have when you have when you have better Wi-Fi and better Wi-Fi and better Wi-Fi and internet access, you internet access, you internet access, you can do it can do it can do it at home. So the first thing at home. So the first thing at home. So the first thing you need to do you need to do you need to do is, I assume you've is, I assume you've is, I assume you've checked out checked out checked out the repository, logged the repository, logged the repository, logged into the into the into the research research research task repository, started task repository, started task repository, started bun_1, we'll start our bun_1, we'll start our bun_1, we'll start our server using server using server using bun_1. I will make it bun_1. I will make it bun_1. I will make it bigger. I assume bigger. I assume bigger. I assume no one sees. no one sees. no one sees. Okay, is that better? Yes Okay, is that better? Yes . . . And then, if you have any And then, if you have any And then, if you have any questions questions questions during during during Gabe Gabe's presentation, Gabe Gabe's presentation, Gabe Gabe's presentation, feel free to raise your feel free to raise your feel free to raise your hand and I can hand and I can hand and I can run to you with the run to you with the run to you with the microphone.

microphone. microphone. We have helpers, We have helpers, We have helpers, so if you have any so if you have any so if you have any problems, problems, problems, raise your hand. People raise your hand. People raise your hand. People will come and will come and will come and help you. Steeply. help you. Steeply. help you. Steeply. So, if you've made it So, if you've made it So, if you've made it this far, you this far, you this far, you should be able to download should be able to download should be able to download this website. Uh, and the this website. Uh, and the this website. Uh, and the first thing you first thing you first thing you need to do is need to do is need to do is go to the go to the go to the settings page. Uh, is everyone settings page. Uh, is everyone settings page. Uh, is everyone with me yet? Yes? with me yet? Yes? with me yet? Yes? Good. Uh, now we're Good. Uh, now we're Good. Uh, now we're going to...Uh, going to...Uh, going to...Uh, okay, as soon as he okay, as soon as he okay, as soon as he takes the picture, takes the picture, takes the picture, we're...we're going to we're...we're going to we're...we're going to start our start our start our master class. So, in master class. So, in master class. So, in Cloud Code, you can Cloud Code, you can Cloud Code, you can type "start a type "start a type "start a masterclass." Uh, masterclass." Uh, masterclass." Uh, he'll start doing he'll start doing he'll start doing your master class. We will your master class. We will your master class. We will do this in six do this in six do this in six acts. Sorry, acts. Sorry, acts. Sorry, there is no intermission between them there is no intermission between them there is no intermission between them . I hope . I hope . I hope you all agree. But we're you all agree. But we're you all agree. But we're going to walk going to walk going to walk through it. Claude... through it. Claude... Of course. We Of course. We Of course. We will switch to will switch to will switch to automatic mode. automatic mode. automatic mode. Claude Claude Claude Claude Claude Claude will track every will track every will track every action so you can action so you can action so you can continue. You continue. You continue. You can keep can keep can keep typing "workshop next" typing "workshop next" typing "workshop next" to have it, you know, to have it, you know, to have it, you know, keep keep keep moving through this moving through this moving through this thing. I'll ask thing. I'll ask thing. I'll ask Claude to do it for Claude to do it for Claude to do it for me, but hold my me, but hold my me, but hold my hand and teach me hand and teach me hand and teach me what he does. And so the what he does. And so the what he does. And so the first thing he does first thing he does first thing he does is make sure that is make sure that is make sure that our preconditions our preconditions our preconditions are set. It will are set. It will are set. It will copy the .env.example file to copy the .env.example file to copy the .env.example file to .env.local. Emm. Oh no, .env.local. Emm. Oh no, .env.local. Emm. Oh no, stop. We're not stop. We're not stop. We're not there yet. Good. He waits a little

there yet. Good. He waits a little there yet. Good. He waits a little impatiently. impatiently. impatiently. Okay, it will copy Okay, it will copy Okay, it will copy this file, this file, this file, there are some there are some there are some values ​​in this file for you. You will values ​​in this file for you. You will values ​​in this file for you. You will need to need to need to insert your API key. insert your API key. insert your API key. You will need to You will need to You will need to insert "SEC wants insert "SEC wants insert "SEC wants to know who you are when you to know who you are when you to know who you are when you call their API". call their API". call their API". So you will need to So you will need to So you will need to enter your name and enter your name and enter your name and email. This is just so email. This is just so they know who is they know who is calling them. And that, in calling them. And that, in calling them. And that, in principle, is all. Once principle, is all. Once principle, is all. Once you've done all that you've done all that you've done all that ...Well, we'll ...Well, we'll ...Well, we'll move on to the move on to the move on to the next one. So, is everyone next one. So, is everyone next one. So, is everyone with me? You entered your with me? You entered your with me? You entered your API key in your .env.local. API key in your .env.local. API key in your .env.local. You have entered your You have entered your You have entered your Edgar ID. Edgar ID. Edgar ID. Raise your hands. Raise your hands. Raise your hands. Please quickly Please quickly Please quickly raise your hands or raise your hands or raise your hands or tell me how many of tell me how many of tell me how many of you were able to you were able to you were able to follow along, who follow along, who follow along, who might have Wi-Fi or... might have Wi-Fi or... might have Wi-Fi or... Okay, we have Okay, we have Okay, we have at least one at least one at least one person. So, MVP. person. So, MVP. person. So, MVP. Sorry? Oh, oh, oh, Sorry? Oh, oh, oh, Sorry? Oh, oh, oh, about the repository. This one about the repository. This one ? ? Uh, Rowan, do you want to Uh, Rowan, do you want to hand them the microphone?

hand them the microphone? hand them the microphone? Yes, thank you. Yes, thank you. Yes, thank you. Yes, a slide with an Yes, a slide with an Yes, a slide with an ID. That's ID. That's ID. That's all. Settings. all. Settings. all. Settings. This? This? This? Thank you. Thank you. Thank you. Okay, of course. Okay, of course. Okay, of course. And then, and then, And then, and then, And then, and then, given that given that given that people don't have Wi-Fi, people don't have Wi-Fi, people don't have Wi-Fi, please stop us and please stop us and please stop us and ask questions ask questions ask questions so we can at least so we can at least so we can at least make it more make it more make it more interactive and interactive and interactive and help you help you help you understand the concepts understand the concepts understand the concepts so you can build so you can build so you can build on your own time after the on your own time after the on your own time after the workshop. workshop. Good. Hmm, yes, I Good. Hmm, yes, I have... Yes, it's an application have... Yes, it's an application have... Yes, it's an application that runs that runs that runs when you clone it when you clone it when you clone it and download it to a CD and download it to a CD and download it to a CD at the Research Desk and at the Research Desk and at the Research Desk and run... here's the run... here's the website you website you website you get... get... get... Uh, can I ask a Uh, can I ask a Uh, can I ask a question? question? question? Yeah, I can't see you, but Yeah, I can't see you, but ... ... ... Oh, I'm here...Uh, I'm Oh, I'm here...Uh, I'm Oh, I'm here...Uh, I'm curious, you know, when curious, you know, when curious, you know, when you talk about cloud you talk about cloud you talk about cloud agent management, agent management, agent management, where does that work? Does where does that work? Does where does that work? Does this work on AWS? this work on AWS? this work on AWS? Yes, that's a good Yes, that's a good Yes, that's a good question. Uh, we're question. Uh, we're question. Uh, we're going to run going to run going to run this on our own this on our own this on our own infrastructure. infrastructure. infrastructure. Today we have something like a Today we have something like a single P offering that we're single P offering that we're working on, but we working on, but we working on, but we also have a cloud also have a cloud also have a cloud platform on AWS if platform on AWS if platform on AWS if you want to work you want to work you want to work on AWS using on AWS using on AWS using your AWS credits. What we your AWS credits. What we your AWS credits. What we also offer, and also offer, and also offer, and maybe we can maybe we can maybe we can talk about this talk about this talk about this after this after this after this demo, is demo, is demo, is self-hosted self-hosted self-hosted sandboxes. So, if sandboxes. So, if sandboxes. So, if you want to create you want to create you want to create your own sandbox your own sandbox your own sandbox through various through various through various providers like providers like providers like Vercel, Modal, Daytona or a bunch of Vercel, Modal, Daytona or a bunch of Vercel, Modal, Daytona or a bunch of others, you can others, you can others, you can do that. Uh, even, do that. Uh, even, do that. Uh, even, you know, through Google Cloud you know, through Google Cloud you know, through Google Cloud and AWS. If you want and AWS. If you want and AWS. If you want to run your to run your to run your sandbox in this, uh sandbox in this, uh , environment, you , environment, you , environment, you can. Um, okay, I'll can. Um, okay, I'll can. Um, okay, I'll continue continue continue the demonstration. Uh, the demonstration. Uh, the demonstration. Uh, the first thing you

the first thing you the first thing you need to do is need to do is need to do is click this button, click this button, click this button, which will create, uh, which will create, uh, which will create, uh, your agent. So, your agent. So, two things will be created for us. Uh, two things will be created for us. Uh, the environment that the environment that the environment that Priyanka was talking about is a Priyanka was talking about is a Priyanka was talking about is a template of what a template of what a sandbox would look like. Uh, for sandbox would look like. Uh, for this demonstration, we're going to this demonstration, we're going to this demonstration, we're going to give her access to give her access to give her access to a few websites that a few websites that a few websites that she can she can she can go to, go to, go to, mainly the SEC. We'll add, mainly the SEC. We'll add, mainly the SEC. We'll add, uh, some Python packages uh, some Python packages uh, some Python packages so that it can so that it can so that it can address, uh, SEC, address, uh, SEC, address, uh, SEC, using, you know, using, you know, using, you know, some predefined some predefined Python packages. It will also Python packages. It will also create an agent. So create an agent. So , she's going to create this , she's going to create this , she's going to create this very simple agent. very simple agent. very simple agent. He has no He has no He has no tools now. He tools now. He tools now. He only has a clue. Well, only has a clue. Well, only has a clue. Well, he has the basic he has the basic he has the basic tools, but tools, but tools, but no skills, no no skills, no no skills, no coordination of coordination of coordination of subagents, nothing subagents, nothing subagents, nothing like that. I'm going to like that. I'm going to like that. I'm going to tell Claude tell Claude tell Claude to go ahead and to go ahead and to go ahead and implement an implement an implement an interface that interface that interface that we we we can use to communicate can use to communicate can use to communicate with this agent in our with this agent in our with this agent in our program. As program. As program. As Bianca mentioned, the Bianca mentioned, the Bianca mentioned, the main way to main way to main way to communicate with a communicate with a communicate with a session is through the flow of session is through the flow of session is through the flow of events. So, you events. So, you events. So, you listen to this listen to this listen to this stream for any stream for any stream for any events that the agent events that the agent events that the agent sends back to you, and sends back to you, and sends back to you, and you can you can you can use this use this use this endpoint to endpoint to endpoint to send events send events send events back. So, back. So, back. So, really everything that really everything that really everything that happens in happens in happens in communication for these communication for these chat-like applications happens in chat-like applications happens in this thread. So what we this thread. So what we this thread. So what we see here is what see here is what see here is what Claude is doing is implementing a Claude is doing is implementing a Claude is doing is implementing a section so that when I section so that when I section so that when I press Enter in my press Enter in my press Enter in my chat window, it chat window, it chat window, it sends a sends a sends a user.message event with the content user.message event with the content user.message event with the content that I'm sending to that I'm sending to that I'm sending to that session. And then

that session. And then that session. And then it would route the it would route the it would route the SSC stream back to my browser SSC stream back to my browser SSC stream back to my browser from my from my from my backend. So, it backend. So, it backend. So, it listened for events listened for events listened for events coming in through a coming in through a coming in through a streaming connection to streaming connection to streaming connection to our API, and then our API, and then our API, and then retransmitted retransmitted retransmitted them directly them directly them directly back to the frontend back to the frontend back to the frontend so I could so I could so I could display them in a display them in a display them in a nice way. And nice way. And nice way. And once Claude once Claude once Claude does that, we can does that, we can does that, we can go to this go to this go to this table tab and just table tab and just table tab and just say hello to our say hello to our say hello to our agent. And if everything agent. And if everything agent. And if everything worked, worked, hopefully, great. hopefully, great. So, we see that So, we see that So, we see that we are talking we are talking we are talking to our agent right now. If to our agent right now. If to our agent right now. If you wanted to create a you wanted to create a chat-like application, this is where you would chat-like application, this is where you would stop. Which is great stop. Which is great , but we want to , but we want to , but we want to do more. The whole do more. The whole do more. The whole point of point of point of Claude-driven agents Claude-driven agents Claude-driven agents is that you is that you is that you can execute can execute can execute long-running asynchronous long-running asynchronous long-running asynchronous tasks, ahem, tasks, ahem, tasks, ahem, for just a few for just a few for just a few minutes, but also hours, minutes, but also hours, minutes, but also hours, days, weeks, and even days, weeks, and even days, weeks, and even months. So, months. So, months. So, let's see let's see let's see what we need to what we need to what we need to do next, shall we? The do next, shall we? The do next, shall we? The next thing we'll next thing we'll next thing we'll want to do is, want to do is, want to do is, um, oh, Claudia is um, oh, Claudia is um, oh, Claudia is really really really impatient today. Good. The impatient today. Good. The impatient today. Good. The next thing we next thing we next thing we want to do is want to do is want to do is expand our expand our expand our agent, right? We want to agent, right? We want to agent, right? We want to give our agent give our agent give our agent the skill so we can the skill so we can the skill so we can call the SEC. We call the SEC. We call the SEC. We want to want to want to use use use multi-agent multi-agent multi-agent coordination so that, um, coordination so that, um, coordination so that, um, our session can our session can our session can delegate work to delegate work to delegate work to sub-agents that are sub-agents that are sub-agents that are more more more specialized and specialized and specialized and capable of performing capable of performing capable of performing some tasks, um, and some tasks, um, and some tasks, um, and we want to essentially we want to essentially we want to essentially climb the ladder of climb the ladder of climb the ladder of intelligence. So, intelligence. So, intelligence. So, let's do this, let's do this, let's do this, um, saying, um, um, saying, um, um, saying, um, seminar next. Um, seminar next. Um, seminar next. Um, now Claudia now Claudia now Claudia will implement a

will implement a will implement a provisioning scheme to provisioning scheme to provisioning scheme to upgrade our upgrade our upgrade our agent to agent to agent to use, uh, use, uh, use, uh, more subagents more subagents more subagents or any or any or any subagents. Um, we're, subagents. Um, we're, uh, going to give him a uh, going to give him a uh, going to give him a skill, a skill that skill, a skill that skill, a skill that will allow him to, uh, will allow him to, uh, will allow him to, uh, communicate with the SEC. We communicate with the SEC. We communicate with the SEC. We have described how to have described how to have described how to use the use the use the Python package. We described Python package. We described Python package. We described that he should that he should that he should use use use Edgar's ID Edgar's ID Edgar's ID to call. Um, we, uh to call. Um, we, uh , described all of that. And , described all of that. And , described all of that. And we're going to give we're going to give we're going to give him a special him a special him a special tool for a tool for a tool for a follow-up workshop. follow-up workshop. follow-up workshop. Um, this special Um, this special Um, this special tool is something that tool is something that tool is something that we can we can we can use to use to use to tell our tell our tell our principal principal principal investigator agent to create investigator agent to create investigator agent to create multiple sessions that multiple sessions that multiple sessions that we'll we'll we'll coordinate, uh, for a kind of coordinate, uh, for a kind of specialized specialized pooling. So, here's what pooling. So, here's what pooling. So, here's what Claudia is doing here Claudia is doing here . Um, she's updating . Um, she's updating . Um, she's updating her progress, creating her progress, creating her progress, creating this special this special this special tool called tool called dispatch analysis. We'll dispatch analysis. We'll come back to that come back to that come back to that in a second. Um, we in a second. Um, we in a second. Um, we give him the diagram. give him the diagram. Um, she's updating the Um, she's updating the master master master agent's description to agent's description to agent's description to create a new version of create a new version of create a new version of that agent. Um, this that agent. Um, this that agent. Um, this new version will have, new version will have, new version will have, um, oh, PagerDuty. Um, uh, this um, oh, PagerDuty. Um, uh, this um, oh, PagerDuty. Um, uh, this new version will have, uh, new version will have, uh, new version will have, uh, a few different a few different a few different subagents that it subagents that it subagents that it can delegate can delegate can delegate work to, as well as work to, as well as work to, as well as references to itself.

references to itself. references to itself. So if the main So if the main So if the main agent ever agent ever agent ever wants to create wants to create wants to create a copy of itself with the a copy of itself with the a copy of itself with the same request, it same request, it same request, it can do so. Um, can do so. Um, can do so. Um, this is important because, uh, this is important because, uh, this is important because, uh, sometimes you don't sometimes you don't sometimes you don't necessarily know necessarily know necessarily know what tasks what tasks what tasks your session is going to want to do, your session is going to want to do, your session is going to want to do, but you have a general but you have a general but you have a general idea that you idea that you idea that you want to have want to have want to have subagents that, you subagents that, you subagents that, you know, run know, run know, run and do the work. and do the work. and do the work. Um, you can Um, you can Um, you can define a reference define a reference define a reference to yourself, for which Cloud to yourself, for which Cloud to yourself, for which Cloud can then create a can then create a can then create a subagent. Um, it's subagent. Um, it's subagent. Um, it's important, uh, this issue important, uh, this issue important, uh, this issue has been raised has been raised has been raised before. It does not copy before. It does not copy before. It does not copy the transcript of this the transcript of this the transcript of this session to the subagent. session to the subagent. session to the subagent. It just copies the It just copies the It just copies the tool description and tool description and tool description and the schema, uh, and the query, uh, into the schema, uh, and the query, uh, into the schema, uh, and the query, uh, into this subagent. Um, this subagent. Um, this subagent. Um, cool. So now, cool. So now, cool. So now, if we go in here, if we go in here, if we go in here, we should see we should see we should see that there is a new agent. Oh, I that there is a new agent. Oh, I that there is a new agent. Oh, I think I need to get think I need to get think I need to get them ready. So, them ready. So, them ready. So, in our in our in our settings there should settings there should settings there should be a button that says be a button that says be a button that says "fill table". When "fill table". When "fill table". When I do that, it will I do that, it will I do that, it will actually start actually start . Oh. Um, okay. I've . Oh. Um, okay. I've . Oh. Um, okay. I've run this before.

run this before. run this before. Let me Let me Let me delete the skill so delete the skill so delete the skill so I can recreate it I can recreate it I can recreate it for you. Um, I for you. Um, I for you. Um, I tried to do a tried to do a tried to do a magic cooking magic cooking magic cooking show, but that's okay. show, but that's okay. Um, okay, then we'll Um, okay, then we'll delete the skill. We delete the skill. We delete the skill. We will ask Cloud will ask Cloud to reconfigure it. Um, it to reconfigure it. Um, it will create all these will create all these will create all these resources, and then we resources, and then we resources, and then we can see can see can see all the subagents here, all the subagents here, all the subagents here, the skill that we the skill that we the skill that we created, um, the created, um, the created, um, the memory store. So, for these memory store. So, for these memory store. So, for these sessions, we also sessions, we also sessions, we also support support support long-term long-term long-term memory storage that can be memory storage that can be shared between sessions. shared between sessions. Um, and we're going to Um, and we're going to Um, and we're going to use this so that use this so that use this so that different sessions can different sessions can different sessions can learn different learn different learn different financial financial financial instruments, and they instruments, and they instruments, and they can share that can share that can share that knowledge, um, with each knowledge, um, with each knowledge, um, with each other. And at the end of this other. And at the end of this other. And at the end of this seminar, I'll show you seminar, I'll show you how you can how you can how you can use this use this use this long-term memory long-term memory long-term memory to continue to continue to continue to improve your to improve your to improve your agents in the agents in the agents in the long long long term over term over term over many, many many, many many, many sessions, days, months, sessions, days, months, sessions, days, months, years. Um, cool. So, years. Um, cool. So, years. Um, cool. So, we should see that we should see that we should see that the agent who is here now the agent who is here now the agent who is here now has the skill that we has the skill that we has the skill that we described. It has described. It has described. It has subagents with their own subagents with their own subagents with their own prompts, prompts, prompts, tools, and tools, and tools, and permissions. He has a permissions. He has a permissions. He has a clue and a model. And clue and a model. And clue and a model. And I think we're ready to I think we're ready to I think we're ready to use this use this use this new dedicated new dedicated new dedicated subagent to subagent to subagent to run our run our run our first asynchronous first asynchronous first asynchronous long-running long-running long-running task. So let's task. So let's task. So let's tell Claude to tell Claude to tell Claude to move on to the move on to the move on to the next one. The next thing next one. The next thing next one. The next thing we want to do we want to do we want to do is say to Claude, not in a is say to Claude, not in a is say to Claude, not in a chat-like chat-like chat-like application, but more application, but more application, but more asynchronously: "Hey, asynchronously: "Hey, asynchronously: "Hey, research this research this research this financial financial financial instrument and give instrument and give instrument and give me a report in a format

me a report in a format me a report in a format that I define and that I'm that I define and that I'm that I define and that I'm interested in." And we interested in." And we interested in." And we will will will use what are use what are use what are called called called results. results. results. Results are how Results are how Results are how we created we created we created self-control. So in self-control. So in self-control. So in the end you say to the end you say to the end you say to Claude, “This is your Claude, “This is your Claude, “This is your assignment. This is your assignment. This is your assignment. This is your rubric for grading rubric for grading rubric for grading your assignment." He your assignment." He your assignment." He will do this job. And will do this job. And will do this job. And then, when he then, when he then, when he considers considers considers the task complete, the task complete, the task complete, he will ask he will ask he will ask the subagent the subagent the subagent to review the rubric to review the rubric to review the rubric and the work done by Claude and the work done by Claude and the work done by Claude and evaluate: " and evaluate: " Did he do it Did he do it Did he do it correctly and did correctly and did correctly and did he do it to the he do it to the he do it to the end?" If so, end?" If so, end?" If so, great. And if not, great. And if not, great. And if not, he'll try again. he'll try again. he'll try again. So Claude is So Claude is So Claude is essentially implementing this user-defined result event for us essentially implementing this user-defined result event for us , so that , so that whenever we whenever we whenever we start a session in start a session in start a session in our UI our UI our UI , which I'm going to , which I'm going to , which I'm going to show you for a show you for a show you for a particular ticker, it particular ticker, it particular ticker, it will send this will send this user-defined user-defined result with a specific result with a specific result with a specific rubric, a rubric, a rubric, a maximum maximum maximum number of iterations, number of iterations, number of iterations, and a description of the current and a description of the current and a description of the current task. And then Claude task. And then Claude task. And then Claude will complete this task will complete this task will complete this task for us. And how does it for us. And how does it for us. And how does it look in the look in the user interface? If you user interface? If you look at this look at this look at this scorecard, I scorecard, I scorecard, I can say to Claude, " can say to Claude, " Hey, I want to Hey, I want to Hey, I want to analyze Nvidia."

analyze Nvidia." analyze Nvidia." A. Please. Okay, A. Please. Okay, A. Please. Okay, great. So, he great. So, he great. So, he will start an will start an will start an Nvidia analysis session. This will take Nvidia analysis session. This will take Nvidia analysis session. This will take some time. I'm some time. I'm some time. I'm going to do a going to do a going to do a cooking show and cooking show and cooking show and show you a show you a show you a preview of what preview of what preview of what happened. Uh, happened. Uh, happened. Uh, it'll take about 10 it'll take about 10 it'll take about 10 minutes. But here's something minutes. But here's something minutes. But here's something I did yesterday. I did yesterday. I did yesterday. The first thing we will tell The first thing we will tell The first thing we will tell Claude is the result. I Claude is the result. I Claude is the result. I want you to want you to want you to analyze analyze Nvidia's latest quarterly reports to the SEC. You will Nvidia's latest quarterly reports to the SEC. You will use use use Edgar's identity. Edgar's identity. Edgar's identity. This is where This is where This is where your your your Edgar identities are. You have a Edgar identities are. You have a Edgar identities are. You have a memory, so check memory, so check memory, so check if you've done if you've done if you've done the analysis before and the analysis before and the analysis before and make sure you're make sure you're make sure you're using it using it using it instead of instead of instead of regurgitating and regurgitating and regurgitating and researching what researching what researching what you already know. Uh, I you already know. Uh, I you already know. Uh, I want you to find want you to find want you to find the newest form that the newest form that the newest form that interests me and interests me and interests me and do an analysis on it do an analysis on it . Uh, and then I . Uh, and then I . Uh, and then I need two need two need two results. I want results. I want results. I want you to write a you to write a you to write a note in my note in my note in my memory, uh, for this memory, uh, for this memory, uh, for this particular company.

particular company. particular company. And I want you to And I want you to And I want you to write a write a write a scorecard in a well- scorecard in a well- scorecard in a well- defined defined defined structured structured structured format, and I want format, and I want format, and I want you to save it in a you to save it in a you to save it in a specific location in the file. specific location in the file. specific location in the file. Uh, then Claude went Uh, then Claude went Uh, then Claude went to work, started to work, started to work, started calling for calling for calling for tools. He didn't tools. He didn't tools. He didn't find any find any find any notes in his notes in his notes in his memory storage. Um, memory storage. Um, memory storage. Um, it, uh, looked in it, uh, looked in it, uh, looked in a few different a few different a few different places, and then places, and then places, and then it decided, “Okay, I it decided, “Okay, I it decided, “Okay, I know what to do. I'm know what to do. I'm know what to do. I'm going to use going to use going to use my subagents to my subagents to my subagents to do the do the do the bulk of this work." bulk of this work." bulk of this work." Um, I'm going to Um, I'm going to Um, I'm going to run a run a run a financial, uh, financial, uh, financial, uh, expert expert expert analyst, agent, uh analyst, agent, uh , or extractor, and , or extractor, and , or extractor, and then a then a then a risk assessor analysis. And risk assessor analysis. And risk assessor analysis. And you can see you can see you can see all of this in the, uh, console all of this in the, uh, console all of this in the, uh, console here for complete here for complete here for complete observation. observation. observation. Each of these lines Each of these lines Each of these lines here represents a here represents a here represents a thread, and a thread thread, and a thread thread, and a thread represents work represents work represents work performed by a subagent. performed by a subagent. performed by a subagent. And so, each of the And so, each of the And so, each of the subagents got down to subagents got down to subagents got down to work. And then when work. And then when work. And then when they were done, they they were done, they they were done, they told the lead told the lead told the lead agent, "Hey, this is what agent, "Hey, this is what agent, "Hey, this is what you asked me to do." Uh, you asked me to do." Uh, you asked me to do." Uh, and then the main and then the main and then the main agent woke up, uh, agent woke up, uh, agent woke up, uh, got this got this got this information from information from information from the subagents, and then, uh the subagents, and then, uh , submitted whatever it , submitted whatever it , submitted whatever it needed to needed to needed to do and created the do and created the do and created the result. He result. He result. He created a JSON file that, created a JSON file that, created a JSON file that, you know, we specified, uh you know, we specified, uh , and he made sure to , and he made sure to , and he made sure to refresh the memory. And refresh the memory. And refresh the memory. And then then then the self-control began. So, the self-control began. So, the self-control began. So, he ran he ran he ran the subagent to the subagent to the subagent to check if check if the result we defined was being achieved. Um, he the result we defined was being achieved. Um, he looked at the column.

looked at the column. looked at the column. He analyzed He analyzed He analyzed all the work done all the work done all the work done during the session and during the session and during the session and analyzed analyzed the artifacts he created. And ultimately he the artifacts he created. And ultimately he decided that he was decided that he was decided that he was satisfied that this, satisfied that this, satisfied that this, you know, you know, you know, scorecard was scorecard was scorecard was delivered, that the numbers were delivered, that the numbers were delivered, that the numbers were clearly defined, that the clearly defined, that the clearly defined, that the memory refresh memory refresh memory refresh was correct, and was correct, and was correct, and everything looked good. everything looked good. everything looked good. Um, I don't think it's Um, I don't think it's Um, I don't think it's working yet, but once working yet, but once working yet, but once it's it's it's done, you'll done, you'll done, you'll see a table here of see a table here of see a table here of each of these tickers each of these tickers , because we can , because we can , because we can pull the report, the pull the report, the pull the report, the scorecard from the session, scorecard from the session, scorecard from the session, and load it. You and load it. You and load it. You put it in the put it in the put it in the database. For this database. For this database. For this demonstration, we're demonstration, we're demonstration, we're hosting it hosting it hosting it locally, but I think locally, but I think locally, but I think you get the idea. Um, you get the idea. Um, you get the idea. Um, okay, that's great, but what okay, that's great, but what okay, that's great, but what if you if you if you want to do 100 or 10 want to do 100 or 10 want to do 100 or 10 tickers at once? Wouldn't it be tickers at once? Wouldn't it be tickers at once? Wouldn't it be great great great if I could if I could if I could talk to an talk to an talk to an agent to create agent to create agent to create my own system that would my own system that would my own system that would delegate the work, uh delegate the work, uh , for each of the , for each of the , for each of the tickers into a separate tickers into a separate tickers into a separate session? And then, when session? And then, when session? And then, when everything is ready, everything is ready, everything is ready, compile the results compile the results compile the results and give me, uh, and give me, uh, and give me, uh, a bunch of points. Hmm, here's what a bunch of points. Hmm, here's what a bunch of points. Hmm, here's what we'll do next. Um, we'll do next. Um, we'll do next. Um, we're going to we're going to we're going to use this use this use this tool, our own tool, our own tool, our own tool that we tool that we tool that we built, to built, to built, to create our create our create our own own own orchestration system on orchestration system on orchestration system on our server, uh, our server, uh, our server, uh, where when we where when we where when we talk to our talk to our talk to our master agent, at master agent, at master agent, at some point this some point this some point this master agent master agent master agent will decide, "Hey, I want will decide, "Hey, I want will decide, "Hey, I want to run, uh, 10 to run, uh, 10 to run, uh, 10 different sessions to different sessions to different sessions to do work." Hmm do work." Hmm , we'll do it. Oh, I , we'll do it. Oh, I , we'll do it. Oh, I missed a step.

missed a step. missed a step. Sorry. Hmm, because we're Sorry. Hmm, because we're Sorry. Hmm, because we're going to upgrade going to upgrade going to upgrade the skill, uh, first the skill, uh, first . Hmm, so I wanted . Hmm, so I wanted . Hmm, so I wanted to show you that these to show you that these to show you that these skills are active skills are active skills are active artifacts that you artifacts that you artifacts that you can upgrade. Hmm can upgrade. Hmm , so you can , so you can , so you can improve the skill, uh improve the skill, uh , in this case, uh , in this case, uh , we're adding , we're adding , we're adding the ability the ability the ability to view the new to view the new to view the new form. Uh, and then form. Uh, and then form. Uh, and then when you do that, when you do that, when you do that, any new sessions any new sessions any new sessions that reference that that reference that that reference that skill will get that skill will get that skill will get that new information. Uh, new information. Uh, new information. Uh, this is something you can this is something you can this is something you can keep updating. keep updating. keep updating. You can You can You can view view view previous sessions to previous sessions to previous sessions to see how they see how they see how they worked. We'll worked. We'll worked. We'll see what, uh, at see what, uh, at see what, uh, at the end of this the end of this the end of this demonstration, this demonstration, this demonstration, this happens with what happens with what happens with what we call we call we call dreams. Uh, but yeah, dreams. Uh, but yeah, dreams. Uh, but yeah, that's what happens. that's what happens. Uh, okay, I'll move on Uh, okay, I'll move on to the orchestration. Uh, to the orchestration. Uh, to the orchestration. Uh, now that the skills have essentially now that the skills have essentially now that the skills have essentially been published been published been published . Uh, I think . Uh, I think . Uh, I think if I go over here, if I go over here, if I go over here, we should see we should see we should see if there are if there are if there are any questions at any questions at any questions at this point? Uh, I this point? Uh, I this point? Uh, I know there's a lot to know there's a lot to understand. Uh, understand. Uh, Travis, here's a Travis, here's a Travis, here's a question. Uh, and I know question. Uh, and I know question. Uh, and I know someone came up to someone came up to someone came up to me and asked me and asked me and asked how long it how long it how long it takes to takes to takes to get loans. It get loans. It get loans. It should only be 5 minutes should only be 5 minutes . Uh, I know the . Uh, I know the . Uh, I know the form says 1-2 form says 1-2 form says 1-2 days, but you should days, but you should days, but you should see it within see it within see it within 5 minutes.

5 minutes. 5 minutes. Uh, I was wondering Uh, I was wondering Uh, I was wondering about access to data about access to data , uh, from the SEC that a , uh, from the SEC that a research agent receives. research agent receives. Does this require Does this require Does this require any credentials, any credentials, any credentials, and are you going to and are you going to and are you going to explain how explain how explain how this credential injection this credential injection this credential injection or brokering works? or brokering works? or brokering works? Yes, the SEC Yes, the SEC Yes, the SEC only needs your email only needs your email only needs your email address and your name. address and your name. address and your name. Hmm, for this Hmm, for this Hmm, for this demonstration we're demonstration we're demonstration we're just going to insert this just going to insert this just going to insert this into the query. But in a into the query. But in a into the query. But in a real working real working real working application, we would application, we would application, we would use something use something use something called Vault. Um, called Vault. Um, called Vault. Um, the way Vaults work the way Vaults work the way Vaults work is that you is that you is that you define the secret define the secret define the secret you want to you want to you want to put in an put in an put in an environment variable. And instead of environment variable. And instead of environment variable. And instead of inserting the inserting the inserting the real secret, what real secret, what real secret, what we insert is a we insert is a we insert is a placeholder. Um, and placeholder. Um, and placeholder. Um, and then when Claude then when Claude then when Claude does work and wants does work and wants does work and wants to use that to use that to use that environment variable, environment variable, environment variable, he just inserts the he just inserts the he just inserts the placeholder value placeholder value . We at Egress replace . We at Egress replace . We at Egress replace it with the real it with the real it with the real value only on those value only on those value only on those host domains you have host domains you have host domains you have specified are allowed specified are allowed specified are allowed for this secret. for this secret. for this secret. So if you say So if you say So if you say my top secret my top secret my top secret environment can environment can only be used for the SEC and then only be used for the SEC and then Claude tries to Claude tries to Claude tries to use it use it use it for anthropic.com, we won't for anthropic.com, we won't for anthropic.com, we won't enter, we won't enter, we won't enter, we won't make a substitution. But make a substitution. But make a substitution. But when he tries when he tries when he tries to call SEC.com, he to call SEC.com, he to call SEC.com, he will make a substitute for will make a substitute for will make a substitute for him.

him. him. And it's essentially like an API And it's essentially like an API And it's essentially like an API that you can that you can that you can use in the use in the use in the console where you just console where you just console where you just create a repository, create a repository, create a repository, give it a name, give it a name, give it a name, get get get an ID, and an ID, and an ID, and then you essentially then you essentially then you essentially include that include that include that ID in your ID in your ID in your session so that we can session so that we can session so that we can authenticate authenticate authenticate with your MCPs, with your MCPs, with your MCPs, your CLI. For example, your CLI. For example, your CLI. For example, we could do we could do we could do something like a something like a Google Google Workspace CLI that you Workspace CLI that you Workspace CLI that you install on your install on your install on your sandbox, and then you sandbox, and then you sandbox, and then you can give us can give us can give us your API key as your API key as environment credentials, and again environment credentials, and again , we'll , we'll , we'll follow the same follow the same follow the same method that method that method that Gabe talked about. Gabe talked about. Gabe talked about. So this allows us to So this allows us to So this allows us to essentially essentially essentially also work with an also work with an also work with an OAuth grant. And they OAuth grant. And they OAuth grant. And they work too, work too, work too, Gabe, I don't know if Gabe, I don't know if Gabe, I don't know if you just want you just want you just want to show it on the to show it on the to show it on the console, as fast as console, as fast as console, as fast as creating a repository. creating a repository. creating a repository. Yes. Um, you can Yes. Um, you can Yes. Um, you can create a repository here, create a repository here, create a repository here, and then in the repository you and then in the repository you and then in the repository you merge the merge the merge the credentials. Um, you can credentials. Um, you can credentials. Um, you can have have have MCP authentication, um, maybe, you MCP authentication, um, maybe, you MCP authentication, um, maybe, you know, we have some know, we have some predefined things for predefined things for OAuth things. So, OAuth things. So, OAuth things. So, whenever you whenever you whenever you reference a session, reference a session, reference a session, you insert that you insert that storage ID, and it storage ID, and it merges all the merges all the merges all the credentials into your session, and credentials into your session, and credentials into your session, and whenever it whenever it whenever it tries tries tries to use to use any of the domains that any of the domains that any of the domains that you specified in your you specified in your you specified in your credentials, it credentials, it credentials, it replaces them with the replaces them with the replaces them with the real real real um credentials instead of um credentials instead of um credentials instead of the placeholder. Yes.

the placeholder. Yes. the placeholder. Yes. Quick question. What Quick question. What Quick question. What part of the capabilities part of the capabilities part of the capabilities are in the are in the are in the cloud agent SDK, and what cloud agent SDK, and what cloud agent SDK, and what is above it? is above it? is above it? Uh, can you...What do you... Uh, can you...What do you... What do you mean? What do you mean? You have the agent SDK, right You have the agent SDK, right ? I think it's ? I think it's ? I think it's built on top of the built on top of the built on top of the agent SDK. I want to know agent SDK. I want to know agent SDK. I want to know what part of what part of what part of the capabilities the capabilities the capabilities is at the is at the is at the agent SDK level and what agent SDK level and what agent SDK level and what is outside it? is outside it? is outside it? So the agent SDK So the agent SDK So the agent SDK is packaged, uh, sort of a is packaged, uh, sort of a is packaged, uh, sort of a headless headless headless version of the cloud code. version of the cloud code. version of the cloud code. So we actually So we actually So we actually created a generic created a generic created a generic version of this. We took version of this. We took version of this. We took a lot of inspiration from a lot of inspiration from a lot of inspiration from the benefits and the benefits and the benefits and lessons of the agent SDK, but lessons of the agent SDK, but lessons of the agent SDK, but we reworked the, uh, we reworked the, uh, we reworked the, uh, system itself to be system itself to be system itself to be more more more general and not general and not general and not just just just coding-specific. coding-specific. coding-specific. So this is something like a So this is something like a So this is something like a fork? Uh fork? Uh fork? Uh , yes. , yes. , yes. Yes, it's actually like its Yes, it's actually like its Yes, it's actually like its own system. An agent own system. An agent , as it were, is essentially , as it were, is essentially , as it were, is essentially like cloud code like cloud code like cloud code in the form of a binary in the form of a binary in the form of a binary file that you can file that you can file that you can take and run take and run take and run anywhere. This is a fully anywhere. This is a fully anywhere. This is a fully hosted solution hosted solution hosted solution running in our cloud running in our cloud running in our cloud and we have our own harness and we have our own harness and we have our own harness which is heavily which is heavily which is heavily inspired by the inspired by the inspired by the cloud code harness and we cloud code harness and we cloud code harness and we often say often say often say stay stay stay tuned for any tuned for any tuned for any improvements around the improvements around the improvements around the world. Um, I saw world. Um, I saw world. Um, I saw another hand in the back another hand in the back another hand in the back . There is a . There is a . There is a question here. Yes. A question here. Yes. A question here. Yes. A few more.

few more. few more. Thank you. Um, I think the Thank you. Um, I think the Thank you. Um, I think the issue is that issue is that issue is that you're you're you're orchestrating between orchestrating between orchestrating between multiple agents. Um multiple agents. Um , are there any protective , are there any protective , are there any protective barriers to keep these agents from barriers to keep these agents from barriers to keep these agents from getting stuck and getting stuck and getting stuck and consuming a lot of consuming a lot of consuming a lot of tokens while tokens while tokens while generating or generating or generating or collaborating? collaborating? collaborating? Um, some of them we Um, some of them we Um, some of them we leave up to you to leave up to you to orchestrate and orchestrate and figure out. I figure out. I figure out. I mean, if you're mean, if you're mean, if you're creating creating creating your own harnesses like that, there's nothing your own harnesses like that, there's nothing your own harnesses like that, there's nothing stopping you from stopping you from stopping you from using, uh, using, uh, using, uh, other providers other providers other providers if you want to if you want to if you want to create a create a create a multi- multi- multi- provider system. We're provider system. We're provider system. We're exploring the idea of exploring the idea of tighter tighter budget controls so that we can budget controls so that we can budget controls so that we can say, " say, " Don't use Don't use Don't use more than this amount of more than this amount of more than this amount of money per session." " money per session." " money per session." " Use no Use no Use no more than this more than this more than this number of tokens per number of tokens per number of tokens per session." But session." But session." But we don't we don't we don't offer that today. Yes, offer that today. Yes, offer that today. Yes, come on. come on. come on. More hands. More hands. More hands. Question: Earlier you Question: Earlier you Question: Earlier you showed us the showed us the manager transcript and then manager transcript and then the subagents in the console, and you the subagents in the console, and you the subagents in the console, and you could look through them could look through them could look through them and see what they were and see what they were and see what they were doing. If you doing. If you doing. If you encounter, perhaps, a encounter, perhaps, a encounter, perhaps, a mistake, or if mistake, or if mistake, or if something went something went something went the wrong way, the wrong way, the wrong way, or you want to or you want to or you want to change the story, it is change the story, it is change the story, it is obvious that this is what obvious that this is what obvious that this is what happened. Is there any happened. Is there any happened. Is there any way to fix this way to fix this way to fix this or go back to or go back to or go back to that point in time and that point in time and that point in time and start over, start over, start over, try with try with try with additional additional additional instructions? Is there instructions? Is there instructions? Is there anything similar here?

anything similar here? anything similar here? You can control You can control You can control this. So I could, I this. So I could, I this. So I could, I could, I think could, I think could, I think if I say, uh, if I say, uh, if I say, uh, send an event, I send an event, I send an event, I could send a new could send a new user message event, uh, user message event, uh, period, and control the period, and control the period, and control the agent. Uh, that'll agent. Uh, that'll agent. Uh, that'll put him in put him in put him in line. You know, you line. You know, you line. You know, you can send can send can send about 10 about 10 about 10 messages messages messages at a time, and that's at a time, and that's at a time, and that's fine. fine. fine. We don't have a way to We don't have a way to We don't have a way to rewind rewind rewind and fast forward right now, but that's definitely something that and fast forward right now, but that's definitely something that and fast forward right now, but that's definitely something that , uh, if people are , uh, if people are , uh, if people are interested, we'll interested, we'll interested, we'll look into. look into. look into. Yes, Yes, Yes, session branching is essentially what session branching is essentially what session branching is essentially what you're asking for, you're asking for, you're asking for, and we've had several and we've had several and we've had several customers ask customers ask customers ask for it. This is something that is in for it. This is something that is in for it. This is something that is in our plan, so our plan, so our plan, so hopefully we will have hopefully we will have hopefully we will have it someday. it someday. it someday. Thank you. Uh, yeah, I do Thank you. Uh, yeah, I do Thank you. Uh, yeah, I do n't like n't like n't like seeing a pair of hands here. seeing a pair of hands here. seeing a pair of hands here. If you want, If you want, If you want, share. Yes, share. Yes, share. Yes, thank you, Rohan. thank you, Rohan. thank you, Rohan. A. So, A. So, A. So, where are you? Good, good, where are you? Good, good, where are you? Good, good, good. Yes, good. Yes, good. Yes, continue. continue. continue. Assuming this all Assuming this all Assuming this all runs in the cloud, like runs in the cloud, like in... Do you have in... Do you have in... Do you have any roadmap any roadmap any roadmap for getting for getting for getting collective knowledge collective knowledge collective knowledge from these agents? from these agents? from these agents? For example, in this For example, in this For example, in this one particular one particular one particular application, you have 10 application, you have 10 application, you have 10 users. Could users. Could you please speak up? I'm you please speak up? I'm a little hard of hearing. a little hard of hearing. a little hard of hearing. A. So, A. So, A. So, assuming that this is all assuming that this is all assuming that this is all running in the cloud, and this running in the cloud, and this running in the cloud, and this application application application is distributed across is distributed across is distributed across dozens and hundreds of dozens and hundreds of dozens and hundreds of users, and users, and users, and they're all using they're all using they're all using it for the purpose of, you it for the purpose of, you it for the purpose of, you know, conducting know, conducting know, conducting financial financial financial research. Do you have research. Do you have ...I know you have ...I know you have ...I know you have session logs. Is there a session logs. Is there a session logs. Is there a collective memory collective memory collective memory that you are exploring, that you are exploring, that you are exploring, or even a roadmap or even a roadmap or even a roadmap ? We have it ? We have it ? We have it today.

today. today. We actually We actually We actually use it in use it in use it in this demonstration. this demonstration. this demonstration. So, we So, we So, we use a use a use a memory store that memory store that memory store that allows you allows you allows you to share to share to share knowledge between sessions. knowledge between sessions. knowledge between sessions. Uh, and you can Uh, and you can Uh, and you can see here what see here what see here what happens happens happens next: agents next: agents next: agents fill in fill in fill in individual individual individual information about what information about what information about what they've done so far. Uh they've done so far. Uh , and then I'll show you , and then I'll show you , and then I'll show you at the end how you can at the end how you can at the end how you can consolidate this consolidate this consolidate this knowledge over time knowledge over time knowledge over time through dreams, through dreams, through dreams, so that you can continually so that you can continually so that you can continually refine refine refine memory stores as memory stores as memory stores as different agents different agents different agents encounter different encounter different encounter different extreme cases extreme cases extreme cases and collectively and collectively and collectively learn from each learn from each learn from each other. other. other. So just to So just to So just to continue. So you continue. So you continue. So you store it in the store it in the store it in the Anthropic infrastructure. Anthropic infrastructure. Anthropic infrastructure. Let's say you Let's say you Let's say you want to want to want to decouple this and decouple this and decouple this and run something in AWS run something in AWS run something in AWS Bedrock, and I have my own Bedrock, and I have my own Bedrock, and I have my own data on our data on our data on our clusters or something for a clusters or something for a clusters or something for a "sandbox". I think "sandbox". I think "sandbox". I think you mentioned you mentioned you mentioned self-hosting. self-hosting. self-hosting. To what extent is this part To what extent is this part To what extent is this part tied to a specific tied to a specific tied to a specific system, and how system, and how system, and how flexible is it? flexible is it? flexible is it? So, it's all So, it's all So, it's all API-based, so API-based, so API-based, so you can pull you can pull you can pull memory and memory and memory and put it on put it on put it on different systems if different systems if different systems if you want. We currently do not you want. We currently do not you want. We currently do not support other support other memory vendors in the session out of memory vendors in the session out of the box, but you would the box, but you would the box, but you would simply create an MCP simply create an MCP simply create an MCP or a special or a special or a special tool for that.

tool for that. tool for that. This is how I imagine This is how I imagine This is how I imagine it. it. it. And also, to add to the And also, to add to the And also, to add to the initial question initial question initial question of memory availability of memory availability of memory availability at the user at the user at the user [or] level, we [or] level, we [or] level, we allow allow allow multiple memory sources to be connected multiple memory sources to be connected multiple memory sources to be connected . So you . So you . So you can imagine a can imagine a user-level memory store and user-level memory store and then some kind of then some kind of organization-level memory store, and then organization-level memory store, and then you carry both you carry both you carry both into each session to into each session to into each session to start collecting start collecting start collecting memory at the user level as well as the memory at the user level as well as the organization level. organization level. And then, as Gabe said And then, as Gabe said , none of this is , none of this is , none of this is tied to a particular tied to a particular tied to a particular system. You can system. You can system. You can take your data and take your data and take your data and go anywhere you want go anywhere you want go anywhere you want if you want if you want . Everything is an API call. . Everything is an API call. . Everything is an API call. Yes, I had a Yes, I had a Yes, I had a quick question. quick question. quick question. When you were going through the When you were going through the agent workflow, I noticed that agent workflow, I noticed that the evaluator is the evaluator is the evaluator is at the lead at the lead at the lead agent level, right? So how do you agent level, right? So how do you agent level, right? So how do you evaluate these evaluate these evaluate these subagent tasks that are subagent tasks that are subagent tasks that are being executed, and then being executed, and then being executed, and then how do you manage how do you manage how do you manage memory between them? memory between them? memory between them? Yes, that's a good Yes, that's a good Yes, that's a good question. So, there are question. So, there are question. So, there are two dimensions here. Subagents two dimensions here. Subagents , as we have created them, , as we have created them, , as we have created them, are designed to are designed to are designed to extend your extend your extend your context and context and context and leverage the leverage the leverage the capabilities of your capabilities of your capabilities of your session through session through session through collaboration between them. collaboration between them. The determination of The determination of the result the result the result occurs at the occurs at the occurs at the general general general session level. So, you session level. So, you session level. So, you define the outcome define the outcome define the outcome and say this is what and say this is what and say this is what I want you to I want you to I want you to achieve. Then the achieve. Then the achieve. Then the main agent main agent main agent uses it, uses it, uses it, you know, uses you know, uses you know, uses subagents to subagents to subagents to do this do this do this work. He work. He work. He collaborates between them and collaborates between them and collaborates between them and eventually says, "I'm eventually says, "I'm eventually says, "I'm done." There is currently done." There is currently done." There is currently no support for no support for no support for defining defining defining a sub-result. A a sub-result. A short sequel, short sequel, for example, since for example, since for example, since you have this you have this you have this dream feature, right? So how dream feature, right? So how dream feature, right? So how do you know if do you know if do you know if subagents have subagents have subagents have any, you know,

any, you know, any, you know, of course, actions, so how do of course, actions, so how do of course, actions, so how do you learn from that you learn from that ? ? ? Yes, today Yes, today Yes, today dreams are limited by dreams are limited by dreams are limited by memory stores. memory stores. memory stores. So, I'll show you this So, I'll show you this So, I'll show you this in a second, but in a second, but in a second, but basically how basically how basically how the dream works is they the dream works is they the dream works is they analyze analyze analyze transcripts and transcripts and transcripts and memories created memories created memories created over multiple over multiple over multiple sessions so they sessions so they sessions so they can update can update can update their memories so their memories so their memories so future agents future agents future agents know what to pull out. know what to pull out. But you can imagine But you can imagine that we can that we can that we can continue to continue to continue to extend this for extend this for extend this for skills, tips, and skills, tips, and skills, tips, and agent definitions agent definitions agent definitions so that you have this so that you have this continuous continuous learning feature. But right now learning feature. But right now learning feature. But right now we only have this for we only have this for we only have this for memory storage. memory storage. See, here's the hand, and See, here's the hand, and then I'll go back to then I'll go back to ... ... ... Hello. Hello. Hello. Yes. Yes. Yes. Here. Here. Here. Where, where is this? Where, where is this? Where, where is this? Here. Here. Here. Oh, hello. Oh, hello. Oh, hello. Thank you. If you have, Thank you. If you have, Thank you. If you have, let's say, your let's say, your let's say, your company's data, or I don't company's data, or I don't company's data, or I don't know, traditional know, traditional know, traditional storage, or rag, or storage, or rag, or storage, or rag, or something like that, and you something like that, and you something like that, and you want to expose it to want to expose it to want to expose it to this type of agent this type of agent this type of agent through a custom through a custom through a custom CLI or MCP, what would be a CLI or MCP, what would be a CLI or MCP, what would be a safe way to safe way to safe way to deploy it in deploy it in deploy it in your environment?

your environment? your environment? Yes, that's a great Yes, that's a great Yes, that's a great question. There are two question. There are two question. There are two ways to do this. ways to do this. ways to do this. If you If you If you are using, if are using, if are using, if you can you can you can access this via CLI access this via CLI access this via CLI using API or using API or using API or via MCP, you just via MCP, you just via MCP, you just need to create an MCP, need to create an MCP, need to create an MCP, give it an give it an give it an authentication store and it authentication store and it authentication store and it will be able will be able will be able to call your MCP directly. If to call your MCP directly. If to call your MCP directly. If you don't want to you don't want to you don't want to expose your MCP expose your MCP expose your MCP to the internet, we have to the internet, we have to the internet, we have a feature called MCP- a feature called MCP- tunnels that allows you tunnels that allows you tunnels that allows you to create a tunnel from to create a tunnel from to create a tunnel from your network to ours your network to ours your network to ours so you don't expose so you don't expose so you don't expose it to the internet, and it to the internet, and it to the internet, and we'll do it this we'll do it this we'll do it this way. If this is way. If this is way. If this is too much of a too much of a too much of a hurdle, we hurdle, we hurdle, we support support support self-hosted self-hosted self-hosted sandboxes so you sandboxes so you sandboxes so you can create can create can create your own sandbox that your own sandbox that your own sandbox that peers with your peers with your peers with your network and has access network and has access network and has access to your applications and to your applications and to your applications and services, and the agent services, and the agent services, and the agent will simply call will simply call will simply call the sandbox you the sandbox you the sandbox you manage and manage and manage and host and host and host and thus have access thus have access thus have access to your resources. to your resources. Yes. Yes. I had a question I had a question I had a question about dependencies for about dependencies for about dependencies for environments. So, I environments. So, I environments. So, I saw something that you saw something that you saw something that you can download in can download in can download in dependencies. Is this during dependencies. Is this during dependencies. Is this during execution or execution or execution or just before just before just before creation?

creation? creation? It's creation time, uh, It's creation time, uh, It's creation time, uh, and then we cache and then we cache and then we cache it and it and it and reuse, uh, reuse, uh, reuse, uh, this environment, this environment, this environment, for example, for for example, for for example, for repeated repeated sandbox instances. sandbox instances. So, can you So, can you So, can you specify internal specify internal specify internal dependencies, or do you dependencies, or do you dependencies, or do you need to choose an need to choose an need to choose an NPM package? NPM package? NPM package? Now, uh, they're all there. Now, uh, they're all there. Now, uh, they're all there. They should all be They should all be They should all be available via NPM, Cargo, available via NPM, Cargo, available via NPM, Cargo, uh, pip, whatever. uh, pip, whatever. uh, pip, whatever. But you can But you can But you can upload files, upload files, upload files, uh, directly into uh, directly into uh, directly into the session if, or in a the session if, or in a the session if, or in a GitHub repository if GitHub repository if GitHub repository if you want to install them you want to install them you want to install them that way. that way. that way. Good. And another, uh, Good. And another, uh, Good. And another, uh, deployment, I see a deployment, I see a deployment, I see a tab for that. tab for that. tab for that. Yes, yes. I'm going to Yes, yes. I'm going to Yes, yes. I'm going to get to that. I'm get to that. I'm get to that. I'm going to follow going to follow going to follow this up with a demonstration, and this up with a demonstration, and this up with a demonstration, and then we'll answer then we'll answer then we'll answer more questions in a more questions in a more questions in a second.