Agent Frameworks Considered Harmful — Rémi Louf, .txt

Hi everyone. So originally I thought I Hi everyone. So originally I thought I was going to give a very technical talk was going to give a very technical talk was going to give a very technical talk but I saw was in the leadership track but I saw was in the leadership track but I saw was in the leadership track which I'm not sure what it means but I which I'm not sure what it means but I which I'm not sure what it means but I was like okay I'm going to do half high was like okay I'm going to do half high was like okay I'm going to do half high level and half technical. So it's more a level and half technical. So it's more a level and half technical. So it's more a story about what I you know what I did story about what I you know what I did story about what I you know what I did in January because in around December um in January because in around December um in January because in around December um agents kind of became really good uh you agents kind of became really good uh you agents kind of became really good uh you know there was a step function something know there was a step function something know there was a step function something happened uh I think it was opus 4.6 six happened uh I think it was opus 4.6 six happened uh I think it was opus 4.6 six and that's when I realized and I work in and that's when I realized and I work in and that's when I realized and I work in AI where I was like okay this thing is AI where I was like okay this thing is AI where I was like okay this thing is really happening and so I took two weeks really happening and so I took two weeks really happening and so I took two weeks out I took two weeks away so I'm the CEO out I took two weeks away so I'm the CEO out I took two weeks away so I'm the CEO of text uh you know which is 15 people of text uh you know which is 15 people of text uh you know which is 15 people company I just told my CTO I was like company I just told my CTO I was like company I just told my CTO I was like okay I'm just going to go away for two okay I'm just going to go away for two okay I'm just going to go away for two weeks and I'm just going to dive in this weeks and I'm just going to dive in this weeks and I'm just going to dive in this thing and try to understand what we can thing and try to understand what we can thing and try to understand what we can get out of it and how good it is and so get out of it and how good it is and so get out of it and how good it is and so the story is you know it is the story of the story is you know it is the story of the story is you know it is the story of me scratching my own itch for two weeks me scratching my own itch for two weeks me scratching my own itch for two weeks and trying to figure out how we can use and trying to figure out how we can use and trying to figure out how we can use actually use agents and what are good actually use agents and what are good actually use agents and what are good primitives to build agents and whether primitives to build agents and whether primitives to build agents and whether you know it already exists. Um this was you know it already exists. Um this was you know it already exists. Um this was a really clickbait title but actually it a really clickbait title but actually it a really clickbait title but actually it turns out to be a good title even for turns out to be a good title even for turns out to be a good title even for this talk. Um so what you can see here this talk. Um so what you can see here this talk. Um so what you can see here on the left the castle is my office. Uh on the left the castle is my office. Uh on the left the castle is my office. Uh that's true. I do rent an office in that's true. I do rent an office in that's true. I do rent an office in Castle and the small thing with an arrow Castle and the small thing with an arrow Castle and the small thing with an arrow uh that you can see is like this robot uh that you can see is like this robot uh that you can see is like this robot mower which kind of works unattended all mower which kind of works unattended all mower which kind of works unattended all day every day. Uh it just does its stuff day every day. Uh it just does its stuff day every day. Uh it just does its stuff in the background without anyone having

in the background without anyone having in the background without anyone having to use a remote control or think about to use a remote control or think about to use a remote control or think about it or anything. And I kind of wanted the it or anything. And I kind of wanted the it or anything. And I kind of wanted the same thing for my morning because my same thing for my morning because my same thing for my morning because my mornings are always the same thing the mornings are always the same thing the mornings are always the same thing the first couple hours. It's browse market first couple hours. It's browse market first couple hours. It's browse market news review of like linear could be Jira news review of like linear could be Jira news review of like linear could be Jira my CRM and also I like to walk for an my CRM and also I like to walk for an my CRM and also I like to walk for an about an hour in the morning and then about an hour in the morning and then about an hour in the morning and then the next hour is spent trying to process the next hour is spent trying to process the next hour is spent trying to process the really long voice note that you know the really long voice note that you know the really long voice note that you know was recorded while walking and you know was recorded while walking and you know was recorded while walking and you know all I wanted was my morning briefing all I wanted was my morning briefing all I wanted was my morning briefing with my coffee and that's kind of what with my coffee and that's kind of what with my coffee and that's kind of what we've been told for a couple of years we've been told for a couple of years we've been told for a couple of years like what the future would be. Um, but like what the future would be. Um, but like what the future would be. Um, but then when you really start working with then when you really start working with then when you really start working with it, even if you're not coding, all you it, even if you're not coding, all you it, even if you're not coding, all you get is a TUI today. Uh, so it's amazing. get is a TUI today. Uh, so it's amazing. get is a TUI today. Uh, so it's amazing. You can code. You can actually, you You can code. You can actually, you You can code. You can actually, you know, I started doing things that were know, I started doing things that were know, I started doing things that were not coding in it. They're great for not coding in it. They're great for not coding in it. They're great for this. Uh, agents are great for this, but this. Uh, agents are great for this, but this. Uh, agents are great for this, but it's kind of the equivalent of having a it's kind of the equivalent of having a it's kind of the equivalent of having a robot like a tractor mower that you robot like a tractor mower that you robot like a tractor mower that you still have to stay on even if it's still have to stay on even if it's still have to stay on even if it's driving by itself, right? It's kind of driving by itself, right? It's kind of driving by itself, right? It's kind of very frustrating because you have to it very frustrating because you have to it very frustrating because you have to it can do many things but you still have to can do many things but you still have to can do many things but you still have to be on and so of course the labs didn't be on and so of course the labs didn't be on and so of course the labs didn't stop there and they came up with apps uh stop there and they came up with apps uh stop there and they came up with apps uh which I call basically SSH with vibes which I call basically SSH with vibes which I call basically SSH with vibes that's great but in this situation when that's great but in this situation when that's great but in this situation when that came up I was like wow that's that came up I was like wow that's that came up I was like wow that's awesome I don't have to use like a ter awesome I don't have to use like a ter awesome I don't have to use like a ter like SSH on my phone anymore codeex is like SSH on my phone anymore codeex is like SSH on my phone anymore codeex is great however I noticed I just started

great however I noticed I just started great however I noticed I just started you know I was on my walk and I was just you know I was on my walk and I was just you know I was on my walk and I was just instructing the agent to do things while instructing the agent to do things while instructing the agent to do things while I was walking and so I wasn't thinking I was walking and so I wasn't thinking I was walking and so I wasn't thinking you know very clearly anymore. I just you know very clearly anymore. I just you know very clearly anymore. I just started running agents on my phone started running agents on my phone started running agents on my phone during my morning walk. And this is not during my morning walk. And this is not during my morning walk. And this is not great because this is the equivalent of great because this is the equivalent of great because this is the equivalent of this is you're kind of midway uh you this is you're kind of midway uh you this is you're kind of midway uh you know it's not the tractor that you have know it's not the tractor that you have know it's not the tractor that you have to stay on. It can actually do something to stay on. It can actually do something to stay on. It can actually do something without you being right next to it. But without you being right next to it. But without you being right next to it. But you still have this remote control that you still have this remote control that you still have this remote control that you know you kind of have to change the you know you kind of have to change the you know you kind of have to change the trajectory every now and then. Uh that's trajectory every now and then. Uh that's trajectory every now and then. Uh that's useful. It's kind of absurd uh when you useful. It's kind of absurd uh when you useful. It's kind of absurd uh when you think about it and actually when you think about it and actually when you think about it and actually when you look at people like on their phone all look at people like on their phone all look at people like on their phone all the time just doing this, it's kind of the time just doing this, it's kind of the time just doing this, it's kind of absurd and it's clearly transitional. absurd and it's clearly transitional. absurd and it's clearly transitional. Like surely we're not it's not it's not Like surely we're not it's not it's not Like surely we're not it's not it's not going to stop there. Um and so I did a going to stop there. Um and so I did a going to stop there. Um and so I did a very dumb thing as a co which is I very dumb thing as a co which is I very dumb thing as a co which is I started coding uh don't tell my board started coding uh don't tell my board started coding uh don't tell my board and I started to build the dumbest thing and I started to build the dumbest thing and I started to build the dumbest thing that could possibly work. And of course that could possibly work. And of course that could possibly work. And of course it became a really a crazy rabbit hole. it became a really a crazy rabbit hole. it became a really a crazy rabbit hole. Uh the repo is there if you want to take Uh the repo is there if you want to take Uh the repo is there if you want to take a look at it. The code is not amazing.

a look at it. The code is not amazing. a look at it. The code is not amazing. Uh but it works. Um so the first thing Uh but it works. Um so the first thing Uh but it works. Um so the first thing is that you know I started using is that you know I started using is that you know I started using frameworks. Uh I mean there are great frameworks. Uh I mean there are great frameworks. Uh I mean there are great frameworks. I'm not going to name any frameworks. I'm not going to name any frameworks. I'm not going to name any frameworks because they're all good in frameworks because they're all good in frameworks because they're all good in their own way and they all have flows in their own way and they all have flows in their own way and they all have flows in their own way which is fine. Uh but I their own way which is fine. Uh but I their own way which is fine. Uh but I spent all my time actually editing the spent all my time actually editing the spent all my time actually editing the prompt within the code and I was like prompt within the code and I was like prompt within the code and I was like this is actually not very useful. So I'm this is actually not very useful. So I'm this is actually not very useful. So I'm like everyone here, I hate YAML u like like everyone here, I hate YAML u like like everyone here, I hate YAML u like the next guy, but I still found that the next guy, but I still found that the next guy, but I still found that this was actually a lot easier to start this was actually a lot easier to start this was actually a lot easier to start implementing agents without code. Uh you implementing agents without code. Uh you implementing agents without code. Uh you can version it, you can diff it, uh you can version it, you can diff it, uh you can version it, you can diff it, uh you can review in the PR. Um but it's just can review in the PR. Um but it's just can review in the PR. Um but it's just and it's just so easy. You can just, you and it's just so easy. You can just, you and it's just so easy. You can just, you know, write your file, you drop it in a know, write your file, you drop it in a know, write your file, you drop it in a folder, and then it just magically folder, and then it just magically folder, and then it just magically appears once you have the runtime and it appears once you have the runtime and it appears once you have the runtime and it just magically works. Um, and you know, just magically works. Um, and you know, just magically works. Um, and you know, then I needed like my market watch to then I needed like my market watch to then I needed like my market watch to run every morning while I'm, you know, run every morning while I'm, you know, run every morning while I'm, you know, while I'm walking in the fields. And for while I'm walking in the fields. And for while I'm walking in the fields. And for that we have things that, you know, uh, that we have things that, you know, uh, that we have things that, you know, uh, have been around for a while, which is have been around for a while, which is have been around for a while, which is chron jobs. uh and schedules specify you chron jobs. uh and schedules specify you chron jobs. uh and schedules specify you know when the agents need to be run and know when the agents need to be run and know when the agents need to be run and also we'll see it's very important later also we'll see it's very important later also we'll see it's very important later uh they publish uh they publish events uh they publish uh they publish events uh they publish uh they publish events and you know markdown and chron and you know markdown and chron and you know markdown and chron obviously you know it's much more obviously you know it's much more obviously you know it's much more complicated than under the hood but the complicated than under the hood but the complicated than under the hood but the interface is this you don't write code interface is this you don't write code interface is this you don't write code and that's the whole product so far and and that's the whole product so far and and that's the whole product so far and honestly just mostly worked at this

honestly just mostly worked at this honestly just mostly worked at this point I'll I'll come back on mostly uh point I'll I'll come back on mostly uh point I'll I'll come back on mostly uh later And so this is actually a real later And so this is actually a real later And so this is actually a real picture of my one of my morning walks. picture of my one of my morning walks. picture of my one of my morning walks. And so what I do is I record voice notes And so what I do is I record voice notes And so what I do is I record voice notes while I'm walking. Uh but chron you know while I'm walking. Uh but chron you know while I'm walking. Uh but chron you know chron jobs I mean people would use chron chron jobs I mean people would use chron chron jobs I mean people would use chron jobs for this because that's what's jobs for this because that's what's jobs for this because that's what's available in codeex today but they're available in codeex today but they're available in codeex today but they're not ideal because they cover when but not ideal because they cover when but not ideal because they cover when but this is just one point in time. It this is just one point in time. It this is just one point in time. It doesn't cover because this happened and doesn't cover because this happened and doesn't cover because this happened and you know things that happened in our you know things that happened in our you know things that happened in our system like automatically when you drop system like automatically when you drop system like automatically when you drop the voice note now in the system it will the voice note now in the system it will the voice note now in the system it will emit an event and an agent will react to emit an event and an agent will react to emit an event and an agent will react to that event and it's the same thing when that event and it's the same thing when that event and it's the same thing when you have a new email a new entry in the you have a new email a new entry in the you have a new email a new entry in the CRM I mean anything a new PR that's open CRM I mean anything a new PR that's open CRM I mean anything a new PR that's open a new PR that's merged etc just reacts a new PR that's merged etc just reacts a new PR that's merged etc just reacts to events it's not just a chrome job and to events it's not just a chrome job and to events it's not just a chrome job and that and that means that you know agents that and that means that you know agents that and that means that you know agents of the Voice note processor is just you of the Voice note processor is just you of the Voice note processor is just you know not a chron job but here you have know not a chron job but here you have know not a chron job but here you have accepts and returns. So it just declare accepts and returns. So it just declare accepts and returns. So it just declare what it accepts and what it returns as what it accepts and what it returns as what it accepts and what it returns as an event and here it accepts a voice an event and here it accepts a voice an event and here it accepts a voice note transcribes it turn it into durable note transcribes it turn it into durable note transcribes it turn it into durable notes on the right and it emits a new notes on the right and it emits a new notes on the right and it emits a new event and for that it uses structured event and for that it uses structured event and for that it uses structured outputs. We'll come back to this. And outputs. We'll come back to this. And outputs. We'll come back to this. And you know, now we finally have the future you know, now we finally have the future you know, now we finally have the future we're promised because that voice note we're promised because that voice note we're promised because that voice note agent emits voice note processed. And agent emits voice note processed. And agent emits voice note processed. And then I have my daily brief agent that then I have my daily brief agent that then I have my daily brief agent that actually will take the output of the actually will take the output of the actually will take the output of the chrome job. We'll take the output of the

chrome job. We'll take the output of the chrome job. We'll take the output of the voice note agents and we create my daily voice note agents and we create my daily voice note agents and we create my daily brief which is posted as a slack brief which is posted as a slack brief which is posted as a slack message. So the slack message.post post message. So the slack message.post post message. So the slack message.post post event is actually uh is actually like um event is actually uh is actually like um event is actually uh is actually like um a process actually subscribes to this a process actually subscribes to this a process actually subscribes to this and emits uh and sends a slack message and emits uh and sends a slack message and emits uh and sends a slack message to me. It's actually this is a real this to me. It's actually this is a real this to me. It's actually this is a real this is a real thing. It's working. I can is a real thing. It's working. I can is a real thing. It's working. I can show you after on my phone and you know show you after on my phone and you know show you after on my phone and you know there are frameworks that are going to there are frameworks that are going to there are frameworks that are going to sell you the fact that you need graphs sell you the fact that you need graphs sell you the fact that you need graphs for this in code. Uh you do not need for this in code. Uh you do not need for this in code. Uh you do not need graph uh in this case. All you need is graph uh in this case. All you need is graph uh in this case. All you need is events. You have no edges to maintain. events. You have no edges to maintain. events. You have no edges to maintain. Agents sub simply subscribe to events. Agents sub simply subscribe to events. Agents sub simply subscribe to events. Anyone can come in and edit this. You Anyone can come in and edit this. You Anyone can come in and edit this. You don't need to. Yeah, you don't need to don't need to. Yeah, you don't need to don't need to. Yeah, you don't need to know how to code. You just need to know know how to code. You just need to know know how to code. You just need to know what events exist in the system. Fun and what events exist in the system. Fun and what events exist in the system. Fun and fan out are free. No code and it's just fan out are free. No code and it's just fan out are free. No code and it's just drop a file and the topology emerges drop a file and the topology emerges drop a file and the topology emerges whatever the log says happened. whatever the log says happened. whatever the log says happened. And And And you know then of course I I tried to run you know then of course I I tried to run you know then of course I I tried to run it. So the first version took about I it. So the first version took about I it. So the first version took about I mean you know I cheated I cheated I used mean you know I cheated I cheated I used mean you know I cheated I cheated I used uh I used codeex and it took about like uh I used codeex and it took about like uh I used codeex and it took about like a day to write like the first thing uh a day to write like the first thing uh a day to write like the first thing uh out of my week but of course I tried it out of my week but of course I tried it out of my week but of course I tried it and it broke uh so these are real and it broke uh so these are real and it broke uh so these are real examples actually the dates know but examples actually the dates know but examples actually the dates know but it's real examples it's like the first it's real examples it's like the first it's real examples it's like the first day delay brief was posted to Slack day delay brief was posted to Slack day delay brief was posted to Slack twice um on Wednesday one of my voice twice um on Wednesday one of my voice twice um on Wednesday one of my voice notes completely vanished and then you

notes completely vanished and then you notes completely vanished and then you know towards the end of the week. I I know towards the end of the week. I I know towards the end of the week. I I kind of like played with the prompts all kind of like played with the prompts all kind of like played with the prompts all week and the market brief was garbage, week and the market brief was garbage, week and the market brief was garbage, but I didn't version uh I didn't version but I didn't version uh I didn't version but I didn't version uh I didn't version my changes and I couldn't remember my changes and I couldn't remember my changes and I couldn't remember actually what I changed in the prompt actually what I changed in the prompt actually what I changed in the prompt that made the thing completely useless. that made the thing completely useless. that made the thing completely useless. Now, if there are distributed or existed Now, if there are distributed or existed Now, if there are distributed or existed engineers in the room, you probably know engineers in the room, you probably know engineers in the room, you probably know this shopping list already. uh there is this shopping list already. uh there is this shopping list already. uh there is nothing new under the sun and you know nothing new under the sun and you know nothing new under the sun and you know each failure so each of these failure each failure so each of these failure each failure so each of these failure modes that you found actually led to modes that you found actually led to modes that you found actually led to building one piece of what turned out to building one piece of what turned out to building one piece of what turned out to be a runtime. So the lost note actually be a runtime. So the lost note actually be a runtime. So the lost note actually turned into a log. I just wanted turned into a log. I just wanted turned into a log. I just wanted everything to be saved forever so that I everything to be saved forever so that I everything to be saved forever so that I could go back to it and look into uh could go back to it and look into uh could go back to it and look into uh into what happened. the duplicates. It into what happened. the duplicates. It into what happened. the duplicates. It was because I was not following, you was because I was not following, you was because I was not following, you know, it did several attempts and I was know, it did several attempts and I was know, it did several attempts and I was not following them. I didn't have a not following them. I didn't have a not following them. I didn't have a proper queue. I wasn't counting the proper queue. I wasn't counting the proper queue. I wasn't counting the attempts, etc., etc. And then probably attempts, etc., etc. And then probably attempts, etc., etc. And then probably the most interesting part is the last the most interesting part is the last the most interesting part is the last prompt. I got into a really deep rabbit prompt. I got into a really deep rabbit prompt. I got into a really deep rabbit hole in there and I just ended up uh hole in there and I just ended up uh hole in there and I just ended up uh building a content like a content building a content like a content building a content like a content addressed system for this uh content addressed system for this uh content addressed system for this uh content addressed system. You can think of git addressed system. You can think of git addressed system. You can think of git uh you can think of nyx and any other uh you can think of nyx and any other uh you can think of nyx and any other build system. And you know that was and build system. And you know that was and build system. And you know that was and I didn't do this because I wanted to I didn't do this because I wanted to I didn't do this because I wanted to design a runtime. I mean by that point I design a runtime. I mean by that point I design a runtime. I mean by that point I still just wanted my agents to work and still just wanted my agents to work and still just wanted my agents to work and I also like the distraction. Uh and I

I also like the distraction. Uh and I I also like the distraction. Uh and I just paid off debt as it appeared like just paid off debt as it appeared like just paid off debt as it appeared like errors as they appeared. errors as they appeared. errors as they appeared. I hope my board won't see this talk. Uh I hope my board won't see this talk. Uh I hope my board won't see this talk. Uh so the log the log is the systems so the log the log is the systems so the log the log is the systems memory. Uh nothing is lost and memory. Uh nothing is lost and memory. Uh nothing is lost and everything is observed. uh you can you everything is observed. uh you can you everything is observed. uh you can you know you only have one append only know you only have one append only know you only have one append only events table on the left it's a real events table on the left it's a real events table on the left it's a real common line uh in like command za events common line uh in like command za events common line uh in like command za events and you get all the events they are and you get all the events they are and you get all the events they are causally linked as well like you know causally linked as well like you know causally linked as well like you know which event triggered which event which which event triggered which event which which event triggered which event which happens to be super useful when you're happens to be super useful when you're happens to be super useful when you're debugging and you know even with three debugging and you know even with three debugging and you know even with three four agents you start having like major four agents you start having like major four agents you start having like major debugging headaches so that was super debugging headaches so that was super debugging headaches so that was super super helpful Um super helpful Um super helpful Um and everything is querable which again and everything is querable which again and everything is querable which again for debugging. for debugging. for debugging. The second thing is you know okay we The second thing is you know okay we The second thing is you know okay we have a log so we can trace back things have a log so we can trace back things have a log so we can trace back things etc. But it's still really hard to know etc. But it's still really hard to know etc. But it's still really hard to know what went into the like what went to the what went into the like what went to the what went into the like what went to the model what prompt was sent to the model model what prompt was sent to the model model what prompt was sent to the model again because what you see when you're again because what you see when you're again because what you see when you're using codecs is kind of a lie. like you using codecs is kind of a lie. like you using codecs is kind of a lie. like you kind of have like a live chat session kind of have like a live chat session kind of have like a live chat session with the model and so you tend to think with the model and so you tend to think with the model and so you tend to think that oh that's what the model saw and that oh that's what the model saw and that oh that's what the model saw and you know that's exactly so I can you know that's exactly so I can you know that's exactly so I can understand what happened the truth is understand what happened the truth is understand what happened the truth is that's not exactly what the model saw um that's not exactly what the model saw um that's not exactly what the model saw um there are many reasons for that one is I there are many reasons for that one is I there are many reasons for that one is I mean compaction obviously is a big part mean compaction obviously is a big part mean compaction obviously is a big part is a big thing but also you know there is a big thing but also you know there is a big thing but also you know there are just quirks also you know openai

are just quirks also you know openai are just quirks also you know openai doesn't share or enthropic for that doesn't share or enthropic for that doesn't share or enthropic for that matter don't share the thinking with you matter don't share the thinking with you matter don't share the thinking with you the thinking traces so you have no idea the thinking traces so you have no idea the thinking traces so you have no idea I mean kind of have an idea of what went I mean kind of have an idea of what went I mean kind of have an idea of what went in but not completely either. And so you in but not completely either. And so you in but not completely either. And so you need something different. Uh you need need something different. Uh you need need something different. Uh you need something different and that was the big something different and that was the big something different and that was the big rabbit hole which is trying to find a rabbit hole which is trying to find a rabbit hole which is trying to find a way or build a system where you can way or build a system where you can way or build a system where you can trace back to what the model saw trace back to what the model saw trace back to what the model saw internally. And so what I did was internally. And so what I did was internally. And so what I did was basically built I mean nothing new. This basically built I mean nothing new. This basically built I mean nothing new. This is basically how build system works. Uh is basically how build system works. Uh is basically how build system works. Uh so you have different parts for a so you have different parts for a so you have different parts for a prompt. You have your system prompt. You prompt. You have your system prompt. You prompt. You have your system prompt. You have a description of your first skill have a description of your first skill have a description of your first skill of a second skill. Then you have the of a second skill. Then you have the of a second skill. Then you have the description of your tools. You have your description of your tools. You have your description of your tools. You have your user message which is the question to user message which is the question to user message which is the question to the model. Each one of those is stored the model. Each one of those is stored the model. Each one of those is stored and addressed and you know stored and addressed and you know stored and addressed and you know stored somewhere as a identifier which is a somewhere as a identifier which is a somewhere as a identifier which is a hash. And so when we build a prompt hash. And so when we build a prompt hash. And so when we build a prompt instead of building a piece of I mean instead of building a piece of I mean instead of building a piece of I mean before rendering the text we actually before rendering the text we actually before rendering the text we actually represent the prompt as a list of these represent the prompt as a list of these represent the prompt as a list of these u of these hashes. And so what that u of these hashes. And so what that u of these hashes. And so what that means is that down the line when I have means is that down the line when I have means is that down the line when I have a model answer, which by the way is also a model answer, which by the way is also a model answer, which by the way is also stored in the same way, we can trace stored in the same way, we can trace stored in the same way, we can trace back to the prompt very easily. And then back to the prompt very easily. And then back to the prompt very easily. And then from that prompt, we can know exactly from that prompt, we can know exactly from that prompt, we can know exactly what went into the model's context, what went into the model's context, what went into the model's context, which actually matters a lot. I mean, it which actually matters a lot. I mean, it which actually matters a lot. I mean, it matters a lot for debugging, but it also matters a lot for debugging, but it also matters a lot for debugging, but it also matters. I mean, it makes compaction a matters. I mean, it makes compaction a matters. I mean, it makes compaction a lot easier. You're just manipulating a lot easier. You're just manipulating a lot easier. You're just manipulating a graph, right? You're not manipulating graph, right? You're not manipulating graph, right? You're not manipulating strings. It's just a lot easier. And it

strings. It's just a lot easier. And it strings. It's just a lot easier. And it makes KV cache management a lot easier makes KV cache management a lot easier makes KV cache management a lot easier as well indirectly. And but I think that as well indirectly. And but I think that as well indirectly. And but I think that when you know I guess probably when you know I guess probably when you know I guess probably the main advantage that's when you use the main advantage that's when you use the main advantage that's when you use that scale is really auditability. It's that scale is really auditability. It's that scale is really auditability. It's like you can know exactly what happened like you can know exactly what happened like you can know exactly what happened with that agent and why it returned what with that agent and why it returned what with that agent and why it returned what it returned. And so you know I'm just it returned. And so you know I'm just it returned. And so you know I'm just going to go pretty pretty quickly over going to go pretty pretty quickly over going to go pretty pretty quickly over this. Uh what you get once you have this this. Uh what you get once you have this this. Uh what you get once you have this graph is you get diffs like you can say graph is you get diffs like you can say graph is you get diffs like you can say okay what changed between these two runs okay what changed between these two runs okay what changed between these two runs like which com which components changed like which com which components changed like which com which components changed was it just my message did I like give was it just my message did I like give was it just my message did I like give the model a different skill did I give the model a different skill did I give the model a different skill did I give it a different tool so you can just yeah it a different tool so you can just yeah it a different tool so you can just yeah you can just run this function and it you can just run this function and it you can just run this function and it will show you you know the difference will show you you know the difference will show you you know the difference between uh the runs so here you have you between uh the runs so here you have you between uh the runs so here you have you [snorts] know three components that were [snorts] know three components that were [snorts] know three components that were identical there's one which is you know identical there's one which is you know identical there's one which is you know the user message changed And then you the user message changed And then you the user message changed And then you had all these other messages that were had all these other messages that were had all these other messages that were actually you know that were continuing. actually you know that were continuing. actually you know that were continuing. It's continuation of a single session.

It's continuation of a single session. It's continuation of a single session. Uh then you have another thing for free Uh then you have another thing for free Uh then you have another thing for free which is replace. Uh replays turned out which is replace. Uh replays turned out which is replace. Uh replays turned out to be really useful for me because to be really useful for me because to be really useful for me because after a while I mean when I saw the cost after a while I mean when I saw the cost after a while I mean when I saw the cost ramp up like the thing when you have ramp up like the thing when you have ramp up like the thing when you have observability is that you do realize observability is that you do realize observability is that you do realize that cost increase very quickly. I that cost increase very quickly. I that cost increase very quickly. I wanted to try with open source models wanted to try with open source models wanted to try with open source models and so I wanted to rebuild old request and so I wanted to rebuild old request and so I wanted to rebuild old request for to eval. and see if I got the same for to eval. and see if I got the same for to eval. and see if I got the same thing out, if I got something thing out, if I got something thing out, if I got something satisfactory, if I need to change satisfactory, if I need to change satisfactory, if I need to change anything. And turns out that once you anything. And turns out that once you anything. And turns out that once you have uh you know this content addressing have uh you know this content addressing have uh you know this content addressing system, you can rebuild the request from system, you can rebuild the request from system, you can rebuild the request from the graph and you can just replay it the graph and you can just replay it the graph and you can just replay it exactly the same and you can you know exactly the same and you can you know exactly the same and you can you know resend, you can use a different model. resend, you can use a different model. resend, you can use a different model. Uh you can use a different request if Uh you can use a different request if Uh you can use a different request if you want, you can uh you can change it you want, you can uh you can change it you want, you can uh you can change it and so yeah, you get actually a lot of and so yeah, you get actually a lot of and so yeah, you get actually a lot of things I mean for free. You need to things I mean for free. You need to things I mean for free. You need to implement a thing. Um and so this is implement a thing. Um and so this is implement a thing. Um and so this is kind of different um from what you find kind of different um from what you find kind of different um from what you find I mean what I found when I started doing I mean what I found when I started doing I mean what I found when I started doing this it might be different today because this it might be different today because this it might be different today because it was a couple of months ago is that it was a couple of months ago is that it was a couple of months ago is that out there you had a lot of libraries so out there you had a lot of libraries so out there you had a lot of libraries so it's just frameworks and frameworks just it's just frameworks and frameworks just it's just frameworks and frameworks just call code uh your agents leave inside call code uh your agents leave inside call code uh your agents leave inside their abstractions their abstractions their abstractions um and um and um and I don't like analogies with you know I don't like analogies with you know I don't like analogies with you know operating system okay everyone has used operating system okay everyone has used operating system okay everyone has used that analogy but okay let's say a kernel that analogy but okay let's say a kernel that analogy but okay let's say a kernel like runs processes and your agent kind like runs processes and your agent kind like runs processes and your agent kind of is a process. It doesn't matter what of is a process. It doesn't matter what of is a process. It doesn't matter what it does actually. Uh but the system can it does actually. Uh but the system can it does actually. Uh but the system can schedule it is built to isolate it. It schedule it is built to isolate it. It schedule it is built to isolate it. It can isolate it and journals it with the can isolate it and journals it with the can isolate it and journals it with the log and the agent definition. So the

log and the agent definition. So the log and the agent definition. So the markdown is user land like you don't markdown is user land like you don't markdown is user land like you don't need to use it with that system if you need to use it with that system if you need to use it with that system if you don't want to actually have a front end don't want to actually have a front end don't want to actually have a front end that doesn't use this markdown uh this that doesn't use this markdown uh this that doesn't use this markdown uh this markdown format at all. And okay, here's markdown format at all. And okay, here's markdown format at all. And okay, here's a very important point and you know a very important point and you know a very important point and you know that's kind of a takeaway and it's also that's kind of a takeaway and it's also that's kind of a takeaway and it's also what justifies me working on this what justifies me working on this what justifies me working on this because disclaimer structured outputs is because disclaimer structured outputs is because disclaimer structured outputs is our specialty and we've been working on our specialty and we've been working on our specialty and we've been working on this for three years and it just ended this for three years and it just ended this for three years and it just ended up being a big dog fooding project and up being a big dog fooding project and up being a big dog fooding project and the reason why I did this at the the reason why I did this at the the reason why I did this at the beginning was not because I absolutely beginning was not because I absolutely beginning was not because I absolutely wanted to use our software. I didn't wanted to use our software. I didn't wanted to use our software. I didn't necessarily want to you know forklP to necessarily want to you know forklP to necessarily want to you know forklP to other software etc. It's just because other software etc. It's just because other software etc. It's just because Infropic was terrible at structured Infropic was terrible at structured Infropic was terrible at structured outputs and so like 20% of my events outputs and so like 20% of my events outputs and so like 20% of my events were wrong and were rejected by the were wrong and were rejected by the were wrong and were rejected by the system. So that's why I ended up doing system. So that's why I ended up doing system. So that's why I ended up doing this. And the goal, you know, the job of this. And the goal, you know, the job of this. And the goal, you know, the job of the kernel is actually to make bad the kernel is actually to make bad the kernel is actually to make bad actions impossible, not just unlikely. actions impossible, not just unlikely. actions impossible, not just unlikely. And so you have this two boundaries with And so you have this two boundaries with And so you have this two boundaries with between agents and the external world. between agents and the external world. between agents and the external world. The first one is type tool calls. The The first one is type tool calls. The The first one is type tool calls. The two tool calls uh you don't want, you two tool calls uh you don't want, you two tool calls uh you don't want, you know, you don't want to call tools that know, you don't want to call tools that know, you don't want to call tools that don't exist, etc., etc. and also the don't exist, etc., etc. and also the don't exist, etc., etc. and also the boundary with other agents which is boundary with other agents which is boundary with other agents which is typed events and this is non-negotiable typed events and this is non-negotiable typed events and this is non-negotiable I found like you can get a lot of errors I found like you can get a lot of errors I found like you can get a lot of errors just from this uh I wrote a really long just from this uh I wrote a really long just from this uh I wrote a really long blog post about this uh it's if you blog post about this uh it's if you blog post about this uh it's if you follow the QR code you'll find it and follow the QR code you'll find it and follow the QR code you'll find it and yeah and the result of that is I yeah and the result of that is I yeah and the result of that is I actually deployed it uh within the actually deployed it uh within the actually deployed it uh within the company after I built this and now today company after I built this and now today company after I built this and now today after a month of deploying it we have 20 after a month of deploying it we have 20 after a month of deploying it we have 20 agents on the left that are not just agents on the left that are not just agents on the left that are not just contributed by technical people by the

contributed by technical people by the contributed by technical people by the way which is kind of what markdown way which is kind of what markdown way which is kind of what markdown uh what Maron gives you and then on the uh what Maron gives you and then on the uh what Maron gives you and then on the right is you know we deploy it's called right is you know we deploy it's called right is you know we deploy it's called the internet there's the briefs there's the internet there's the briefs there's the internet there's the briefs there's a bunch of I mean there's a bunch of a bunch of I mean there's a bunch of a bunch of I mean there's a bunch of things as you can as you can see things as you can as you can see things as you can as you can see kind of like a few you know as a kind of like a few you know as a kind of like a few you know as a conclusion a few lessons uh the first conclusion a few lessons uh the first conclusion a few lessons uh the first one is that well executed background one is that well executed background one is that well executed background agents are really magical uh they feel agents are really magical uh they feel agents are really magical uh they feel like this you know robot mower that I like this you know robot mower that I like this you know robot mower that I had at the beginning is I really just had at the beginning is I really just had at the beginning is I really just sit down and I come back and I have this sit down and I come back and I have this sit down and I come back and I have this morning brief that is probably even morning brief that is probably even morning brief that is probably even better than what I would have had just better than what I would have had just better than what I would have had just doing it manually and it just appears in doing it manually and it just appears in doing it manually and it just appears in my inbox every day and processes my you my inbox every day and processes my you my inbox every day and processes my you know random thoughts. Uh the know random thoughts. Uh the know random thoughts. Uh the difficulties that you meet doing this difficulties that you meet doing this difficulties that you meet doing this kind of thing is just good old kind of thing is just good old kind of thing is just good old engineering problems. I mean there's engineering problems. I mean there's engineering problems. I mean there's really nothing new under the sun when it really nothing new under the sun when it really nothing new under the sun when it comes to orchestrating these things. comes to orchestrating these things. comes to orchestrating these things. It's just good old software It's just good old software It's just good old software orchestration. Uh open source models are orchestration. Uh open source models are orchestration. Uh open source models are there. Uh they're good enough. I there. Uh they're good enough. I there. Uh they're good enough. I replaced so I don't have any third party replaced so I don't have any third party replaced so I don't have any third party APIs anymore. Now I just use open source APIs anymore. Now I just use open source APIs anymore. Now I just use open source models and even on my laptop I use a models and even on my laptop I use a models and even on my laptop I use a local model. Uh so it's good enough for local model. Uh so it's good enough for local model. Uh so it's good enough for what I do with it. For coding I don't what I do with it. For coding I don't what I do with it. For coding I don't know but for what I do with this it's know but for what I do with this it's know but for what I do with this it's good enough. The infro category is good enough. The infro category is good enough. The infro category is definitely unsettled. Uh I tried a few definitely unsettled. Uh I tried a few definitely unsettled. Uh I tried a few things before I started building myself.

things before I started building myself. things before I started building myself. And I would advise that today like And I would advise that today like And I would advise that today like definitely start building before you definitely start building before you definitely start building before you buy. Uh so if you're a small company, if buy. Uh so if you're a small company, if buy. Uh so if you're a small company, if you're a tech CEO, it's kind of an you're a tech CEO, it's kind of an you're a tech CEO, it's kind of an advantage because you can just do this advantage because you can just do this advantage because you can just do this without, you know, tasking engineers to without, you know, tasking engineers to without, you know, tasking engineers to do this and get them off track. But I do this and get them off track. But I do this and get them off track. But I would definitely try to build before I would definitely try to build before I would definitely try to build before I buy just to know exactly what I need and buy just to know exactly what I need and buy just to know exactly what I need and you know the limitations of what exist. you know the limitations of what exist. you know the limitations of what exist. Uh also I will say that to people Uh also I will say that to people Uh also I will say that to people building uh frameworks for this is building uh frameworks for this is building uh frameworks for this is please eat your own dog food. Sometimes please eat your own dog food. Sometimes please eat your own dog food. Sometimes it's pretty clear that people are it's pretty clear that people are it's pretty clear that people are building you know agent orchestration building you know agent orchestration building you know agent orchestration frameworks etc but not eating their own frameworks etc but not eating their own frameworks etc but not eating their own dog food. So please do and the other dog food. So please do and the other dog food. So please do and the other thing is I'm really glad I took this two thing is I'm really glad I took this two thing is I'm really glad I took this two weeks off to play with the field because weeks off to play with the field because weeks off to play with the field because that completely changed I mean that that completely changed I mean that that completely changed I mean that changed the trajectory of the company. I changed the trajectory of the company. I changed the trajectory of the company. I know we're an AI company we should be in know we're an AI company we should be in know we're an AI company we should be in it etc but you know business is such it etc but you know business is such it etc but you know business is such that you're always thinking about the that you're always thinking about the that you're always thinking about the next thing the next thing the next thing next thing the next thing the next thing next thing the next thing the next thing and it's the same everywhere. But what and it's the same everywhere. But what and it's the same everywhere. But what I'm urging you to do is to stop and I'm urging you to do is to stop and I'm urging you to do is to stop and actually immerse yourself in this and actually immerse yourself in this and actually immerse yourself in this and try to see how useful it can be for your try to see how useful it can be for your try to see how useful it can be for your company. Uh so you can steal the code.

company. Uh so you can steal the code. company. Uh so you can steal the code. Uh it's not a product that we sell and Uh it's not a product that we sell and Uh it's not a product that we sell and we don't intend to sell this. Uh you can we don't intend to sell this. Uh you can we don't intend to sell this. Uh you can read our blog as well. Uh so I haven't read our blog as well. Uh so I haven't read our blog as well. Uh so I haven't explained uh this yet but I will publish explained uh this yet but I will publish explained uh this yet but I will publish something about it. And thank you for something about it. And thank you for something about it. And thank you for your attention.