The Spatial Harness: Bringing Agents to the Canvas — Max Drake, tldraw

Thank you for coming here to my talk to Thank you for coming here to my talk to watch me talk about uh agents on the watch me talk about uh agents on the watch me talk about uh agents on the canvas. Um the first thing I'm going to canvas. Um the first thing I'm going to canvas. Um the first thing I'm going to do though is uh before I have to record do though is uh before I have to record do though is uh before I have to record my screen. Uh the first thing I'm going my screen. Uh the first thing I'm going my screen. Uh the first thing I'm going to do is I'm going to ask my agent to do to do is I'm going to ask my agent to do to do is I'm going to ask my agent to do something on the canvas. something on the canvas. something on the canvas. And what I'm going to do is say, "Hey, And what I'm going to do is say, "Hey, And what I'm going to do is say, "Hey, uh, my colleague Spencer just emailed me uh, my colleague Spencer just emailed me uh, my colleague Spencer just emailed me a link to a notion document, uh, for a a link to a notion document, uh, for a a link to a notion document, uh, for a really cool demo we could build with the really cool demo we could build with the really cool demo we could build with the TL desktop app. Can you like find that TL desktop app. Can you like find that TL desktop app. Can you like find that document and then can you build it on document and then can you build it on document and then can you build it on the desktop app?" the desktop app?" the desktop app?" Thank you. Okay, so that's going to build uh, and Okay, so that's going to build uh, and then we're going to come back to it then we're going to come back to it then we're going to come back to it later and hopefully it'll work. Um, hi later and hopefully it'll work. Um, hi later and hopefully it'll work. Um, hi everyone, my name is Max Reich. Thanks everyone, my name is Max Reich. Thanks everyone, my name is Max Reich. Thanks so much for coming. Um I work on uh so much for coming. Um I work on uh so much for coming. Um I work on uh agents on the canvas at uh TLA. I'm a agents on the canvas at uh TLA. I'm a agents on the canvas at uh TLA. I'm a product engineer there. Um so first product engineer there. Um so first product engineer there. Um so first things first, am I qualified to be things first, am I qualified to be things first, am I qualified to be giving this talk? I like to think so. giving this talk? I like to think so. giving this talk? I like to think so. I've been doing like agents on the I've been doing like agents on the I've been doing like agents on the canvas stuff since before chat GBT came canvas stuff since before chat GBT came canvas stuff since before chat GBT came out. Um I think it's really cool. I out. Um I think it's really cool. I out. Um I think it's really cool. I think there's like so much UX stuff you think there's like so much UX stuff you think there's like so much UX stuff you can do with when you get LM, you have can do with when you get LM, you have can do with when you get LM, you have them working in space. Uh and I think them working in space. Uh and I think them working in space. Uh and I think it's really interesting. I've been doing it's really interesting. I've been doing it's really interesting. I've been doing it for about as long as you can have it for about as long as you can have it for about as long as you can have been doing it. Um more recently I've been doing it. Um more recently I've been doing it. Um more recently I've been talking about this a lot. Um, been talking about this a lot. Um, been talking about this a lot. Um, here's some proof. Um, and yeah, so I here's some proof. Um, and yeah, so I here's some proof. Um, and yeah, so I work at this company called Teal Draw.

work at this company called Teal Draw. work at this company called Teal Draw. Um, can I get a quick show of hands? Has Um, can I get a quick show of hands? Has Um, can I get a quick show of hands? Has anybody ever heard of or used Teal Draw anybody ever heard of or used Teal Draw anybody ever heard of or used Teal Draw before? before? before? >> Yeah. Okay. Um, awesome. So, yeah, the >> Yeah. Okay. Um, awesome. So, yeah, the >> Yeah. Okay. Um, awesome. So, yeah, the thing that you've probably used if you thing that you've probably used if you thing that you've probably used if you use Teal Draw is, um, this app right use Teal Draw is, um, this app right use Teal Draw is, um, this app right here. So, this is all Teal Draw. Uh, here. So, this is all Teal Draw. Uh, here. So, this is all Teal Draw. Uh, this is a free uh, infinite canvas this is a free uh, infinite canvas this is a free uh, infinite canvas whiteboarding app. Um, you know, we have whiteboarding app. Um, you know, we have whiteboarding app. Um, you know, we have selections and arrows and resizing and, selections and arrows and resizing and, selections and arrows and resizing and, you know, all the things that you need you know, all the things that you need you know, all the things that you need in a in a whiteboard. Um, Traw is also in a in a whiteboard. Um, Traw is also in a in a whiteboard. Um, Traw is also the company uh that makes this app. It's the company uh that makes this app. It's the company uh that makes this app. It's based in London. It's where I work. Um, based in London. It's where I work. Um, based in London. It's where I work. Um, but the last thing that TLAR is, which but the last thing that TLAR is, which but the last thing that TLAR is, which is, I think, in my opinion, the most is, I think, in my opinion, the most is, I think, in my opinion, the most important, is it's the Infinite Canvas important, is it's the Infinite Canvas important, is it's the Infinite Canvas SDKs that that powers this app. Um, and SDKs that that powers this app. Um, and SDKs that that powers this app. Um, and so what that means is that, you know, so what that means is that, you know, so what that means is that, you know, this is kind of the teal draw. The SDK this is kind of the teal draw. The SDK this is kind of the teal draw. The SDK is the engine that powers a lot of is the engine that powers a lot of is the engine that powers a lot of Infinite canvas experiences. Um because Infinite canvas experiences. Um because Infinite canvas experiences. Um because it turns out it's really hard to get it turns out it's really hard to get it turns out it's really hard to get that kind of stuff right. Um and uh so that kind of stuff right. Um and uh so that kind of stuff right. Um and uh so if you ever want to build a Muro if you ever want to build a Muro if you ever want to build a Muro competitor or a slide designer or if competitor or a slide designer or if competitor or a slide designer or if you're like Replet um Replet has their you're like Replet um Replet has their you're like Replet um Replet has their whole new uh agent canvas stuff built on whole new uh agent canvas stuff built on whole new uh agent canvas stuff built on top of TLR. Um, and so the reason we top of TLR. Um, and so the reason we top of TLR. Um, and so the reason we built TL Draw in the first place was built TL Draw in the first place was built TL Draw in the first place was that we were running into this issue or that we were running into this issue or that we were running into this issue or people were running into this issue people were running into this issue people were running into this issue where they had this idea for this like where they had this idea for this like where they had this idea for this like really great killer canvas app. Uh, and really great killer canvas app. Uh, and really great killer canvas app. Uh, and they went to go build it and everybody they went to go build it and everybody they went to go build it and everybody would run into the same problem where would run into the same problem where would run into the same problem where they would run into they would run into they would run into they would have trouble making the they would have trouble making the they would have trouble making the actual canvas part of of the app and actual canvas part of of the app and actual canvas part of of the app and they would, you know, trying to deal they would, you know, trying to deal they would, you know, trying to deal with resizing and selection and, you with resizing and selection and, you with resizing and selection and, you know, all the matrix math. And the issue know, all the matrix math. And the issue know, all the matrix math. And the issue is that they wouldn't be able to build is that they wouldn't be able to build is that they wouldn't be able to build their actual app itself. They would get their actual app itself. They would get their actual app itself. They would get stuck on the canvas. And so we built

stuck on the canvas. And so we built stuck on the canvas. And so we built Teal Draw to kind of be the engine um Teal Draw to kind of be the engine um Teal Draw to kind of be the engine um that could power that could be the that could power that could be the that could power that could be the canvas so that they could focus on the canvas so that they could focus on the canvas so that they could focus on the actual app. Uh when LLM came out, we actual app. Uh when LLM came out, we actual app. Uh when LLM came out, we like a lot of other people saw that this like a lot of other people saw that this like a lot of other people saw that this is going to be this weird new type of is going to be this weird new type of is going to be this weird new type of software. I don't know if anyone, you software. I don't know if anyone, you software. I don't know if anyone, you know, I'm sure a lot of you were know, I'm sure a lot of you were know, I'm sure a lot of you were building in 2022 and it was really building in 2022 and it was really building in 2022 and it was really exciting. Um and a lot of people it was exciting. Um and a lot of people it was exciting. Um and a lot of people it was the exact same thing. People had the the exact same thing. People had the the exact same thing. People had the idea had an idea for this cool app that idea had an idea for this cool app that idea had an idea for this cool app that would, you know, involve LM on the would, you know, involve LM on the would, you know, involve LM on the canvas having them manipulating things canvas having them manipulating things canvas having them manipulating things in space. but then they would try to in space. but then they would try to in space. but then they would try to build it and they'd get stuck. Um there build it and they'd get stuck. Um there build it and they'd get stuck. Um there were no best practices. People didn't were no best practices. People didn't were no best practices. People didn't really know how to do it. And so at TL really know how to do it. And so at TL really know how to do it. And so at TL Draw we uh realized that we need to make Draw we uh realized that we need to make Draw we uh realized that we need to make it easy for people to build uh with it easy for people to build uh with it easy for people to build uh with agents with LLMs on the canvas. Um and agents with LLMs on the canvas. Um and agents with LLMs on the canvas. Um and also so Teal Draw the SDK as well as the also so Teal Draw the SDK as well as the also so Teal Draw the SDK as well as the app has multiplayer built in with like app has multiplayer built in with like app has multiplayer built in with like live sync. It's really nice. There's live sync. It's really nice. There's live sync. It's really nice. There's cursors. There's you can see your cursors. There's you can see your cursors. There's you can see your collaborators cursors and selections and collaborators cursors and selections and collaborators cursors and selections and viewports. And I think all of the things viewports. And I think all of the things viewports. And I think all of the things that make just the canvas in general a that make just the canvas in general a that make just the canvas in general a really great uh place for interacting really great uh place for interacting really great uh place for interacting with and collaborating with your with and collaborating with your with and collaborating with your colleagues also make it a really great colleagues also make it a really great colleagues also make it a really great place for interacting uh and place for interacting uh and place for interacting uh and collaborating with agents. Uh and I hope collaborating with agents. Uh and I hope collaborating with agents. Uh and I hope I'm going to be able to show you guys I'm going to be able to show you guys I'm going to be able to show you guys some of that um in the demos that come some of that um in the demos that come some of that um in the demos that come up. Um so before we talk about agents on up. Um so before we talk about agents on up. Um so before we talk about agents on the canvas, uh really quickly I want to the canvas, uh really quickly I want to the canvas, uh really quickly I want to talk about agents not on the canvas. Um talk about agents not on the canvas. Um talk about agents not on the canvas. Um I'm sure you guys have all used an app I'm sure you guys have all used an app I'm sure you guys have all used an app that looks like this, you know, claude that looks like this, you know, claude that looks like this, you know, claude code. And I'm going to really code. And I'm going to really code. And I'm going to really oversimplify here, but basically, uh, oversimplify here, but basically, uh, oversimplify here, but basically, uh, part of the reason why these apps are so part of the reason why these apps are so part of the reason why these apps are so good and why they work is because good and why they work is because good and why they work is because they're, you know, the medium in which they're, you know, the medium in which they're, you know, the medium in which they're working, writing code is they're working, writing code is they're working, writing code is essentially the medium in which they essentially the medium in which they essentially the medium in which they were trained. You know, it's it's text

were trained. You know, it's it's text were trained. You know, it's it's text in, text out. That's how they were in, text out. That's how they were in, text out. That's how they were trained. Um, and when we work with them, trained. Um, and when we work with them, trained. Um, and when we work with them, we give them a prompt and they write we give them a prompt and they write we give them a prompt and they write code. It's text in, text out. Uh, you code. It's text in, text out. Uh, you code. It's text in, text out. Uh, you know, again, oversimplifying, but that's know, again, oversimplifying, but that's know, again, oversimplifying, but that's essentially how they work. I don't know essentially how they work. I don't know essentially how they work. I don't know if you guys have ever, you know, tried if you guys have ever, you know, tried if you guys have ever, you know, tried to get your agents to do like UI stuff to get your agents to do like UI stuff to get your agents to do like UI stuff and try to get them align to align and try to get them align to align and try to get them align to align something. found that they could not do something. found that they could not do something. found that they could not do that whatsoever. Um because it turns out that whatsoever. Um because it turns out that whatsoever. Um because it turns out agents are really really bad at working agents are really really bad at working agents are really really bad at working in 2D space and understanding 2D space in 2D space and understanding 2D space in 2D space and understanding 2D space and actually requires like a lot of and actually requires like a lot of and actually requires like a lot of engineering work to get them to uh do engineering work to get them to uh do engineering work to get them to uh do it. And that's kind of the project that it. And that's kind of the project that it. And that's kind of the project that we've been embarking on at Teal Draw we've been embarking on at Teal Draw we've been embarking on at Teal Draw recently. Um and so the first thing we recently. Um and so the first thing we recently. Um and so the first thing we had to do, this is an older project, but had to do, this is an older project, but had to do, this is an older project, but the first thing we had to do is get them the first thing we had to do is get them the first thing we had to do is get them to teach them teach the agents or at to teach them teach the agents or at to teach them teach the agents or at this point not agents LLM to understand this point not agents LLM to understand this point not agents LLM to understand the canvas and understand kind of what the canvas and understand kind of what the canvas and understand kind of what they're even looking at. Um, so we had they're even looking at. Um, so we had they're even looking at. Um, so we had this project called teach um where um we this project called teach um where um we this project called teach um where um we tal So I'm going to I'm going to prompt tal So I'm going to I'm going to prompt tal So I'm going to I'm going to prompt this really quick. I'm going to say, this really quick. I'm going to say, this really quick. I'm going to say, "Hey, make the mouse blow out the "Hey, make the mouse blow out the "Hey, make the mouse blow out the candle." candle." candle." Yeah. So that's going to take a second. Yeah. So that's going to take a second. Yeah. So that's going to take a second. This is this is an older uh older This is this is an older uh older This is this is an older uh older project, but basically what we had to do project, but basically what we had to do project, but basically what we had to do is we had to kind of like teach the LMS is we had to kind of like teach the LMS is we had to kind of like teach the LMS how to take the like the screenshot that how to take the like the screenshot that how to take the like the screenshot that we give it and the JSON and all of the we give it and the JSON and all of the we give it and the JSON and all of the other information about the canvas and other information about the canvas and other information about the canvas and Oh yeah, there we Okay. So yeah, that's Oh yeah, there we Okay. So yeah, that's Oh yeah, there we Okay. So yeah, that's some that's some wind. Uh it's some that's some wind. Uh it's some that's some wind. Uh it's is it sometimes it gives us smoke as is it sometimes it gives us smoke as is it sometimes it gives us smoke as well. Yeah, and we got a little smoke as well. Yeah, and we got a little smoke as well. Yeah, and we got a little smoke as well. So So we basically had to take it well. So So we basically had to take it well. So So we basically had to take it how to like and I want to be very clear how to like and I want to be very clear how to like and I want to be very clear this is not um this is not like a this is not um this is not like a this is not um this is not like a special mouse shape. These are just like special mouse shape. These are just like special mouse shape. These are just like uh you know the these are just shapes uh

uh you know the these are just shapes uh uh you know the these are just shapes uh on the canvas. This is um and so the on the canvas. This is um and so the on the canvas. This is um and so the work behind this it's a singleshot work behind this it's a singleshot work behind this it's a singleshot prompt but we basically we tell the prompt but we basically we tell the prompt but we basically we tell the agent how to interpret uh both via agent how to interpret uh both via agent how to interpret uh both via screenshots and via the data um what is screenshots and via the data um what is screenshots and via the data um what is actually on the canvas like what it's actually on the canvas like what it's actually on the canvas like what it's looking at um which is actually you know looking at um which is actually you know looking at um which is actually you know it's not a trivial problem and then also it's not a trivial problem and then also it's not a trivial problem and then also how to we teach it how to actually act how to we teach it how to actually act how to we teach it how to actually act on the canvas and to understand how the on the canvas and to understand how the on the canvas and to understand how the actions that it produces will affect the actions that it produces will affect the actions that it produces will affect the canvas. So, you know, it got it, you canvas. So, you know, it got it, you canvas. So, you know, it got it, you know, it made the it made the smoke, it know, it made the it made the smoke, it know, it made the it made the smoke, it made the it made the wind, it got the made the it made the wind, it got the made the it made the wind, it got the positions right, and it understood what positions right, and it understood what positions right, and it understood what it was doing. Um, so we we got this we it was doing. Um, so we we got this we it was doing. Um, so we we got this we kind of figured out how the um like kind kind of figured out how the um like kind kind of figured out how the um like kind of we got we taught it what the canvas of we got we taught it what the canvas of we got we taught it what the canvas is. Uh, but this was like a single shot, is. Uh, but this was like a single shot, is. Uh, but this was like a single shot, single prompt kind of thing. Um, and so single prompt kind of thing. Um, and so single prompt kind of thing. Um, and so the next thing we built is the uh teal the next thing we built is the uh teal the next thing we built is the uh teal draw agent starter kit. Uh, which draw agent starter kit. Uh, which draw agent starter kit. Uh, which basically turns that and wraps in a basically turns that and wraps in a basically turns that and wraps in a harness that lets an agent work harness that lets an agent work harness that lets an agent work agentically on the canvas. Um, the code agentically on the canvas. Um, the code agentically on the canvas. Um, the code is also MIT licensed. You can find it on is also MIT licensed. You can find it on is also MIT licensed. You can find it on um that you can find on the website. So, um that you can find on the website. So, um that you can find on the website. So, here's a little here's a little cat. I'm here's a little here's a little cat. I'm here's a little here's a little cat. I'm going to make this a little bigger. Um going to make this a little bigger. Um going to make this a little bigger. Um but what I'm going to say is, "Hey, so but what I'm going to say is, "Hey, so but what I'm going to say is, "Hey, so somewhere else on the canvas uh there somewhere else on the canvas uh there somewhere else on the canvas uh there are uh some friends for the cat. Can you are uh some friends for the cat. Can you are uh some friends for the cat. Can you please uh bring one of them over to the please uh bring one of them over to the please uh bring one of them over to the cat? Um her favorite color is red."

cat? Um her favorite color is red." cat? Um her favorite color is red." And so I'm going to zoom out. I'm going And so I'm going to zoom out. I'm going And so I'm going to zoom out. I'm going to show you guys what's actually going to show you guys what's actually going to show you guys what's actually going on. So you can see the the view of the on. So you can see the the view of the on. So you can see the the view of the agent. There's some there's some agent. There's some there's some agent. There's some there's some potential friends over here. And um you potential friends over here. And um you potential friends over here. And um you know if you read the All right. So and know if you read the All right. So and know if you read the All right. So and basically what's going on is that the basically what's going on is that the basically what's going on is that the agent has kind of like we've given it a agent has kind of like we've given it a agent has kind of like we've given it a prompt and uh using the information it prompt and uh using the information it prompt and uh using the information it has about the canvas. It's uh going to has about the canvas. It's uh going to has about the canvas. It's uh going to kind of like make some goals for itself. kind of like make some goals for itself. kind of like make some goals for itself. You can see there's some to-dos in the You can see there's some to-dos in the You can see there's some to-dos in the corner here. It's it changed its view in corner here. It's it changed its view in corner here. It's it changed its view in order to see um what was you know the order to see um what was you know the order to see um what was you know the other stuff that was on the canvas. same other stuff that was on the canvas. same other stuff that was on the canvas. same way that if you ask a coding agent um way that if you ask a coding agent um way that if you ask a coding agent um you know you ask it you know where where you know you ask it you know where where you know you ask it you know where where do we define this thing in the codebase do we define this thing in the codebase do we define this thing in the codebase it can go and it can search it can find it can go and it can search it can find it can go and it can search it can find it. So this is kind of like turning that it. So this is kind of like turning that it. So this is kind of like turning that singleshot uh prompting experience into singleshot uh prompting experience into singleshot uh prompting experience into this kind of like agentic thing that you this kind of like agentic thing that you this kind of like agentic thing that you can uh have you know it can autonomously can uh have you know it can autonomously can uh have you know it can autonomously set goals and and work towards them. The set goals and and work towards them. The set goals and and work towards them. The uh next thing we did um we did this this uh next thing we did um we did this this uh next thing we did um we did this this uh project called fairies and so we uh project called fairies and so we uh project called fairies and so we basically we had this agent experience basically we had this agent experience basically we had this agent experience but we realized that you know teal draw but we realized that you know teal draw but we realized that you know teal draw and you know the canvas in general is so and you know the canvas in general is so and you know the canvas in general is so collaborative it's so multiplayer and we collaborative it's so multiplayer and we collaborative it's so multiplayer and we wanted to basically we wanted people to wanted to basically we wanted people to wanted to basically we wanted people to be able to work together with their be able to work together with their be able to work together with their agents and we also wanted the agents to agents and we also wanted the agents to agents and we also wanted the agents to be able to work together be able to work together be able to work together so this is this is a ferry um there's so this is this is a ferry um there's so this is this is a ferry um there's also you guys want to scan this QR code also you guys want to scan this QR code also you guys want to scan this QR code you can actually this is multiplayer you you can actually this is multiplayer you you can actually this is multiplayer you can join if you want you it requires a can join if you want you it requires a can join if you want you it requires a Gmail signup But you don't need to pay Gmail signup But you don't need to pay Gmail signup But you don't need to pay for tokens. Um, this is this is what the for tokens. Um, this is this is what the for tokens. Um, this is this is what the link is. So, basically, this is a this link is. So, basically, this is a this link is. So, basically, this is a this is a fairy. Um, this fair's name is is is a fairy. Um, this fair's name is is is a fairy. Um, this fair's name is is Joan. They don't like being they don't

Joan. They don't like being they don't Joan. They don't like being they don't like being grabbed. Um, you can you can like being grabbed. Um, you can you can like being grabbed. Um, you can you can throw them around. You know, we added a throw them around. You know, we added a throw them around. You know, we added a lot of really important stuff. You can lot of really important stuff. You can lot of really important stuff. You can um you can you can change its hat. Um, um you can you can change its hat. Um, um you can you can change its hat. Um, you can change the the color and and you can change the the color and and you can change the the color and and this seems silly, but it's actually this seems silly, but it's actually this seems silly, but it's actually really important. uh and I'll talk about really important. uh and I'll talk about really important. uh and I'll talk about this a little bit more later, but this a little bit more later, but this a little bit more later, but actually understanding uh when you get a actually understanding uh when you get a actually understanding uh when you get a high level view of when you see your high level view of when you see your high level view of when you see your agents working on the canvas, it's agents working on the canvas, it's agents working on the canvas, it's important to know which one is which. important to know which one is which. important to know which one is which. And so differentiating them is actually And so differentiating them is actually And so differentiating them is actually important, which is why, of course, we important, which is why, of course, we important, which is why, of course, we added the leg slider. Um added the leg slider. Um added the leg slider. Um but so, you know, I can say like, you but so, you know, I can say like, you but so, you know, I can say like, you know, I can I can I can say hey to it. know, I can I can I can say hey to it. know, I can I can I can say hey to it. Um Um Um and I can say, you know, something like and I can say, you know, something like and I can say, you know, something like draw a cat. Uh and I can have it work. draw a cat. Uh and I can have it work. draw a cat. Uh and I can have it work. But the most important thing here is But the most important thing here is But the most important thing here is that fairies have friends, right? and that fairies have friends, right? and that fairies have friends, right? and they can here we go. Uh they can here we go. Uh they can here we go. Uh >> and they can fairies can work together. >> and they can fairies can work together. >> and they can fairies can work together. And so we kind of designed this like And so we kind of designed this like And so we kind of designed this like multi- aent collaboration system that multi- aent collaboration system that multi- aent collaboration system that works on the canvas. Um and I'm going to works on the canvas. Um and I'm going to works on the canvas. Um and I'm going to actually I'm going to go to actually I'm going to go to actually I'm going to go to I think one of my I think one of my I think one of my Yeah, I think so. My my colleagues Yeah, I think so. My my colleagues Yeah, I think so. My my colleagues agents are here working uh making this agents are here working uh making this agents are here working uh making this this really great scene. Um I'm going to this really great scene. Um I'm going to this really great scene. Um I'm going to bring mine over um summon bring mine over um summon bring mine over um summon and I'm going to give them a slightly and I'm going to give them a slightly and I'm going to give them a slightly different prompt. So, I'm gonna select different prompt. So, I'm gonna select different prompt. So, I'm gonna select them all. And now I have a group chat of them all. And now I have a group chat of them all. And now I have a group chat of of the agents, right? And I'm going to of the agents, right? And I'm going to of the agents, right? And I'm going to say, "Hey, I have a board meeting coming say, "Hey, I have a board meeting coming say, "Hey, I have a board meeting coming up in like 10 minutes, and I don't have up in like 10 minutes, and I don't have up in like 10 minutes, and I don't have any of my my figures. Can you draw up any of my my figures. Can you draw up any of my my figures. Can you draw up like a little memo for all of my uh like a little memo for all of my uh like a little memo for all of my uh financial data for fiscal year 2025?

financial data for fiscal year 2025? financial data for fiscal year 2025? Thank you." Thank you." Thank you." Okay. So, what what's going to happen Okay. So, what what's going to happen Okay. So, what what's going to happen there basically is this kind of like there basically is this kind of like there basically is this kind of like creates this uh multi- aent, you know, creates this uh multi- aent, you know, creates this uh multi- aent, you know, coordination thing. We have uh one of coordination thing. We have uh one of coordination thing. We have uh one of the fairies is writing uh writing out a the fairies is writing uh writing out a the fairies is writing uh writing out a plan. You can see it. Um, and again, the plan. You can see it. Um, and again, the plan. You can see it. Um, and again, the animations are kind of cute and funny, animations are kind of cute and funny, animations are kind of cute and funny, but it's actually really important. I but it's actually really important. I but it's actually really important. I don't have to read a chat or go through, don't have to read a chat or go through, don't have to read a chat or go through, you know, imagine if I have 10 agents you know, imagine if I have 10 agents you know, imagine if I have 10 agents working. I don't have to read a chat in working. I don't have to read a chat in working. I don't have to read a chat in order to know what's actually going on. order to know what's actually going on. order to know what's actually going on. I can look at the state of the agents. I can look at the state of the agents. I can look at the state of the agents. Uh, and I can actually, you know, I can Uh, and I can actually, you know, I can Uh, and I can actually, you know, I can can see what's happening. So, we we have can see what's happening. So, we we have can see what's happening. So, we we have a task here that's been defined. Um, a task here that's been defined. Um, a task here that's been defined. Um, it seems like, you know, the the ferry it seems like, you know, the the ferry it seems like, you know, the the ferry is is uh waiting for that to to finish. is is uh waiting for that to to finish. is is uh waiting for that to to finish. Yeah. So, that one's one's board. That Yeah. So, that one's one's board. That Yeah. So, that one's one's board. That one's waiting. So, this is the one's waiting. So, this is the one's waiting. So, this is the orchestrator ferry. What what it's done orchestrator ferry. What what it's done orchestrator ferry. What what it's done is it's assigned the it's assigned the is it's assigned the it's assigned the is it's assigned the it's assigned the task. Uh, and now it's waiting for the task. Uh, and now it's waiting for the task. Uh, and now it's waiting for the other ones to start and finish it. Um, other ones to start and finish it. Um, other ones to start and finish it. Um, and it's going to get notified. It's and it's going to get notified. It's and it's going to get notified. It's going to get prompted in order in order going to get prompted in order in order going to get prompted in order in order to review. Um, it seems like forever to review. Um, it seems like forever to review. Um, it seems like forever reason's not working, but thank Oh, reason's not working, but thank Oh, reason's not working, but thank Oh, never mind. So, yeah, we have one we never mind. So, yeah, we have one we never mind. So, yeah, we have one we have this one. So yeah, we have one have this one. So yeah, we have one have this one. So yeah, we have one fairy who made the made the task, one fairy who made the made the task, one fairy who made the made the task, one fairy who's working on it. And so this fairy who's working on it. And so this fairy who's working on it. And so this is this kind of um you know, multi- aent is this kind of um you know, multi- aent is this kind of um you know, multi- aent coordination system on the canvas. I coordination system on the canvas. I coordination system on the canvas. I have, you know, you can see my colleague have, you know, you can see my colleague have, you know, you can see my colleague has his agents over here. Uh they're has his agents over here. Uh they're has his agents over here. Uh they're they're working as well. Um and so you they're working as well. Um and so you they're working as well. Um and so you can kind of collaborate with people and can kind of collaborate with people and can kind of collaborate with people and with agents in this uh environment. And with agents in this uh environment. And with agents in this uh environment. And I don't know, I think that's really I don't know, I think that's really I don't know, I think that's really cool.

cool. cool. the the the so the the next thing. So the problem so the the next thing. So the problem so the the next thing. So the problem with fairies is that um they're kind of with fairies is that um they're kind of with fairies is that um they're kind of trapped in the canvas and all the stuff trapped in the canvas and all the stuff trapped in the canvas and all the stuff you've seen before. This requires if you you've seen before. This requires if you you've seen before. This requires if you want to build something like this, this want to build something like this, this want to build something like this, this requires like the you to like opt in and requires like the you to like opt in and requires like the you to like opt in and have your entire harness be a like have your entire harness be a like have your entire harness be a like canvas harness. Um and the downside of canvas harness. Um and the downside of canvas harness. Um and the downside of that is that it makes it really hard to that is that it makes it really hard to that is that it makes it really hard to have any of this work with stuff like have any of this work with stuff like have any of this work with stuff like outside in the real world. The fairies outside in the real world. The fairies outside in the real world. The fairies are the fairies are trapped in the are the fairies are trapped in the are the fairies are trapped in the canvas. Um, and canvas. Um, and canvas. Um, and so I I I built this experiment. We had a so I I I built this experiment. We had a so I I I built this experiment. We had a little hackathon uh internally. Um, but little hackathon uh internally. Um, but little hackathon uh internally. Um, but first as a quick motivation for that, first as a quick motivation for that, first as a quick motivation for that, um, at Teal Draw whenever we have an um, at Teal Draw whenever we have an um, at Teal Draw whenever we have an whenever we're getting closer to a whenever we're getting closer to a whenever we're getting closer to a launch, we like abandon all of our task launch, we like abandon all of our task launch, we like abandon all of our task tracking software and we make just one tracking software and we make just one tracking software and we make just one massive dependency graph of how like so massive dependency graph of how like so massive dependency graph of how like so this is what an actual this is a real this is what an actual this is a real this is what an actual this is a real thing from when we launched fairies thing from when we launched fairies thing from when we launched fairies actually. Um, and so this is what it actually. Um, and so this is what it actually. Um, and so this is what it looks like when we're like really like looks like when we're like really like looks like when we're like really like when is hitting the fan at at when is hitting the fan at at when is hitting the fan at at tealraw when we're launching something. tealraw when we're launching something. tealraw when we're launching something. Um, and I really like this interface Um, and I really like this interface Um, and I really like this interface because it it kind of lets you This is because it it kind of lets you This is because it it kind of lets you This is not like a special app. This is still not like a special app. This is still not like a special app. This is still just tealraw.com. You can, you know, just tealraw.com. You can, you know, just tealraw.com. You can, you know, move your shapes around and things like move your shapes around and things like move your shapes around and things like that. Um, but I really like this because that. Um, but I really like this because that. Um, but I really like this because it both it lets you see like what it both it lets you see like what it both it lets you see like what depends on what. It lets you know what's depends on what. It lets you know what's depends on what. It lets you know what's coming next. It lets you get a highle coming next. It lets you get a highle coming next. It lets you get a highle overview. You know, these are all green overview. You know, these are all green overview. You know, these are all green because we finished them, but you know, because we finished them, but you know, because we finished them, but you know, you can imagine during the project, some you can imagine during the project, some you can imagine during the project, some of them are in process. And I really of them are in process. And I really of them are in process. And I really want I really wanted something like want I really wanted something like want I really wanted something like this. Um, but something that I could this. Um, but something that I could this. Um, but something that I could actually that could actually do the work actually that could actually do the work actually that could actually do the work itself. And so I um prototyped this

itself. And so I um prototyped this itself. And so I um prototyped this thing. It's called the tech tree app. thing. It's called the tech tree app. thing. It's called the tech tree app. And basically, it's similar to this. And basically, it's similar to this. And basically, it's similar to this. It's a dependency graph, but each of It's a dependency graph, but each of It's a dependency graph, but each of these tasks um is a a coding agent that these tasks um is a a coding agent that these tasks um is a a coding agent that you can kick off and you can have your you can kick off and you can have your you can kick off and you can have your agent kind of like be running and doing agent kind of like be running and doing agent kind of like be running and doing them autonomously. Um the project itself them autonomously. Um the project itself them autonomously. Um the project itself that's working on it's this little um that's working on it's this little um that's working on it's this little um this is just kind of like a demo app. Um this is just kind of like a demo app. Um this is just kind of like a demo app. Um but this is Can I but this is Can I but this is Can I Yeah. So this is this is a little Yeah. So this is this is a little Yeah. So this is this is a little fun, you know, multimodal input thing. I fun, you know, multimodal input thing. I fun, you know, multimodal input thing. I haven't written any of the code for haven't written any of the code for haven't written any of the code for this. This is um this is all written by this. This is um this is all written by this. This is um this is all written by agents. But I can manage all of the work agents. But I can manage all of the work agents. But I can manage all of the work is being done in this desktop app uh or is being done in this desktop app uh or is being done in this desktop app uh or in this app here. And so I can do in this app here. And so I can do in this app here. And so I can do something like I can see this one has something like I can see this one has something like I can see this one has has finished uh building some gesture has finished uh building some gesture has finished uh building some gesture controls for the canvas. So I can open controls for the canvas. So I can open controls for the canvas. So I can open the PR uh and unfortunately sorry the PR uh and unfortunately sorry the PR uh and unfortunately sorry Jeffrey I am just going to merge this. Jeffrey I am just going to merge this. Jeffrey I am just going to merge this. I'm not going to uh have it be explained I'm not going to uh have it be explained I'm not going to uh have it be explained to me. But so this is [snorts] to me. But so this is [snorts] to me. But so this is [snorts] and so yeah great. Awesome. It looks and so yeah great. Awesome. It looks and so yeah great. Awesome. It looks good. Um and then you know eventually good. Um and then you know eventually good. Um and then you know eventually this is going to get marked uh as as this is going to get marked uh as as this is going to get marked uh as as complete and this is also multiplayer complete and this is also multiplayer complete and this is also multiplayer which is really cool and you can have which is really cool and you can have which is really cool and you can have people working together. Yeah. So that's people working together. Yeah. So that's people working together. Yeah. So that's finished. Um and I you can also prompt finished. Um and I you can also prompt finished. Um and I you can also prompt from like inside the the app you can from like inside the the app you can from like inside the the app you can draw and have a prompt. So I can draw and have a prompt. So I can draw and have a prompt. So I can basically I can just take all of this uh basically I can just take all of this uh basically I can just take all of this uh and I can draw a little like so this is and I can draw a little like so this is and I can draw a little like so this is my prompt and I can wrap it in a task um my prompt and I can wrap it in a task um my prompt and I can wrap it in a task um and I can you know call it facial and I can you know call it facial and I can you know call it facial animation canvas control and then I can animation canvas control and then I can animation canvas control and then I can assign that to cloud and I can just hit assign that to cloud and I can just hit assign that to cloud and I can just hit run and so now that's working as well run and so now that's working as well run and so now that's working as well and so this is kind of like you know

and so this is kind of like you know and so this is kind of like you know this is kind of something similar to this is kind of something similar to this is kind of something similar to conductor or openai symfony where you're conductor or openai symfony where you're conductor or openai symfony where you're using a kind of like one abstracted using a kind of like one abstracted using a kind of like one abstracted interface above uh what the actual in interface above uh what the actual in interface above uh what the actual in order to like manage your your multi- order to like manage your your multi- order to like manage your your multi- aent uh coordination and things like aent uh coordination and things like aent uh coordination and things like that And the thing I like about this that And the thing I like about this that And the thing I like about this also is that because this is also is that because this is also is that because this is multiplayer, one of my colleagues can multiplayer, one of my colleagues can multiplayer, one of my colleagues can come and join and add tasks and uh edit come and join and add tasks and uh edit come and join and add tasks and uh edit things and see the work that's been things and see the work that's been things and see the work that's been going on. So, it's much more going on. So, it's much more going on. So, it's much more collaborative than um like your own collaborative than um like your own collaborative than um like your own instance of something. Um here's the instance of something. Um here's the instance of something. Um here's the moment of truth. Let's see if that demo moment of truth. Let's see if that demo moment of truth. Let's see if that demo that I had it build in the beginning that I had it build in the beginning that I had it build in the beginning worked. worked. worked. Um Um Um all right, it's it hasn't built the all right, it's it hasn't built the all right, it's it hasn't built the fluid simulation yet. Uh it's been fluid simulation yet. Uh it's been fluid simulation yet. Uh it's been working for 13 minutes. Um working for 13 minutes. Um working for 13 minutes. Um that's actually fine. So basically this that's actually fine. So basically this that's actually fine. So basically this is the TL Draw desktop app. Um something is the TL Draw desktop app. Um something is the TL Draw desktop app. Um something that's really cool here is that we have that's really cool here is that we have that's really cool here is that we have uh so this is running locally. It's uh so this is running locally. It's uh so this is running locally. It's working on files. We'll see if it working on files. We'll see if it working on files. We'll see if it finishes. We'll we'll let this run. Um finishes. We'll we'll let this run. Um finishes. We'll we'll let this run. Um but basically what this does is this uh but basically what this does is this uh but basically what this does is this uh this basically exposes the editor this basically exposes the editor this basically exposes the editor instance of the TLR app that's running instance of the TLR app that's running instance of the TLR app that's running here. And it has a server that lets any here. And it has a server that lets any here. And it has a server that lets any agent uh for example my cloud code uh agent uh for example my cloud code uh agent uh for example my cloud code uh write just plain JavaScript against the write just plain JavaScript against the write just plain JavaScript against the against the editor. and and it it against the editor. and and it it against the editor. and and it it basically it you know it's it's code basically it you know it's it's code basically it you know it's it's code mode if you've ever used code mode um mode if you've ever used code mode um mode if you've ever used code mode um but you can basically turn your teal but you can basically turn your teal but you can basically turn your teal desktop app into um a like scripting desktop app into um a like scripting desktop app into um a like scripting environment and the one of my colleagues environment and the one of my colleagues environment and the one of my colleagues actually is u I'm going to this is this actually is u I'm going to this is this actually is u I'm going to this is this is the kind of off the rails bit of the is the kind of off the rails bit of the is the kind of off the rails bit of the canvas here um or of the of the talk. So

canvas here um or of the of the talk. So canvas here um or of the of the talk. So here's something my colleague made uh here's something my colleague made uh here's something my colleague made uh using the same thing. So this is he has using the same thing. So this is he has using the same thing. So this is he has the teal draw desktop app in the corner the teal draw desktop app in the corner the teal draw desktop app in the corner here and he's using it as his uh window here and he's using it as his uh window here and he's using it as his uh window manager. Um and what he did the way he manager. Um and what he did the way he manager. Um and what he did the way he did this was he just told claude code to did this was he just told claude code to did this was he just told claude code to because cloud code has access to your because cloud code has access to your because cloud code has access to your actual computer. It's not locked into actual computer. It's not locked into actual computer. It's not locked into the canvas. Uh it basically you know it the canvas. Uh it basically you know it the canvas. Uh it basically you know it it made some rectangles and it uh it made some rectangles and it uh it made some rectangles and it uh probably wrote some Apple script or probably wrote some Apple script or probably wrote some Apple script or something to actually re uh you know something to actually re uh you know something to actually re uh you know move the things around. And so you can move the things around. And so you can move the things around. And so you can kind of make all of these uh like kind of make all of these uh like kind of make all of these uh like ephemeral UIs and have them actually be ephemeral UIs and have them actually be ephemeral UIs and have them actually be doing things in the real world. Um, doing things in the real world. Um, doing things in the real world. Um, another really cool one that he did was, another really cool one that he did was, another really cool one that he did was, um, if this loads, it's Pong on the um, if this loads, it's Pong on the um, if this loads, it's Pong on the desktop. Uh, desktop. Uh, desktop. Uh, let's hit it with a little refresh there let's hit it with a little refresh there let's hit it with a little refresh there and see if it works. Yeah. So, this is and see if it works. Yeah. So, this is and see if it works. Yeah. So, this is he's got in the corner here, you know, he's got in the corner here, you know, he's got in the corner here, you know, you you have uh you have TLR running. you you have uh you have TLR running. you you have uh you have TLR running. This is the desktop app and it's using This is the desktop app and it's using This is the desktop app and it's using the Windows in order to uh play Pong. the Windows in order to uh play Pong. the Windows in order to uh play Pong. And so again, like kind of crazy, but And so again, like kind of crazy, but And so again, like kind of crazy, but there's uh you know, maybe it seems a there's uh you know, maybe it seems a there's uh you know, maybe it seems a little silly, but little silly, but little silly, but let's see if this this worked. Oh, it's let's see if this this worked. Oh, it's let's see if this this worked. Oh, it's still working. Man, it was usually much still working. Man, it was usually much still working. Man, it was usually much faster. Um, but I I think this stuff is faster. Um, but I I think this stuff is faster. Um, but I I think this stuff is so cool because this lets you kind of, so cool because this lets you kind of, so cool because this lets you kind of, you know, do all of the weird kind of you know, do all of the weird kind of you know, do all of the weird kind of like spatial interfaces that you can do like spatial interfaces that you can do like spatial interfaces that you can do uh on the canvas. You get all of like uh on the canvas. You get all of like uh on the canvas. You get all of like the primitives of the canvas. Um, but the primitives of the canvas. Um, but the primitives of the canvas. Um, but you can like you can have your agents you can like you can have your agents you can like you can have your agents working kind of like in the real world.

working kind of like in the real world. working kind of like in the real world. uh it has access to real data. If I uh it has access to real data. If I uh it has access to real data. If I scroll up, I'll show that um scroll up, I'll show that um scroll up, I'll show that um if uh you know this is my cloud code and if uh you know this is my cloud code and if uh you know this is my cloud code and it found it it got the Gmail, it got the it found it it got the Gmail, it got the it found it it got the Gmail, it got the notion doc, it found the spec and it's notion doc, it found the spec and it's notion doc, it found the spec and it's going to to implement it. Um going to to implement it. Um going to to implement it. Um but yeah, so to sum up, I think that but yeah, so to sum up, I think that but yeah, so to sum up, I think that agents working on the canvas is so cool agents working on the canvas is so cool agents working on the canvas is so cool and I think that there's like so much we and I think that there's like so much we and I think that there's like so much we can do if we use like the agent the can do if we use like the agent the can do if we use like the agent the canvas as a a place to work with agents. canvas as a a place to work with agents. canvas as a a place to work with agents. And I think the place part of it is And I think the place part of it is And I think the place part of it is really important because you know when really important because you know when really important because you know when we do you know with remote work we do you know with remote work we do you know with remote work collaboration we do a lot of stuff collaboration we do a lot of stuff collaboration we do a lot of stuff online with each other and we online with each other and we online with each other and we collaborate with people on the canvas collaborate with people on the canvas collaborate with people on the canvas and I think that the yeah the canvas can and I think that the yeah the canvas can and I think that the yeah the canvas can be a place where we uh collaborate with be a place where we uh collaborate with be a place where we uh collaborate with agents and I'm I'm I'm I'm vamping agents and I'm I'm I'm I'm vamping agents and I'm I'm I'm I'm vamping because I'm trying to see if this is because I'm trying to see if this is because I'm trying to see if this is finished but I don't think it's going to finished but I don't think it's going to finished but I don't think it's going to finish. Um, but thank you so much