Training Taste — Thais Castello Branco, Taste Labs

Okay, amazing. It's great to meet Okay, amazing. It's great to meet everyone. I'm Tais. I'm the founder of everyone. I'm Tais. I'm the founder of everyone. I'm Tais. I'm the founder of Taste Labs. Uh for those of you who Taste Labs. Uh for those of you who Taste Labs. Uh for those of you who don't know us, we came out of Stealth a don't know us, we came out of Stealth a don't know us, we came out of Stealth a few weeks ago and our whole mission is few weeks ago and our whole mission is few weeks ago and our whole mission is basically how do we end AI slop? I It's basically how do we end AI slop? I It's basically how do we end AI slop? I It's my personal enemy. Um, and so we really my personal enemy. Um, and so we really my personal enemy. Um, and so we really believe that to solve this problem of believe that to solve this problem of believe that to solve this problem of slop, we have to like decode subjective slop, we have to like decode subjective slop, we have to like decode subjective domains. Uh, there's been so much effort domains. Uh, there's been so much effort domains. Uh, there's been so much effort being put into getting models and agents being put into getting models and agents being put into getting models and agents amazing at things like coding and math. amazing at things like coding and math. amazing at things like coding and math. Uh, and it's time that we put all that Uh, and it's time that we put all that Uh, and it's time that we put all that same effort into making them great at same effort into making them great at same effort into making them great at things like design uh, and writing. And things like design uh, and writing. And things like design uh, and writing. And so design is this first pillar that so design is this first pillar that so design is this first pillar that we're starting with. And it's been it's we're starting with. And it's been it's we're starting with. And it's been it's been incredibly exciting. Um, we work been incredibly exciting. Um, we work been incredibly exciting. Um, we work primarily in two ways. So we work a lot primarily in two ways. So we work a lot primarily in two ways. So we work a lot with the Frontier Labs on how do we with the Frontier Labs on how do we with the Frontier Labs on how do we evaluate their models, understand where evaluate their models, understand where evaluate their models, understand where they're breaking, understand what could they're breaking, understand what could they're breaking, understand what could be better about them, and then construct be better about them, and then construct be better about them, and then construct the right either post- training data or the right either post- training data or the right either post- training data or our environments to basically fix that our environments to basically fix that our environments to basically fix that problem. And part of this is like how do problem. And part of this is like how do problem. And part of this is like how do you take something as fuzzy and large as you take something as fuzzy and large as you take something as fuzzy and large as design and break it down to a level that design and break it down to a level that design and break it down to a level that you can identify what is best solved you can identify what is best solved you can identify what is best solved through each method. What are elements through each method. What are elements through each method. What are elements of design that are almost like once you of design that are almost like once you of design that are almost like once you kind of boil down the problem become so kind of boil down the problem become so kind of boil down the problem become so specific that they almost become specific that they almost become specific that they almost become deterministic. So for example, uh if deterministic. So for example, uh if deterministic. So for example, uh if you're trying to train a model to be you're trying to train a model to be you're trying to train a model to be good at selecting color palettes or have good at selecting color palettes or have good at selecting color palettes or have contrast or alignment, those are things contrast or alignment, those are things contrast or alignment, those are things that if you define the problem and the that if you define the problem and the that if you define the problem and the context in a specific enough way, uh you context in a specific enough way, uh you context in a specific enough way, uh you can get to an answer that's like pretty can get to an answer that's like pretty can get to an answer that's like pretty objective or that at least most experts objective or that at least most experts objective or that at least most experts would agree to. But maybe other things would agree to. But maybe other things would agree to. But maybe other things like uh aesthetics, you naturally will like uh aesthetics, you naturally will like uh aesthetics, you naturally will see this expert disagreement and so then see this expert disagreement and so then see this expert disagreement and so then you want to lean on to things that are you want to lean on to things that are you want to lean on to things that are closer to to data. So anyway, we spent a closer to to data. So anyway, we spent a closer to to data. So anyway, we spent a lot of time thinking about all those lot of time thinking about all those lot of time thinking about all those problems. Uh but on the other side is problems. Uh but on the other side is problems. Uh but on the other side is also without even touching the model

also without even touching the model also without even touching the model layer, right? How do we actually help layer, right? How do we actually help layer, right? How do we actually help agents and app layer companies produce agents and app layer companies produce agents and app layer companies produce better things? And there's a lot that better things? And there's a lot that better things? And there's a lot that goes into that, right? You have these goes into that, right? You have these goes into that, right? You have these different sets of problems at the different sets of problems at the different sets of problems at the application layer because you're using application layer because you're using application layer because you're using an off-the-shelf model that tends to an off-the-shelf model that tends to an off-the-shelf model that tends to collapse in terms uh of style, tends to collapse in terms uh of style, tends to collapse in terms uh of style, tends to collapse with the mean. So, how do we collapse with the mean. So, how do we collapse with the mean. So, how do we force that creativity back to the force that creativity back to the force that creativity back to the system? How do we avoid these patterns system? How do we avoid these patterns system? How do we avoid these patterns of slop which we'll talk about a lot of slop which we'll talk about a lot of slop which we'll talk about a lot today? Uh how do you understand like today? Uh how do you understand like today? Uh how do you understand like user preferences or a brand preferences user preferences or a brand preferences user preferences or a brand preferences preference so that you can uh maintain preference so that you can uh maintain preference so that you can uh maintain adurance to that style. Uh so there's adurance to that style. Uh so there's adurance to that style. Uh so there's lots of things that actually need to be lots of things that actually need to be lots of things that actually need to be solved as context or judgment or solved as context or judgment or solved as context or judgment or verification at the app layer which is verification at the app layer which is verification at the app layer which is why we kind of work across both. why we kind of work across both. why we kind of work across both. But maybe I'll start with more of a But maybe I'll start with more of a But maybe I'll start with more of a philosophical question of like how how philosophical question of like how how philosophical question of like how how do you define something that is great? do you define something that is great? do you define something that is great? Like how do you define greatness? And Like how do you define greatness? And Like how do you define greatness? And for something like math it's easier, for something like math it's easier, for something like math it's easier, right? Because there's kind of one right? Because there's kind of one right? Because there's kind of one objective answer and uh great is the objective answer and uh great is the objective answer and uh great is the same as correct. But then for something same as correct. But then for something same as correct. But then for something like writing or design, it's much like writing or design, it's much like writing or design, it's much harder, right? like how do you define harder, right? like how do you define harder, right? like how do you define what's like a great tweet or what's a what's like a great tweet or what's a what's like a great tweet or what's a great art piece or what's a great great art piece or what's a great great art piece or what's a great website? Um I don't know what's the last website? Um I don't know what's the last website? Um I don't know what's the last time that you interacted with a poem or time that you interacted with a poem or time that you interacted with a poem or walked into a coffee shop and for some walked into a coffee shop and for some walked into a coffee shop and for some reason it kind of like hit different and reason it kind of like hit different and reason it kind of like hit different and it felt very special. Uh but probably it felt very special. Uh but probably it felt very special. Uh but probably it's a combination of things that it it it's a combination of things that it it it's a combination of things that it it felt very unique. It felt almost a felt very unique. It felt almost a felt very unique. It felt almost a little different. It kind of called your little different. It kind of called your little different. It kind of called your attention. Uh it felt like there it was attention. Uh it felt like there it was attention. Uh it felt like there it was made with a lot of care and attention to made with a lot of care and attention to made with a lot of care and attention to to detail and craft and it almost had to detail and craft and it almost had to detail and craft and it almost had this sense of of like authenticity. Um, this sense of of like authenticity. Um, this sense of of like authenticity. Um, and I think that's a lot of what AI is and I think that's a lot of what AI is and I think that's a lot of what AI is missing today is like how do we take uh missing today is like how do we take uh missing today is like how do we take uh things that are not necessarily average, things that are not necessarily average, things that are not necessarily average, right? How do we produce things that are

right? How do we produce things that are right? How do we produce things that are purposely like out of distribution? Um, purposely like out of distribution? Um, purposely like out of distribution? Um, and slop is kind of the opposite of and slop is kind of the opposite of and slop is kind of the opposite of that, right? I think it is hard to that, right? I think it is hard to that, right? I think it is hard to define what is great sometimes, but I define what is great sometimes, but I define what is great sometimes, but I think it's pretty pretty easy to define think it's pretty pretty easy to define think it's pretty pretty easy to define what is slop in the sense that most what is slop in the sense that most what is slop in the sense that most people would agree. I think the sense of people would agree. I think the sense of people would agree. I think the sense of like repetition of kind of soullessness like repetition of kind of soullessness like repetition of kind of soullessness is something that all of us feel right is something that all of us feel right is something that all of us feel right now when using AI. And I think it's now when using AI. And I think it's now when using AI. And I think it's quite magical by the way that AI has quite magical by the way that AI has quite magical by the way that AI has gotten to a point that any human on the gotten to a point that any human on the gotten to a point that any human on the planet that is not even a designer that planet that is not even a designer that planet that is not even a designer that is not an engineer can click a button is not an engineer can click a button is not an engineer can click a button and suddenly make an entire PowerPoint and suddenly make an entire PowerPoint and suddenly make an entire PowerPoint or make a website or make a web app. or make a website or make a web app. or make a website or make a web app. That's pretty cool. But it comes with That's pretty cool. But it comes with That's pretty cool. But it comes with consequences, right? Uh it comes with consequences, right? Uh it comes with consequences, right? Uh it comes with consequences of suddenly now the cost of consequences of suddenly now the cost of consequences of suddenly now the cost of generation is basically going to zero. generation is basically going to zero. generation is basically going to zero. Uh but the average person hasn't Uh but the average person hasn't Uh but the average person hasn't necessarily honeed their taste. Like I necessarily honeed their taste. Like I necessarily honeed their taste. Like I just think about the amount of effort just think about the amount of effort just think about the amount of effort and work that a designer puts in and work that a designer puts in and work that a designer puts in throughout their life to like build up throughout their life to like build up throughout their life to like build up their taste, right? Like there's all their taste, right? Like there's all their taste, right? Like there's all this process of like getting exposed to this process of like getting exposed to this process of like getting exposed to many things and learning to like spot many things and learning to like spot many things and learning to like spot patterns and learning to develop a point patterns and learning to develop a point patterns and learning to develop a point of view and like kind of do things in a of view and like kind of do things in a of view and like kind of do things in a in a courageous way that maybe are a in a courageous way that maybe are a in a courageous way that maybe are a little bit against the norm, learning little bit against the norm, learning little bit against the norm, learning what not to do and how to like have what not to do and how to like have what not to do and how to like have restraint. And that's very hard. Like restraint. And that's very hard. Like restraint. And that's very hard. Like the the average person doesn't the the average person doesn't the the average person doesn't necessarily have the the time nor the necessarily have the the time nor the necessarily have the the time nor the skills to go and develop taste in skills to go and develop taste in skills to go and develop taste in everything, let's say in design. And so everything, let's say in design. And so everything, let's say in design. And so um I think it would be a bad case um I think it would be a bad case um I think it would be a bad case scenario for us to just like be like scenario for us to just like be like scenario for us to just like be like okay the way to fix soft is for everyone okay the way to fix soft is for everyone okay the way to fix soft is for everyone to have taste because I don't think to have taste because I don't think to have taste because I don't think that's necessarily realistic. Um I think that's necessarily realistic. Um I think that's necessarily realistic. Um I think how do we how can we understand this how do we how can we understand this how do we how can we understand this better so that we can make even for the better so that we can make even for the better so that we can make even for the average person the ability to create average person the ability to create average person the ability to create something great and to understand maybe something great and to understand maybe something great and to understand maybe their own taste um easy more more easy.

their own taste um easy more more easy. their own taste um easy more more easy. So uh that's that's a lot of what we're So uh that's that's a lot of what we're So uh that's that's a lot of what we're we're focusing on. Um so yeah I think we're focusing on. Um so yeah I think we're focusing on. Um so yeah I think this phenomenon of slot by the way is this phenomenon of slot by the way is this phenomenon of slot by the way is not new. uh if you were in the internet not new. uh if you were in the internet not new. uh if you were in the internet uh as social media emerged, you probably uh as social media emerged, you probably uh as social media emerged, you probably saw a lot of slop before that. But I do saw a lot of slop before that. But I do saw a lot of slop before that. But I do think that AI has been this kind of like think that AI has been this kind of like think that AI has been this kind of like accelerating force, right, of like being accelerating force, right, of like being accelerating force, right, of like being able to create things very easily uh able to create things very easily uh able to create things very easily uh with a click of a button and the like with a click of a button and the like with a click of a button and the like thoughtlessness around it. And there's thoughtlessness around it. And there's thoughtlessness around it. And there's kind of these three characteristics that kind of these three characteristics that kind of these three characteristics that I I would say repeat and stop. Uh so a I I would say repeat and stop. Uh so a I I would say repeat and stop. Uh so a repetition so you start seeing the same repetition so you start seeing the same repetition so you start seeing the same thing many many many times. Um the thing many many many times. Um the thing many many many times. Um the second is lack of fit which I actually second is lack of fit which I actually second is lack of fit which I actually think is is very related. So fit is kind think is is very related. So fit is kind think is is very related. So fit is kind of this ability for something to feel of this ability for something to feel of this ability for something to feel correct for a specific context right for correct for a specific context right for correct for a specific context right for a specific moment in time for a specific a specific moment in time for a specific a specific moment in time for a specific person. Uh but suddenly if you have person. Uh but suddenly if you have person. Uh but suddenly if you have repetition and let's say one person asks repetition and let's say one person asks repetition and let's say one person asks for a website for their pet shop and the for a website for their pet shop and the for a website for their pet shop and the other one asked for a website for their other one asked for a website for their other one asked for a website for their finance firm and somehow those designs finance firm and somehow those designs finance firm and somehow those designs converge and look the same. That's quite converge and look the same. That's quite converge and look the same. That's quite odd, right? Like if that was in if you odd, right? Like if that was in if you odd, right? Like if that was in if you were actually crafting that with care were actually crafting that with care were actually crafting that with care that wouldn't you wouldn't converge that wouldn't you wouldn't converge that wouldn't you wouldn't converge necessarily on those things. And so this necessarily on those things. And so this necessarily on those things. And so this lack of fit and lack of understanding of lack of fit and lack of understanding of lack of fit and lack of understanding of context is actually a huge problem that context is actually a huge problem that context is actually a huge problem that like leads to slop. Um, and the third is like leads to slop. Um, and the third is like leads to slop. Um, and the third is maybe low intent, which is probably a maybe low intent, which is probably a maybe low intent, which is probably a mix of, yeah, you're going to have a mix of, yeah, you're going to have a mix of, yeah, you're going to have a bunch of people prompting really quickly bunch of people prompting really quickly bunch of people prompting really quickly and maybe just wanting to oneshot and maybe just wanting to oneshot and maybe just wanting to oneshot something. But I think there's actually something. But I think there's actually something. But I think there's actually this like intent interpretation piece this like intent interpretation piece this like intent interpretation piece that's missing in the systems that we're that's missing in the systems that we're that's missing in the systems that we're building. Like how can you help your building. Like how can you help your building. Like how can you help your user, right? Like how can you help them user, right? Like how can you help them user, right? Like how can you help them better understand the intent that they better understand the intent that they better understand the intent that they have? Um, so that you can add more color have? Um, so that you can add more color have? Um, so that you can add more color and add more context on onto what you're and add more context on onto what you're and add more context on onto what you're trying to create.

trying to create. trying to create. Okay? And I I'm a big believer, by the Okay? And I I'm a big believer, by the Okay? And I I'm a big believer, by the way, that you in order to fix something, way, that you in order to fix something, way, that you in order to fix something, you first have to measure it and you you first have to measure it and you you first have to measure it and you first have to understand it. I think first have to understand it. I think first have to understand it. I think that's exactly why we're so focused on that's exactly why we're so focused on that's exactly why we're so focused on like how do we uh turn these domains like how do we uh turn these domains like how do we uh turn these domains into something a bit more verifiable so into something a bit more verifiable so into something a bit more verifiable so that we can attach a measure to it. So, that we can attach a measure to it. So, that we can attach a measure to it. So, uh you'll you'll go on a little bit of a uh you'll you'll go on a little bit of a uh you'll you'll go on a little bit of a research journey with me here now, but research journey with me here now, but research journey with me here now, but we basically wanted to figure out can we we basically wanted to figure out can we we basically wanted to figure out can we measure slop like can we actually measure slop like can we actually measure slop like can we actually measure this quantitatively and spot measure this quantitatively and spot measure this quantitatively and spot this and what does that like look like? this and what does that like look like? this and what does that like look like? So we analyzed over two million websites So we analyzed over two million websites So we analyzed over two million websites from the past like 10 years kind of like from the past like 10 years kind of like from the past like 10 years kind of like way back machine style to try to way back machine style to try to way back machine style to try to understand all the trends across like understand all the trends across like understand all the trends across like design how is the internet changing uh design how is the internet changing uh design how is the internet changing uh how are how is like design changing over how are how is like design changing over how are how is like design changing over time and two things were interesting and time and two things were interesting and time and two things were interesting and we also by the way then kind of we also by the way then kind of we also by the way then kind of synthetically generated a set of uh synthetically generated a set of uh synthetically generated a set of uh design websites so we could kind of like design websites so we could kind of like design websites so we could kind of like compare like how does humanmade sites compare like how does humanmade sites compare like how does humanmade sites compare to AI generated ones and there compare to AI generated ones and there compare to AI generated ones and there were a few things that were interesting were a few things that were interesting were a few things that were interesting so one was that you already kind of saw so one was that you already kind of saw so one was that you already kind of saw a a bit of like a collapse uh on the a a bit of like a collapse uh on the a a bit of like a collapse uh on the internet before even AI. So you saw kind internet before even AI. So you saw kind internet before even AI. So you saw kind of the internet becoming more of the internet becoming more of the internet becoming more homogeneous using more similar color homogeneous using more similar color homogeneous using more similar color palettes using more similar layouts uh palettes using more similar layouts uh palettes using more similar layouts uh which is probably a function of more uh which is probably a function of more uh which is probably a function of more uh I would say this like kind of trend I would say this like kind of trend I would say this like kind of trend spreading more more more quickly let's spreading more more more quickly let's spreading more more more quickly let's say uh but with AI I think you saw this say uh but with AI I think you saw this say uh but with AI I think you saw this repetition happening a lot more and repetition happening a lot more and repetition happening a lot more and being almost more like um identified being almost more like um identified being almost more like um identified kind of regardless of context. So even kind of regardless of context. So even kind of regardless of context. So even in completely different buckets you saw in completely different buckets you saw in completely different buckets you saw patterns that were very similar. So we patterns that were very similar. So we patterns that were very similar. So we we built this I I call this probes but we built this I I call this probes but we built this I I call this probes but basically we uh we did two things. So we

basically we uh we did two things. So we basically we uh we did two things. So we did this like pattern mining on all this did this like pattern mining on all this did this like pattern mining on all this data to understand like what are data to understand like what are data to understand like what are features that we can extract from all features that we can extract from all features that we can extract from all these sites. What are all these these sites. What are all these these sites. What are all these characteristics that we can make more characteristics that we can make more characteristics that we can make more objective right? Colors, typography, objective right? Colors, typography, objective right? Colors, typography, layout, audience like how can we like layout, audience like how can we like layout, audience like how can we like distill this down into things that distill this down into things that distill this down into things that become almost like uh structured and become almost like uh structured and become almost like uh structured and then how do we uh train up these like then how do we uh train up these like then how do we uh train up these like probes? So think of these as like baby probes? So think of these as like baby probes? So think of these as like baby classifiers like how do we train the classifiers like how do we train the classifiers like how do we train the ability to spot this one characteristic ability to spot this one characteristic ability to spot this one characteristic and for all these slop sites we identify and for all these slop sites we identify and for all these slop sites we identify we started identifying like what are the we started identifying like what are the we started identifying like what are the probes that basically mean this site is probes that basically mean this site is probes that basically mean this site is very likely to be AI slop. Um, and very likely to be AI slop. Um, and very likely to be AI slop. Um, and especially when you start combining them especially when you start combining them especially when you start combining them and you see the frequency of multiple of and you see the frequency of multiple of and you see the frequency of multiple of these happening at once, it became very these happening at once, it became very these happening at once, it became very likely that you could actually like likely that you could actually like likely that you could actually like measure uh and predict slop. And we saw measure uh and predict slop. And we saw measure uh and predict slop. And we saw a a super high basically ability to do a a super high basically ability to do a a super high basically ability to do that prediction, which was really cool that prediction, which was really cool that prediction, which was really cool to see. This performed better by the way to see. This performed better by the way to see. This performed better by the way than like most LLM as a judge methods of than like most LLM as a judge methods of than like most LLM as a judge methods of like asking an LLM to like judge if that like asking an LLM to like judge if that like asking an LLM to like judge if that uh is like great human quality versus uh is like great human quality versus uh is like great human quality versus like AI generated slop. Uh so that was like AI generated slop. Uh so that was like AI generated slop. Uh so that was pretty cool to see. I think it kind of pretty cool to see. I think it kind of pretty cool to see. I think it kind of shows this pattern that we see in AI shows this pattern that we see in AI shows this pattern that we see in AI really being uh an actual like really being uh an actual like really being uh an actual like quantitative thing that we can see in quantitative thing that we can see in quantitative thing that we can see in slop uh which I find really cool. But slop uh which I find really cool. But slop uh which I find really cool. But obviously we don't want to stop there, obviously we don't want to stop there, obviously we don't want to stop there, right? We don't want to just measure right? We don't want to just measure right? We don't want to just measure slop. We want to also solve it. And so slop. We want to also solve it. And so slop. We want to also solve it. And so um there's a few I think I mentioned um there's a few I think I mentioned um there's a few I think I mentioned this before, but like the as the cost of this before, but like the as the cost of this before, but like the as the cost of production basically goes to zero. I production basically goes to zero. I production basically goes to zero. I think the thing that becomes expensive think the thing that becomes expensive think the thing that becomes expensive and matters more than ever is judgment.

and matters more than ever is judgment. and matters more than ever is judgment. Um I don't even want to use the word Um I don't even want to use the word Um I don't even want to use the word taste here. uh is judgment I think is taste here. uh is judgment I think is taste here. uh is judgment I think is this ability to discern what's right is this ability to discern what's right is this ability to discern what's right is this ability to break down a problem so this ability to break down a problem so this ability to break down a problem so that you can actually understand it and that you can actually understand it and that you can actually understand it and create solutions for it and so yes create solutions for it and so yes create solutions for it and so yes there's the side of judgment that is there's the side of judgment that is there's the side of judgment that is human judgment that I actually think is human judgment that I actually think is human judgment that I actually think is more valuable than ever but there's also more valuable than ever but there's also more valuable than ever but there's also this side of like how do we build the this side of like how do we build the this side of like how do we build the right tools and systems to like fix right tools and systems to like fix right tools and systems to like fix pieces of this problem right pieces of this problem right pieces of this problem right so yeah how do we how do we fight slop so yeah how do we how do we fight slop so yeah how do we how do we fight slop my my enemy um and by the way I think my my enemy um and by the way I think my my enemy um and by the way I think there's there's a lot of conversation there's there's a lot of conversation there's there's a lot of conversation going around how do you fight slop at going around how do you fight slop at going around how do you fight slop at the model year like how do we make the model year like how do we make the model year like how do we make models better? How do we make models models better? How do we make models models better? How do we make models have a higher bar? Which don't get me have a higher bar? Which don't get me have a higher bar? Which don't get me wrong has to be solved and we're working wrong has to be solved and we're working wrong has to be solved and we're working very hard to solve that too. But I very hard to solve that too. But I very hard to solve that too. But I actually think this problem of inference actually think this problem of inference actually think this problem of inference time is equally if not even more time is equally if not even more time is equally if not even more important because that's actually when important because that's actually when important because that's actually when you interact with the end user and this you interact with the end user and this you interact with the end user and this kind of back and forth of how do you kind of back and forth of how do you kind of back and forth of how do you understand this context and intent understand this context and intent understand this context and intent happens at the moment of inference time. happens at the moment of inference time. happens at the moment of inference time. So I don't think that we can ignore and So I don't think that we can ignore and So I don't think that we can ignore and just make models better and not solve just make models better and not solve just make models better and not solve this otherwise stop will keep existing. this otherwise stop will keep existing. this otherwise stop will keep existing. Um so maybe breaking down a few of those Um so maybe breaking down a few of those Um so maybe breaking down a few of those pieces and kind of um a few of the ways pieces and kind of um a few of the ways pieces and kind of um a few of the ways that we've thought about solving this or that we've thought about solving this or that we've thought about solving this or a few solutions that we built to solve a few solutions that we built to solve a few solutions that we built to solve this. But I think for example for this. But I think for example for this. But I think for example for something like repetition, one of the something like repetition, one of the something like repetition, one of the things that we're working on is I I've things that we're working on is I I've things that we're working on is I I've nicknamed it. I don't know if that's nicknamed it. I don't know if that's nicknamed it. I don't know if that's going to be the official name, but like going to be the official name, but like going to be the official name, but like the creativity API. How can we create a the creativity API. How can we create a the creativity API. How can we create a system that almost becomes an system that almost becomes an system that almost becomes an inspiration machine for your agent so inspiration machine for your agent so inspiration machine for your agent so that it can produce something that's that it can produce something that's that it can produce something that's actually out of distribution instead of actually out of distribution instead of actually out of distribution instead of something that is in that same average something that is in that same average something that is in that same average and kind of mean that we're seeing and kind of mean that we're seeing and kind of mean that we're seeing happen with like the slop sites. Um, so happen with like the slop sites. Um, so happen with like the slop sites. Um, so this is one of the ways that practically this is one of the ways that practically this is one of the ways that practically if we can intentionally produce if we can intentionally produce if we can intentionally produce something that's out of distribution,

something that's out of distribution, something that's out of distribution, you can improve this like overall uh you can improve this like overall uh you can improve this like overall uh quality. And by the way, I I don't think quality. And by the way, I I don't think quality. And by the way, I I don't think that this can be something just like that this can be something just like that this can be something just like randomness. It's not just about like randomness. It's not just about like randomness. It's not just about like turning up a temperature of a model and turning up a temperature of a model and turning up a temperature of a model and and kind of fingers crossed hoping for and kind of fingers crossed hoping for and kind of fingers crossed hoping for the best. I think it's much more like the best. I think it's much more like the best. I think it's much more like how do we understand um even like what how do we understand um even like what how do we understand um even like what are rules or expectations in specific are rules or expectations in specific are rules or expectations in specific domains like let's say that you ask for domains like let's say that you ask for domains like let's say that you ask for a slide deck for for the pitch of your a slide deck for for the pitch of your a slide deck for for the pitch of your startup like what is a what does a good startup like what is a what does a good startup like what is a what does a good pitch deck look like and then how do you pitch deck look like and then how do you pitch deck look like and then how do you almost like intentionally break rules uh almost like intentionally break rules uh almost like intentionally break rules uh to create things that are more creative to create things that are more creative to create things that are more creative right because usually creativity isn't right because usually creativity isn't right because usually creativity isn't like randomness isn't doing something like randomness isn't doing something like randomness isn't doing something that completely feels off for that that completely feels off for that that completely feels off for that situation it's like you intentionally situation it's like you intentionally situation it's like you intentionally maybe diverge on a couple of things maybe diverge on a couple of things maybe diverge on a couple of things while maintaining kind of um a dear to while maintaining kind of um a dear to while maintaining kind of um a dear to to expectations of that category let's to expectations of that category let's to expectations of that category let's say for others. So that's one of the say for others. So that's one of the say for others. So that's one of the things we're working on. The second one things we're working on. The second one things we're working on. The second one on this problem of fit I think um it's on this problem of fit I think um it's on this problem of fit I think um it's interesting but brands as probably a lot interesting but brands as probably a lot interesting but brands as probably a lot of you who are designers know takes so of you who are designers know takes so of you who are designers know takes so much effort to create great brands like much effort to create great brands like much effort to create great brands like great brands are the work of dozens of great brands are the work of dozens of great brands are the work of dozens of designers uh putting in a lot of like designers uh putting in a lot of like designers uh putting in a lot of like craft and thought and care um and so craft and thought and care um and so craft and thought and care um and so we've almost like already pre-done the we've almost like already pre-done the we've almost like already pre-done the work of defining what is great for that work of defining what is great for that work of defining what is great for that specific company and then we're not specific company and then we're not specific company and then we're not using it well. So this like brand using it well. So this like brand using it well. So this like brand endurance actually I think is a huge endurance actually I think is a huge endurance actually I think is a huge problem and one of the things that can problem and one of the things that can problem and one of the things that can very more easily let's say like raise very more easily let's say like raise very more easily let's say like raise that bar of quality. So I I'll touch on that bar of quality. So I I'll touch on that bar of quality. So I I'll touch on an example on this one specifically and an example on this one specifically and an example on this one specifically and then same with like intent and judgment.

then same with like intent and judgment. then same with like intent and judgment. I think the baby classifiers was a good I think the baby classifiers was a good I think the baby classifiers was a good example um like how it how we can example um like how it how we can example um like how it how we can actually like use this to be even become actually like use this to be even become actually like use this to be even become a gate for slop and and not let your a gate for slop and and not let your a gate for slop and and not let your agent uh ship slop. But so the brand API agent uh ship slop. But so the brand API agent uh ship slop. But so the brand API is the first product that we're is the first product that we're is the first product that we're releasing to to the public. This is releasing to to the public. This is releasing to to the public. This is already in in beta testing with a bunch already in in beta testing with a bunch already in in beta testing with a bunch of uh our design partners. And of uh our design partners. And of uh our design partners. And essentially what it does is it can take essentially what it does is it can take essentially what it does is it can take let's say a brand URL and extract this let's say a brand URL and extract this let's say a brand URL and extract this into like very specific components that into like very specific components that into like very specific components that are good for an agent to follow. So are good for an agent to follow. So are good for an agent to follow. So basically how do we turn something as basically how do we turn something as basically how do we turn something as fuzzy as a brand into something so fuzzy as a brand into something so fuzzy as a brand into something so structured that it becomes easy to uh structured that it becomes easy to uh structured that it becomes easy to uh for your agent to follow that but also for your agent to follow that but also for your agent to follow that but also for you to judge against it right for you to judge against it right for you to judge against it right because I think the piece that we can't because I think the piece that we can't because I think the piece that we can't forget here is this judgment and forget here is this judgment and forget here is this judgment and verification. So yes this goes and helps verification. So yes this goes and helps verification. So yes this goes and helps your agent to produce something better. your agent to produce something better. your agent to produce something better. Uh but how can we also add a way for you Uh but how can we also add a way for you Uh but how can we also add a way for you to judge okay is the agent actually to judge okay is the agent actually to judge okay is the agent actually staying on track? Is it actually staying on track? Is it actually staying on track? Is it actually performing well to adhere to this brand performing well to adhere to this brand performing well to adhere to this brand or how is it failing or where is it or how is it failing or where is it or how is it failing or where is it failing? So this is the first flow I failing? So this is the first flow I failing? So this is the first flow I would say that we we are seeing that is would say that we we are seeing that is would say that we we are seeing that is really helping to improve quality. Um really helping to improve quality. Um really helping to improve quality. Um and what's cool is of course we're and what's cool is of course we're and what's cool is of course we're talking here about an example of a brand talking here about an example of a brand talking here about an example of a brand that already exists. But let's say you that already exists. But let's say you that already exists. But let's say you have an agent uh you have an app and uh have an agent uh you have an app and uh have an agent uh you have an app and uh the person that is using your app the person that is using your app the person that is using your app actually doesn't have a brand. Let's say actually doesn't have a brand. Let's say actually doesn't have a brand. Let's say they're an average consumer. Can we they're an average consumer. Can we they're an average consumer. Can we actually one of the things that we're actually one of the things that we're actually one of the things that we're creating is basically like a repository creating is basically like a repository creating is basically like a repository like an index of brands uh of pre almost like an index of brands uh of pre almost like an index of brands uh of pre almost like pre-created brand systems so that like pre-created brand systems so that like pre-created brand systems so that if they want something that feels dreamy if they want something that feels dreamy if they want something that feels dreamy why not retrieve a dreamy brand system why not retrieve a dreamy brand system why not retrieve a dreamy brand system that already has been thought out to be that already has been thought out to be that already has been thought out to be cohesive instead of doing like a cohesive instead of doing like a cohesive instead of doing like a generative approach the moment of that generative approach the moment of that generative approach the moment of that might end up not so great or might end might end up not so great or might end might end up not so great or might end up again in those pillars of slop. And I

up again in those pillars of slop. And I up again in those pillars of slop. And I want to show you a real example of this want to show you a real example of this want to show you a real example of this in action. So um there's this company in action. So um there's this company in action. So um there's this company that I think is awesome called the that I think is awesome called the that I think is awesome called the General Intelligence Copy of New York. General Intelligence Copy of New York. General Intelligence Copy of New York. They have a sick website. You guys They have a sick website. You guys They have a sick website. You guys should check it out. Um, but basically should check it out. Um, but basically should check it out. Um, but basically if you ask Claw Design to create a slide if you ask Claw Design to create a slide if you ask Claw Design to create a slide deck uh in their branding, the the deck uh in their branding, the the deck uh in their branding, the the middle one is basically what it comes up middle one is basically what it comes up middle one is basically what it comes up with. So the one on the left is is the with. So the one on the left is is the with. So the one on the left is is the original brand. Uh, this is kind of the original brand. Uh, this is kind of the original brand. Uh, this is kind of the the default. And if you kind of use this the default. And if you kind of use this the default. And if you kind of use this extraction actually in the process, it extraction actually in the process, it extraction actually in the process, it creates something that's way more high creates something that's way more high creates something that's way more high fidelity with the original. Um, and that fidelity with the original. Um, and that fidelity with the original. Um, and that even like in the details I would say even like in the details I would say even like in the details I would say like feels right. So this is just to like feels right. So this is just to like feels right. So this is just to show an example of it in in action. Um show an example of it in in action. Um show an example of it in in action. Um but yeah, I think we but yeah, I think we but yeah, I think we I think all of us would agree that like I think all of us would agree that like I think all of us would agree that like human human taste and kind of the peak human human taste and kind of the peak human human taste and kind of the peak of human craft is always going to be of human craft is always going to be of human craft is always going to be like deeply valuable and that right now like deeply valuable and that right now like deeply valuable and that right now I think the challenge is we we are I think the challenge is we we are I think the challenge is we we are almost even not earning the right to almost even not earning the right to almost even not earning the right to debate this like how can we have uh debate this like how can we have uh debate this like how can we have uh models like reach this like pinnacle of models like reach this like pinnacle of models like reach this like pinnacle of taste. I don't think it's about that at taste. I don't think it's about that at taste. I don't think it's about that at all. It's like how do we first just like all. It's like how do we first just like all. It's like how do we first just like raise the bar? Like the bar is kind of raise the bar? Like the bar is kind of raise the bar? Like the bar is kind of really I would say on the ground and so really I would say on the ground and so really I would say on the ground and so I think all of this work that we're I think all of this work that we're I think all of this work that we're putting into like how do we decompose a putting into like how do we decompose a putting into like how do we decompose a problem and how do we measure it is problem and how do we measure it is problem and how do we measure it is exactly so that we can at least like exactly so that we can at least like exactly so that we can at least like improve this bar of quality and I think improve this bar of quality and I think improve this bar of quality and I think we have to start with that.

That's it. Uh thank you very much for That's it. Uh thank you very much for for the time. Uh this is this is for the time. Uh this is this is for the time. Uh this is this is awesome. [applause]