SOTA Generative Media Panel — Dumitru Erhan, Shane Gu & Nicole Brichtova, Google DeepMind

and welcome back for those on the stream and welcome back for those on the stream and those those in person. um we take and those those in person. um we take and those those in person. um we take tend to basically take these longer tend to basically take these longer tend to basically take these longer sessions between uh all the sort of sessions between uh all the sort of sessions between uh all the sort of mainstage keynotes to reflect on things mainstage keynotes to reflect on things mainstage keynotes to reflect on things that um you know are particularly that um you know are particularly that um you know are particularly important but like don't have like a important but like don't have like a important but like don't have like a significant like sort of launch moments. significant like sort of launch moments. significant like sort of launch moments. Today we're very lucky to have people Today we're very lucky to have people Today we're very lucky to have people working on Omni and VO Nano Banana like working on Omni and VO Nano Banana like working on Omni and VO Nano Banana like the you know the world's best generative the you know the world's best generative the you know the world's best generative models here with us. Uh, Demetrio, I I I models here with us. Uh, Demetrio, I I I models here with us. Uh, Demetrio, I I I first saw you when you were posting first saw you when you were posting first saw you when you were posting about your office. [laughter] about your office. [laughter] about your office. [laughter] Um, I think you're you're probably Um, I think you're you're probably Um, I think you're you're probably number one uh Google Google's number one number one uh Google Google's number one number one uh Google Google's number one office influencer at least in in San office influencer at least in in San office influencer at least in in San Francisco. I think you like you like to Francisco. I think you like you like to Francisco. I think you like you like to bike as well. You like to take photos of bike as well. You like to take photos of bike as well. You like to take photos of >> bike here. >> bike here. >> bike here. >> Yeah. Um, but you know, but also you >> Yeah. Um, but you know, but also you >> Yeah. Um, but you know, but also you work on video models. work on video models. work on video models. >> That's right. >> That's right. >> That's right. >> Um, Shane, I I met you I think at like a >> Um, Shane, I I met you I think at like a >> Um, Shane, I I met you I think at like a dinner. dinner. dinner. >> Yeah. Um and uh and uh and I I remember >> Yeah. Um and uh and uh and I I remember >> Yeah. Um and uh and uh and I I remember you were trying to get me invested in you were trying to get me invested in you were trying to get me invested in like one of the companies. I forget like one of the companies. I forget like one of the companies. I forget forget which one.

forget which one. forget which one. >> Forget about that. [laughter] >> Forget about that. [laughter] >> Forget about that. [laughter] >> But now but now you're um now you're >> But now but now you're um now you're >> But now but now you're um now you're working on Omni Thinking. Um and and working on Omni Thinking. Um and and working on Omni Thinking. Um and and just you know a bunch of other just you know a bunch of other just you know a bunch of other >> Gemini RL. >> Gemini RL. >> Gemini RL. >> Yeah. Yeah. Uh and Nicole also uh the >> Yeah. Yeah. Uh and Nicole also uh the >> Yeah. Yeah. Uh and Nicole also uh the rest of the gen media models uh nano rest of the gen media models uh nano rest of the gen media models uh nano banana and uh all and everything you banana and uh all and everything you banana and uh all and everything you just launched actually even this week. just launched actually even this week. just launched actually even this week. Uh, Uh, Uh, >> yeah. We launched some APIs. >> yeah. We launched some APIs. >> yeah. We launched some APIs. >> Yeah. Yeah. Yeah. >> Yeah. Yeah. Yeah. >> Yeah. Yeah. Yeah. >> And I haven't tried to convince you to >> And I haven't tried to convince you to >> And I haven't tried to convince you to invest in anything, but maybe I should. invest in anything, but maybe I should. invest in anything, but maybe I should. >> I mean, so I try not to be an investor. >> I mean, so I try not to be an investor. >> I mean, so I try not to be an investor. People just convince me anyway. I'm like People just convince me anyway. I'm like People just convince me anyway. I'm like just, okay, well, I'm not that rich, but just, okay, well, I'm not that rich, but just, okay, well, I'm not that rich, but know like you can't not try to invest in know like you can't not try to invest in know like you can't not try to invest in some of these things. And, you know, for some of these things. And, you know, for some of these things. And, you know, for those of us who are not working at a those of us who are not working at a those of us who are not working at a Frontier Lab, this is the best this Frontier Lab, this is the best this Frontier Lab, this is the best this closest we'll ever get. Um, so yeah, closest we'll ever get. Um, so yeah, closest we'll ever get. Um, so yeah, actually, let's kind of recap since actually, let's kind of recap since actually, let's kind of recap since you're closest to it and we just did it, you're closest to it and we just did it, you're closest to it and we just did it, like what was launched this week? What like what was launched this week? What like what was launched this week? What should people go try out? should people go try out? should people go try out? >> Yeah. Um so yesterday we had two launch >> Yeah. Um so yesterday we had two launch >> Yeah. Um so yesterday we had two launch moments. Uh one of them we launched moments. Uh one of them we launched moments. Uh one of them we launched NanoBanana 2 light uh which is our NanoBanana 2 light uh which is our NanoBanana 2 light uh which is our fastest, cheapest um image model in the fastest, cheapest um image model in the fastest, cheapest um image model in the nano banana model family. Um and it's nano banana model family. Um and it's nano banana model family. Um and it's better than the original NanoBanana. Um better than the original NanoBanana. Um better than the original NanoBanana. Um so really for most people um that model so really for most people um that model so really for most people um that model replaces what you you know used and love replaces what you you know used and love replaces what you you know used and love the original Nano Banana for across like the original Nano Banana for across like the original Nano Banana for across like generation and editing and it gets generation and editing and it gets generation and editing and it gets really close to the frontier quality of really close to the frontier quality of really close to the frontier quality of of the kind of mainland bigger models.

of the kind of mainland bigger models. of the kind of mainland bigger models. So that that's really exciting. I think So that that's really exciting. I think So that that's really exciting. I think if you look at some of the demos or like if you look at some of the demos or like if you look at some of the demos or like things that people have been trying like things that people have been trying like things that people have been trying like getting kind of that like 3 second getting kind of that like 3 second getting kind of that like 3 second latency just unlocks a whole bunch of latency just unlocks a whole bunch of latency just unlocks a whole bunch of things that you can do with like things that you can do with like things that you can do with like ideation and iteration and it's just ideation and iteration and it's just ideation and iteration and it's just really fun and the model's getting to a really fun and the model's getting to a really fun and the model's getting to a point where like the quality is really point where like the quality is really point where like the quality is really good um where um it you know you can use good um where um it you know you can use good um where um it you know you can use it for iteration but you can also use it for iteration but you can also use it for iteration but you can also use some of those outputs as just kind of some of those outputs as just kind of some of those outputs as just kind of like ready um production output. So like ready um production output. So like ready um production output. So that's really exciting. Um and then that's really exciting. Um and then that's really exciting. Um and then second launch we finally um launched the second launch we finally um launched the second launch we finally um launched the Gemini Omni Flash APIs um that we Gemini Omni Flash APIs um that we Gemini Omni Flash APIs um that we pre-announced at IO. So thank you for pre-announced at IO. So thank you for pre-announced at IO. So thank you for waiting. Um and that you know is the waiting. Um and that you know is the waiting. Um and that you know is the first time that we're making the APIs first time that we're making the APIs first time that we're making the APIs available for developers and it's available for developers and it's available for developers and it's basically really exciting kind of video basically really exciting kind of video basically really exciting kind of video generation and editing and we're pricing generation and editing and we're pricing generation and editing and we're pricing it the same as Y31 fast. So we're it the same as Y31 fast. So we're it the same as Y31 fast. So we're getting you kind of like really really getting you kind of like really really getting you kind of like really really good quality for a really awesome price good quality for a really awesome price good quality for a really awesome price hopefully. Um hopefully. Um hopefully. Um >> yeah, I mean that that's incredible. I'm >> yeah, I mean that that's incredible. I'm >> yeah, I mean that that's incredible. I'm actually really So when you guys actually really So when you guys actually really So when you guys launched Omni for the first time, you launched Omni for the first time, you launched Omni for the first time, you also did a podcast uh with Logan who also did a podcast uh with Logan who also did a podcast uh with Logan who couldn't be here today uh and you added couldn't be here today uh and you added couldn't be here today uh and you added like a sloth uh and and Ramen and all like a sloth uh and and Ramen and all like a sloth uh and and Ramen and all these all these things. I actually these all these things. I actually these all these things. I actually really want to do that to our videos. I really want to do that to our videos. I really want to do that to our videos. I just didn't have an API for it because just didn't have an API for it because just didn't have an API for it because obviously I have to automate the whole obviously I have to automate the whole obviously I have to automate the whole thing. So thank you for the API.

thing. So thank you for the API. thing. So thank you for the API. >> Uh that is my favorite use case. >> Uh that is my favorite use case. >> Uh that is my favorite use case. Everybody should do that. Um I got a cat Everybody should do that. Um I got a cat Everybody should do that. Um I got a cat which is probably like the most boring which is probably like the most boring which is probably like the most boring of the animals. Um if you don't know of the animals. Um if you don't know of the animals. Um if you don't know what we're talking about, you should what we're talking about, you should what we're talking about, you should look it up. It's very funny. Feurer. um look it up. It's very funny. Feurer. um look it up. It's very funny. Feurer. um Furer who's um you know on on the team Furer who's um you know on on the team Furer who's um you know on on the team did that. did that. did that. >> Furer is the number one guy you should >> Furer is the number one guy you should >> Furer is the number one guy you should follow for you should follow ideas on follow for you should follow ideas on follow for you should follow ideas on okay what can this thing do? okay what can this thing do? okay what can this thing do? >> Yes. >> Yes. >> Yes. >> Right. >> Right. >> Right. >> Yes. He he's he's amazing at that. >> Yes. He he's he's amazing at that. >> Yes. He he's he's amazing at that. >> I've tried to get him for the last two >> I've tried to get him for the last two >> I've tried to get him for the last two years to come to AIE. He hasn't made it years to come to AIE. He hasn't made it years to come to AIE. He hasn't made it yet. He's actually come in person. He yet. He's actually come in person. He yet. He's actually come in person. He just didn't want to speak because he's just didn't want to speak because he's just didn't want to speak because he's anonymous. anonymous. anonymous. >> I know. >> I know. >> I know. >> I I want to say his real name but I >> I I want to say his real name but I >> I I want to say his real name but I can't say his real name. can't say his real name. can't say his real name. >> No [laughter] no we won't we won't do >> No [laughter] no we won't we won't do >> No [laughter] no we won't we won't do that to him. But you should really that to him. But you should really that to him. But you should really follow him. He's amazing. follow him. He's amazing. follow him. He's amazing. >> He did all that work. I actually met him >> He did all that work. I actually met him >> He did all that work. I actually met him uh in the office uh when we did the uh in the office uh when we did the uh in the office uh when we did the podcast I think and I didn't realize it podcast I think and I didn't realize it podcast I think and I didn't realize it was him. So his badge doesn't say was him. So his badge doesn't say was him. So his badge doesn't say Popers. Popers. Popers. >> Yeah, >> Yeah, >> Yeah, >> I know. >> I know. >> I know. >> So he used to be part of uh Replicate >> So he used to be part of uh Replicate >> So he used to be part of uh Replicate and Replicate had this joke where like and Replicate had this joke where like and Replicate had this joke where like everyone was Deep Fates. Deep Fates is everyone was Deep Fates. Deep Fates is everyone was Deep Fates. Deep Fates is this like kind of mysterious character this like kind of mysterious character this like kind of mysterious character and replicate. Replicate is very cool and replicate. Replicate is very cool and replicate. Replicate is very cool company and both was part of it. Um, so, company and both was part of it. Um, so, company and both was part of it. Um, so, okay, one thing I want to get on there okay, one thing I want to get on there okay, one thing I want to get on there before I go into like sort of the the before I go into like sort of the the before I go into like sort of the the the the sort of omniper is we added the the sort of omniper is we added the the sort of omniper is we added cats, we added sloths, very cool, very cats, we added sloths, very cool, very cats, we added sloths, very cool, very cute, very fun.

cute, very fun. cute, very fun. >> Uh, what are the, you know, inspire >> Uh, what are the, you know, inspire >> Uh, what are the, you know, inspire people as to like what are the more sort people as to like what are the more sort people as to like what are the more sort of workhorse use cases that maybe are of workhorse use cases that maybe are of workhorse use cases that maybe are not just demos, you know? not just demos, you know? not just demos, you know? >> Yeah. So, so obviously the hero >> Yeah. So, so obviously the hero >> Yeah. So, so obviously the hero capability of the model or maybe there's capability of the model or maybe there's capability of the model or maybe there's two like one is the ability to kind of two like one is the ability to kind of two like one is the ability to kind of take in anything as input and then get take in anything as input and then get take in anything as input and then get video on the other side. Obviously in video on the other side. Obviously in video on the other side. Obviously in the future and and we've kind of talked the future and and we've kind of talked the future and and we've kind of talked about this as a pre-announce like we about this as a pre-announce like we about this as a pre-announce like we want to get the other output modalities want to get the other output modalities want to get the other output modalities out as well but basically what that out as well but basically what that out as well but basically what that means is you know you can take a set of means is you know you can take a set of means is you know you can take a set of images that you have as maybe a images that you have as maybe a images that you have as maybe a storyboard. You can take like an audio storyboard. You can take like an audio storyboard. You can take like an audio track as a reference of you know like a track as a reference of you know like a track as a reference of you know like a voice that you want a character to speak voice that you want a character to speak voice that you want a character to speak and then you can get a video on the and then you can get a video on the and then you can get a video on the other side. So like that just unlocks a other side. So like that just unlocks a other side. So like that just unlocks a whole bunch of things that you can do in whole bunch of things that you can do in whole bunch of things that you can do in like you know short film production or like you know short film production or like you know short film production or you know shorts we've launched on you know shorts we've launched on you know shorts we've launched on YouTube as well um to help creators kind YouTube as well um to help creators kind YouTube as well um to help creators kind of like create um content more easily. of like create um content more easily. of like create um content more easily. Um and then the other one is obviously Um and then the other one is obviously Um and then the other one is obviously video editing. Like that's another thing video editing. Like that's another thing video editing. Like that's another thing that we're really excited about that that we're really excited about that that we're really excited about that we're just making easier because now you we're just making easier because now you we're just making easier because now you can use natural language to take a can use natural language to take a can use natural language to take a video, you know, add something, remove video, you know, add something, remove video, you know, add something, remove something. Sloth is obviously like fun something. Sloth is obviously like fun something. Sloth is obviously like fun example. Um, but there there's obviously example. Um, but there there's obviously example. Um, but there there's obviously kind of there's consumer use cases that kind of there's consumer use cases that kind of there's consumer use cases that we kind of had in mind where, you know, we kind of had in mind where, you know, we kind of had in mind where, you know, you could take your beach vacation video you could take your beach vacation video you could take your beach vacation video that was too noisy and you want to clean that was too noisy and you want to clean that was too noisy and you want to clean up that noise. Maybe in the past you up that noise. Maybe in the past you up that noise. Maybe in the past you wouldn't have because you didn't have wouldn't have because you didn't have wouldn't have because you didn't have the tools or you didn't know what the the tools or you didn't know what the the tools or you didn't know what the tools were that you needed to go to. So, tools were that you needed to go to. So, tools were that you needed to go to. So, that's one use case that you can, you that's one use case that you can, you that's one use case that you can, you know, go to. We've seen a lot of folks know, go to. We've seen a lot of folks know, go to. We've seen a lot of folks use it for kind of marketing ad campaign use it for kind of marketing ad campaign use it for kind of marketing ad campaign creation and I'm excited to see more of creation and I'm excited to see more of creation and I'm excited to see more of those use cases as we launch the APIs.

those use cases as we launch the APIs. those use cases as we launch the APIs. um because obviously like we don't we um because obviously like we don't we um because obviously like we don't we don't see all of it in the first party don't see all of it in the first party don't see all of it in the first party products but I'm really excited for products but I'm really excited for products but I'm really excited for people to start to explore that um in people to start to explore that um in people to start to explore that um in the API. So those are just some of the the API. So those are just some of the the API. So those are just some of the kind of like high level um things that kind of like high level um things that kind of like high level um things that have come up. U people also use it to have come up. U people also use it to have come up. U people also use it to create like education materials. Yes. Um create like education materials. Yes. Um create like education materials. Yes. Um and like like that's really exciting. I and like like that's really exciting. I and like like that's really exciting. I think we're all we've all kind of talked think we're all we've all kind of talked think we're all we've all kind of talked about being excited about the future of about being excited about the future of about being excited about the future of education where like everything can be education where like everything can be education where like everything can be kind of customized to you and kind of customized to you and kind of customized to you and personalized to your knowledge level and personalized to your knowledge level and personalized to your knowledge level and the style that you prefer and and so the style that you prefer and and so the style that you prefer and and so this is kind of just like a step in that this is kind of just like a step in that this is kind of just like a step in that direction. direction. direction. >> Yeah. I I I sort of actually used just >> Yeah. I I I sort of actually used just >> Yeah. I I I sort of actually used just none of yesterday, but my my parents are none of yesterday, but my my parents are none of yesterday, but my my parents are visiting and there was there was a very visiting and there was there was a very visiting and there was there was a very fun sort of use case. They I bought some fun sort of use case. They I bought some fun sort of use case. They I bought some gadget off from Amazon that they wanted gadget off from Amazon that they wanted gadget off from Amazon that they wanted and the instructions to use it was were and the instructions to use it was were and the instructions to use it was were only in English and there was plenty of only in English and there was plenty of only in English and there was plenty of diagrams or whatever and I took a diagrams or whatever and I took a diagrams or whatever and I took a picture of it and said, you know, picture of it and said, you know, picture of it and said, you know, translate this into Romanian. Yes. translate this into Romanian. Yes. translate this into Romanian. Yes. >> And keep everything else the same, >> And keep everything else the same, >> And keep everything else the same, right? So it was amazing, right? Like it right? So it was amazing, right? Like it right? So it was amazing, right? Like it was just like, yeah, it looks identical was just like, yeah, it looks identical was just like, yeah, it looks identical and it has, you know, it's perfectly and it has, you know, it's perfectly and it has, you know, it's perfectly translated. I mean, more or less, right? translated. I mean, more or less, right? translated. I mean, more or less, right? But it's it's you know using Gemini But it's it's you know using Gemini But it's it's you know using Gemini under the hood obviously to kind of do under the hood obviously to kind of do under the hood obviously to kind of do the translation and so you can you can the translation and so you can you can the translation and so you can you can see this use case for video as well see this use case for video as well see this use case for video as well right like the the power of text right like the the power of text right like the the power of text rendering in in in Omni is is quite next rendering in in in Omni is is quite next rendering in in in Omni is is quite next level. So and you could you could you level. So and you could you could you level. So and you could you could you could think about plenty of use cases of could think about plenty of use cases of could think about plenty of use cases of like both text rendering translation like both text rendering translation like both text rendering translation internalization all sorts of things that internalization all sorts of things that internalization all sorts of things that would be actually genuinely useful to a would be actually genuinely useful to a would be actually genuinely useful to a lot of different people and sort of lot of different people and sort of lot of different people and sort of broader access to either you could like broader access to either you could like broader access to either you could like redub a video or whatever it is that you redub a video or whatever it is that you redub a video or whatever it is that you wanted to do. like there's plenty of wanted to do. like there's plenty of wanted to do. like there's plenty of different things that you could you different things that you could you different things that you could you could think about doing.

could think about doing. could think about doing. >> Yeah. Um one of the most enlightening >> Yeah. Um one of the most enlightening >> Yeah. Um one of the most enlightening conversations I have on my podcast is conversations I have on my podcast is conversations I have on my podcast is with uh just people researchers at the with uh just people researchers at the with uh just people researchers at the frontier of these things. Um I had one frontier of these things. Um I had one frontier of these things. Um I had one with um Ethan from the XAI video team, with um Ethan from the XAI video team, with um Ethan from the XAI video team, the Grock video team who was basically the Grock video team who was basically the Grock video team who was basically saying like you know the next trend is saying like you know the next trend is saying like you know the next trend is actually not just like single model, actually not just like single model, actually not just like single model, it's more like video agents. Um, and I it's more like video agents. Um, and I it's more like video agents. Um, and I don't know if that terminology resonates don't know if that terminology resonates don't know if that terminology resonates uh obviously for for very relevant for uh obviously for for very relevant for uh obviously for for very relevant for RL. Uh, but it was it was basically kind RL. Uh, but it was it was basically kind RL. Uh, but it was it was basically kind of like giving up on like trying to do of like giving up on like trying to do of like giving up on like trying to do everything in in effectively one pass. everything in in effectively one pass. everything in in effectively one pass. Um, do you feel that same way or is it Um, do you feel that same way or is it Um, do you feel that same way or is it still an open research question which still an open research question which still an open research question which way the trends are going? [snorts] way the trends are going? [snorts] way the trends are going? [snorts] >> Yeah. So um what kind of excite me most >> Yeah. So um what kind of excite me most >> Yeah. So um what kind of excite me most is really when the symbolic kind of is really when the symbolic kind of is really when the symbolic kind of foundational models and this kind of foundational models and this kind of foundational models and this kind of like video foundational model can like video foundational model can like video foundational model can actually kind of really work together actually kind of really work together actually kind of really work together and u in a way the if you look at the and u in a way the if you look at the and u in a way the if you look at the beginning of the generative sort of like beginning of the generative sort of like beginning of the generative sort of like image generation video generation a lot image generation video generation a lot image generation video generation a lot of it kind of started when the language of it kind of started when the language of it kind of started when the language model got good enough to provide a very model got good enough to provide a very model got good enough to provide a very detailed captioning like from stable detailed captioning like from stable detailed captioning like from stable fusion days or kind of dowi 2 days. So fusion days or kind of dowi 2 days. So fusion days or kind of dowi 2 days. So um so basically like language is um so basically like language is um so basically like language is extremely u helpful representation uh extremely u helpful representation uh extremely u helpful representation uh one is that it's kind of universal but one is that it's kind of universal but one is that it's kind of universal but the other kind of more um technical the other kind of more um technical the other kind of more um technical thing like kind of my hypothesis is like thing like kind of my hypothesis is like thing like kind of my hypothesis is like um one very difficult thing about um one very difficult thing about um one very difficult thing about machine learning is um this sort of like machine learning is um this sort of like machine learning is um this sort of like spirious coordination. So you don't know spirious coordination. So you don't know spirious coordination. So you don't know you know if the if this kind of feature you know if the if this kind of feature you know if the if this kind of feature right that's kind of predictive is right that's kind of predictive is right that's kind of predictive is actually causal factor or not. There are actually causal factor or not. There are actually causal factor or not. There are two ways. One is we can have really two ways. One is we can have really two ways. One is we can have really diverse data training data like from

diverse data training data like from diverse data training data like from every intervention of the causal graph. every intervention of the causal graph. every intervention of the causal graph. The other is you condition the causal The other is you condition the causal The other is you condition the causal information and conditioning the information and conditioning the information and conditioning the language is kind of like conditioning language is kind of like conditioning language is kind of like conditioning like a coal information of the of the like a coal information of the of the like a coal information of the of the kind of world. So um kind of world. So um kind of world. So um >> which is a prompt or a concept what >> which is a prompt or a concept what >> which is a prompt or a concept what >> yeah exactly so if you look at like you >> yeah exactly so if you look at like you >> yeah exactly so if you look at like you know how we going to describe this video know how we going to describe this video know how we going to describe this video how this kind of image is actually very how this kind of image is actually very how this kind of image is actually very close to you know how would describe close to you know how would describe close to you know how would describe this kind of causality you know behind this kind of causality you know behind this kind of causality you know behind this like how this is kind of generated. this like how this is kind of generated. this like how this is kind of generated. So one is like that can really allow for So one is like that can really allow for So one is like that can really allow for very rich generalization and then uh very rich generalization and then uh very rich generalization and then uh very kind of just like a good model. Um very kind of just like a good model. Um very kind of just like a good model. Um the other is so eight months ago uh we the other is so eight months ago uh we the other is so eight months ago uh we put the evaluation paper called video put the evaluation paper called video put the evaluation paper called video models zero shot learners and reasoners. models zero shot learners and reasoners. models zero shot learners and reasoners. >> Yes. So that was a kind of you know it's >> Yes. So that was a kind of you know it's >> Yes. So that was a kind of you know it's it's a confirmed paper and then later on it's a confirmed paper and then later on it's a confirmed paper and then later on actually the N banana team followed up actually the N banana team followed up actually the N banana team followed up with a vision banana paper that with a vision banana paper that with a vision banana paper that basically used n banana to do but basically used n banana to do but basically used n banana to do but essentially the idea is uh video model essentially the idea is uh video model essentially the idea is uh video model is extremely good sort of a foundation is extremely good sort of a foundation is extremely good sort of a foundation model for space and time kind of model for space and time kind of model for space and time kind of information. So um classic computer information. So um classic computer information. So um classic computer vision tasks a lot of could be kind of vision tasks a lot of could be kind of vision tasks a lot of could be kind of zero shorted and when you like say feed zero shorted and when you like say feed zero shorted and when you like say feed in some like a visual quiz uh it can you in some like a visual quiz uh it can you in some like a visual quiz uh it can you know there's definitely like a lot to know there's definitely like a lot to know there's definitely like a lot to improve it can kind of solve and it can improve it can kind of solve and it can improve it can kind of solve and it can um like robotics kind of like seeing it um like robotics kind of like seeing it um like robotics kind of like seeing it has really good kind of physical has really good kind of physical has really good kind of physical intuitions like word model uh and I intuitions like word model uh and I intuitions like word model uh and I think the the key is really the kind of think the the key is really the kind of think the the key is really the kind of mix of the visual kind of reasoning and mix of the visual kind of reasoning and mix of the visual kind of reasoning and then the text kind of reasoning kind of then the text kind of reasoning kind of then the text kind of reasoning kind of all tied together Um obviously you know all tied together Um obviously you know all tied together Um obviously you know like whether doing it you know as kind

like whether doing it you know as kind like whether doing it you know as kind of unified model versus like just kind of unified model versus like just kind of unified model versus like just kind of agent coation I think that's more of agent coation I think that's more of agent coation I think that's more like uh it's going to be more kind of like uh it's going to be more kind of like uh it's going to be more kind of incremental you know how it's going to I incremental you know how it's going to I incremental you know how it's going to I imagine everything's going to go into imagine everything's going to go into imagine everything's going to go into like a single model eventually like a single model eventually like a single model eventually >> but right now there's like a lot you can >> but right now there's like a lot you can >> but right now there's like a lot you can do if you uh basically take like really do if you uh basically take like really do if you uh basically take like really good video understanding image good video understanding image good video understanding image understanding Gemini agentically with understanding Gemini agentically with understanding Gemini agentically with anomy and that's actually gonna yeah our anomy and that's actually gonna yeah our anomy and that's actually gonna yeah our team is like exploring a lot team is like exploring a lot team is like exploring a lot >> yeah okay that there's a there's a lot >> yeah okay that there's a there's a lot >> yeah okay that there's a there's a lot in there um I I think uh one question I in there um I I think uh one question I in there um I I think uh one question I I am increasingly starting to wonder is I am increasingly starting to wonder is I am increasingly starting to wonder is does it all trend towards one product does it all trend towards one product does it all trend towards one product for you guys right like now you have for you guys right like now you have for you guys right like now you have multiple models out the naming of omni multiple models out the naming of omni multiple models out the naming of omni does imply that eventually everything does imply that eventually everything does imply that eventually everything will go away and it just goes into omnis will go away and it just goes into omnis will go away and it just goes into omnis um is that the plan um is that the plan um is that the plan >> is it [laughter] I don't know I I think >> is it [laughter] I don't know I I think >> is it [laughter] I don't know I I think I think uh maybe I mean I think I think uh maybe I mean I think I think uh maybe I mean I think eventually I I think there's sort of eventually I I think there's sort of eventually I I think there's sort of different trade-offs engineering different trade-offs engineering different trade-offs engineering research product trade-offs in like it's research product trade-offs in like it's research product trade-offs in like it's like for the same reason like the the like for the same reason like the the like for the same reason like the the sorry how is it called nano banana light sorry how is it called nano banana light sorry how is it called nano banana light I don't know what the product name I don't know what the product name I don't know what the product name >> nanob banana tite >> nanob banana tite >> nanob banana tite >> nano banana too light yeah right it's >> nano banana too light yeah right it's >> nano banana too light yeah right it's it's it's it serves a particular niche it's it's it serves a particular niche it's it's it serves a particular niche right and it probably doesn't right and it probably doesn't right and it probably doesn't necessarily fit immediately in the same necessarily fit immediately in the same necessarily fit immediately in the same model literally checkpoint as uh model literally checkpoint as uh model literally checkpoint as uh something that can do 4K you know uh 30 something that can do 4K you know uh 30 something that can do 4K you know uh 30 secondond videos right like they're secondond videos right like they're secondond videos right like they're probably not like trainable in the same probably not like trainable in the same probably not like trainable in the same quite way, right? Like, so I I don't quite way, right? Like, so I I don't quite way, right? Like, so I I don't know. It depends on how how far into the

know. It depends on how how far into the know. It depends on how how far into the future you look like. Sure, in five future you look like. Sure, in five future you look like. Sure, in five years from now, will they all be the years from now, will they all be the years from now, will they all be the same model? Probably. Uh but like, you same model? Probably. Uh but like, you same model? Probably. Uh but like, you know, six months from now, we'll we'll know, six months from now, we'll we'll know, six months from now, we'll we'll probably still have, you know, multiple probably still have, you know, multiple probably still have, you know, multiple different models doing different things different models doing different things different models doing different things because kind of from pragmatically the because kind of from pragmatically the because kind of from pragmatically the trade-offs are such that we we should trade-offs are such that we we should trade-offs are such that we we should have multiple different kinds of models. have multiple different kinds of models. have multiple different kinds of models. >> Yeah, I >> Yeah, I >> Yeah, I >> I think that's right. And and just on >> I think that's right. And and just on >> I think that's right. And and just on that note, I mean, we did call it Gemini that note, I mean, we did call it Gemini that note, I mean, we did call it Gemini Omni because we wanted to hint at the Omni because we wanted to hint at the Omni because we wanted to hint at the future where Gemini just becomes fully future where Gemini just becomes fully future where Gemini just becomes fully multimodal in and out, right? And so so multimodal in and out, right? And so so multimodal in and out, right? And so so it's definitely a move in that it's definitely a move in that it's definitely a move in that direction. I think we'll probably see a direction. I think we'll probably see a direction. I think we'll probably see a move in the direction where Omni also move in the direction where Omni also move in the direction where Omni also generates images and edits images and generates images and edits images and generates images and edits images and all those kinds of things. But Doo is all those kinds of things. But Doo is all those kinds of things. But Doo is right that I think on the way there, right that I think on the way there, right that I think on the way there, there's a bunch of really really useful there's a bunch of really really useful there's a bunch of really really useful applications of some of these more applications of some of these more applications of some of these more specialized models. And so we we will specialized models. And so we we will specialized models. And so we we will probably continue to work on those as probably continue to work on those as probably continue to work on those as well because like that serves a certain well because like that serves a certain well because like that serves a certain need at this point in time that may not need at this point in time that may not need at this point in time that may not exist you know a year from now. There's exist you know a year from now. There's exist you know a year from now. There's also like a research question about like also like a research question about like also like a research question about like just how much transfer there is between just how much transfer there is between just how much transfer there is between different kinds of modalities, right? I different kinds of modalities, right? I different kinds of modalities, right? I think you may believe that there's some think you may believe that there's some think you may believe that there's some transfer between coding and video transfer between coding and video transfer between coding and video generation and I think most people don't generation and I think most people don't generation and I think most people don't necessarily believe that but they you necessarily believe that but they you necessarily believe that but they you know you could try to think that there know you could try to think that there know you could try to think that there is some some there something there or it is some some there something there or it is some some there something there or it could be a waste right to put them could be a waste right to put them could be a waste right to put them together to try to learn these both together to try to learn these both together to try to learn these both tasks at the same time right so I think tasks at the same time right so I think tasks at the same time right so I think it's it's it's interesting sort of it's it's it's interesting sort of it's it's it's interesting sort of question to which extent like image and question to which extent like image and question to which extent like image and video obviously kind of there's some video obviously kind of there's some video obviously kind of there's some transfer like kind of not that different transfer like kind of not that different transfer like kind of not that different there's value in in learning to output there's value in in learning to output there's value in in learning to output video and audio at the same time because video and audio at the same time because video and audio at the same time because joint audio visual is you know that's joint audio visual is you know that's joint audio visual is you know that's how that's how it is. Um and then

how that's how it is. Um and then how that's how it is. Um and then there's you know other kind of there's you know other kind of there's you know other kind of intersections of modalities that are not intersections of modalities that are not intersections of modalities that are not super obvious right like 3D super obvious right like 3D super obvious right like 3D representation coding I don't know maybe representation coding I don't know maybe representation coding I don't know maybe uh things like that right so like I uh things like that right so like I uh things like that right so like I think it's worth sort of exploring the think it's worth sort of exploring the think it's worth sort of exploring the different corners there and we are different corners there and we are different corners there and we are actively doing that um with a focus actively doing that um with a focus actively doing that um with a focus towards like what people actually want towards like what people actually want towards like what people actually want to do with these models to do with these models to do with these models >> yeah um what one thing I feel I feel >> yeah um what one thing I feel I feel >> yeah um what one thing I feel I feel like uh I'm surprised by but also I feel like uh I'm surprised by but also I feel like uh I'm surprised by but also I feel like it's insufficiently answered is like it's insufficiently answered is like it's insufficiently answered is what is the correct intermediate what is the correct intermediate what is the correct intermediate representation Um, so captioning, right? representation Um, so captioning, right? representation Um, so captioning, right? XI does captioning. Omni does XI does captioning. Omni does XI does captioning. Omni does captioning. Um, and I I I understand how captioning. Um, and I I I understand how captioning. Um, and I I I understand how captioning works for images. Um, and I captioning works for images. Um, and I captioning works for images. Um, and I understand that you can extend it into understand that you can extend it into understand that you can extend it into to video and and sort of guide it across to video and and sort of guide it across to video and and sort of guide it across time. It just feels very inefficient. It time. It just feels very inefficient. It time. It just feels very inefficient. It there's got to be I feel like there there's got to be I feel like there there's got to be I feel like there should be something better. Uh maybe should be something better. Uh maybe should be something better. Uh maybe it's code and maybe we generate you know it's code and maybe we generate you know it's code and maybe we generate you know and obviously I think a lot of um ffmpeg and obviously I think a lot of um ffmpeg and obviously I think a lot of um ffmpeg and mapplot um what's the three blue one and mapplot um what's the three blue one and mapplot um what's the three blue one brown one manm um a lot of like video is brown one manm um a lot of like video is brown one manm um a lot of like video is generated through code and maybe that's generated through code and maybe that's generated through code and maybe that's like the optimal representation uh any like the optimal representation uh any like the optimal representation uh any hypothesis as to like is is it better or hypothesis as to like is is it better or hypothesis as to like is is it better or is just English all you need is just English all you need is just English all you need >> well as so I'm in the Gemini and they >> well as so I'm in the Gemini and they >> well as so I'm in the Gemini and they know we do like a lot of RL agent and of know we do like a lot of RL agent and of know we do like a lot of RL agent and of course kind of coding so yeah We we're course kind of coding so yeah We we're course kind of coding so yeah We we're definitely exploring the coding definitely exploring the coding definitely exploring the coding representations.

representations. representations. >> Yeah. >> Yeah. >> Yeah. >> As kind of better kind of way to >> As kind of better kind of way to >> As kind of better kind of way to represent. Yeah. represent. Yeah. represent. Yeah. >> But you know like do you what's your >> But you know like do you what's your >> But you know like do you what's your probability estimate on like [laughter] probability estimate on like [laughter] probability estimate on like [laughter] if we just output binaries like we just if we just output binaries like we just if we just output binaries like we just you know like just it's just ones and you know like just it's just ones and you know like just it's just ones and zeros. zeros. zeros. >> Um I I guess maybe a kind of similar >> Um I I guess maybe a kind of similar >> Um I I guess maybe a kind of similar discussion was like um basically is the discussion was like um basically is the discussion was like um basically is the language the right representation like language the right representation like language the right representation like right. So uh one kind of question for right. So uh one kind of question for right. So uh one kind of question for example uh professor you know like some example uh professor you know like some example uh professor you know like some ask is like you know why why does the ask is like you know why why does the ask is like you know why why does the channel of thought need to be in the channel of thought need to be in the channel of thought need to be in the natural language? natural language? natural language? >> Yes. >> Yes. >> Yes. >> Can it just be the kind of any kind of >> Can it just be the kind of any kind of >> Can it just be the kind of any kind of like continuous tokens just any amount like continuous tokens just any amount like continuous tokens just any amount of you know additional computations. of you know additional computations. of you know additional computations. >> Um so one is like obviously the test >> Um so one is like obviously the test >> Um so one is like obviously the test like adaptive compute is going to give like adaptive compute is going to give like adaptive compute is going to give like you know better results. So it's like you know better results. So it's like you know better results. So it's that but what really kind of made CH that but what really kind of made CH that but what really kind of made CH thought so you know like four years ago thought so you know like four years ago thought so you know like four years ago I wrote you know the larger model zero I wrote you know the larger model zero I wrote you know the larger model zero sort reasoner and then self-improvement. sort reasoner and then self-improvement. sort reasoner and then self-improvement. So I kind of know from the very early So I kind of know from the very early So I kind of know from the very early day but the reason like it works really day but the reason like it works really day but the reason like it works really well is um right now the recipe that well is um right now the recipe that well is um right now the recipe that works is the pre-training that scales a works is the pre-training that scales a works is the pre-training that scales a lot and then that basically like learns lot and then that basically like learns lot and then that basically like learns a lot of intelligence. there are a lot a lot of intelligence. there are a lot a lot of intelligence. there are a lot of you know scaling RL but those are of you know scaling RL but those are of you know scaling RL but those are still like extremely kind of comput still like extremely kind of comput still like extremely kind of comput incent intensive to extract the incent intensive to extract the incent intensive to extract the information and um you really want to information and um you really want to information and um you really want to rely the intelligence on that so rely the intelligence on that so rely the intelligence on that so basically by tying the sort of like a basically by tying the sort of like a basically by tying the sort of like a reasoning in the natural language you reasoning in the natural language you reasoning in the natural language you basically directly use the intelligence basically directly use the intelligence basically directly use the intelligence of the pre-training to it while if you of the pre-training to it while if you of the pre-training to it while if you remove that kind of constraints then remove that kind of constraints then remove that kind of constraints then you're not um and these days uh I feel

you're not um and these days uh I feel you're not um and these days uh I feel the a lot of advancements in the texts the a lot of advancements in the texts the a lot of advancements in the texts but also doing this kind of multimodal but also doing this kind of multimodal but also doing this kind of multimodal space is very driven by this uh kind of space is very driven by this uh kind of space is very driven by this uh kind of text as a kind of great uh sort of text as a kind of great uh sort of text as a kind of great uh sort of representation. representation. representation. >> Yeah, it's a good backbone. >> Yeah, it's a good backbone. >> Yeah, it's a good backbone. >> Yeah, >> Yeah, >> Yeah, >> I I think to me it's even simpler than >> I I think to me it's even simpler than >> I I think to me it's even simpler than that. It's text is is how we that. It's text is is how we that. It's text is is how we communicate. So I think fundamentally if communicate. So I think fundamentally if communicate. So I think fundamentally if you're building kind of products that you're building kind of products that you're building kind of products that humans will be interfacing with um like humans will be interfacing with um like humans will be interfacing with um like like that we will be using text somehow like that we will be using text somehow like that we will be using text somehow if it's a text interface, right? Not not if it's a text interface, right? Not not if it's a text interface, right? Not not for everything. So I think it's it's for everything. So I think it's it's for everything. So I think it's it's natural to default to that. natural to default to that. natural to default to that. >> Yeah. Obviously there's like a conf >> Yeah. Obviously there's like a conf >> Yeah. Obviously there's like a conf discussion you know some arrow like discussion you know some arrow like discussion you know some arrow like arrow maximalists is like oh we don't arrow maximalists is like oh we don't arrow maximalists is like oh we don't care about you know kind of channel care about you know kind of channel care about you know kind of channel those kind of like stuff it's just just those kind of like stuff it's just just those kind of like stuff it's just just additional compute additional compute additional compute >> sure but I personally yeah >> sure but I personally yeah >> sure but I personally yeah >> ro maximalists I wonder I wonder who who >> ro maximalists I wonder I wonder who who >> ro maximalists I wonder I wonder who who qualifies in that description David qualifies in that description David qualifies in that description David silver silver silver >> ah okay yeah I mean they they've just >> ah okay yeah I mean they they've just >> ah okay yeah I mean they they've just left to to start their thing um left to to start their thing um left to to start their thing um interesting okay so uh I I mean I think interesting okay so uh I I mean I think interesting okay so uh I I mean I think I'm very interested in just like better I'm very interested in just like better I'm very interested in just like better representations because I think that's representations because I think that's representations because I think that's one of our themes that we're curating one of our themes that we're curating one of our themes that we're curating today uh at the world fair is world today uh at the world fair is world today uh at the world fair is world models. You mentioned the word world models. You mentioned the word world models. You mentioned the word world models but it's not something that's models but it's not something that's models but it's not something that's like super well defined. I think like super well defined. I think like super well defined. I think everyone's like sort of converging on everyone's like sort of converging on everyone's like sort of converging on some version of it that is like the some version of it that is like the some version of it that is like the ideal.

ideal. ideal. >> Sure. Everything is a world model now. >> Sure. Everything is a world model now. >> Sure. Everything is a world model now. It's sort of a It's sort of a It's sort of a >> it's not it's not that useful, right? >> it's not it's not that useful, right? >> it's not it's not that useful, right? >> So I just gave a keynote at the IER >> So I just gave a keynote at the IER >> So I just gave a keynote at the IER world model workshop. Yeah. And then uh world model workshop. Yeah. And then uh world model workshop. Yeah. And then uh yeah essentially uh I definitely yeah essentially uh I definitely yeah essentially uh I definitely encourage to check out the definition by encourage to check out the definition by encourage to check out the definition by Jandra Matic. He's like the you know OG Jandra Matic. He's like the you know OG Jandra Matic. He's like the you know OG computer vision professor UC Berkeley. computer vision professor UC Berkeley. computer vision professor UC Berkeley. >> Uh he has pretty you know bit of word to >> Uh he has pretty you know bit of word to >> Uh he has pretty you know bit of word to say about world model say about world model say about world model >> but also kind of Schmidt Herburver's >> but also kind of Schmidt Herburver's >> but also kind of Schmidt Herburver's kind of how he defined the world model kind of how he defined the world model kind of how he defined the world model from 2019 like 1990 sort of uh uh you from 2019 like 1990 sort of uh uh you from 2019 like 1990 sort of uh uh you know like Wayne was just basically just know like Wayne was just basically just know like Wayne was just basically just that kind of model base. Uh for me the that kind of model base. Uh for me the that kind of model base. Uh for me the word model is basically just the model word model is basically just the model word model is basically just the model in the model based RL and I feel that in the model based RL and I feel that in the model based RL and I feel that has sufficient to describe but obviously has sufficient to describe but obviously has sufficient to describe but obviously you know there are like a lot of uh FE you know there are like a lot of uh FE you know there are like a lot of uh FE had a kind of nice blog post about what had a kind of nice blog post about what had a kind of nice blog post about what about yeah this kind of broken down about yeah this kind of broken down about yeah this kind of broken down >> um but yeah >> um but yeah >> um but yeah >> yeah I mean so you know I I'll end this >> yeah I mean so you know I I'll end this >> yeah I mean so you know I I'll end this part of the conversation but like I I do part of the conversation but like I I do part of the conversation but like I I do think that language to me relying on think that language to me relying on think that language to me relying on language as like the sort of like the language as like the sort of like the language as like the sort of like the narrow pipe through which everything narrow pipe through which everything narrow pipe through which everything goes through um still is like a lossy goes through um still is like a lossy goes through um still is like a lossy compression. No, no, no. But we're not compression. No, no, no. But we're not compression. No, no, no. But we're not seeing that, right? We're basically seeing that, right? We're basically seeing that, right? We're basically saying the video model and the language saying the video model and the language saying the video model and the language together.

together. together. >> So, so I think the language alone is uh >> So, so I think the language alone is uh >> So, so I think the language alone is uh not sufficient. That's why we feel like not sufficient. That's why we feel like not sufficient. That's why we feel like the video is a very complement model. the video is a very complement model. the video is a very complement model. Right? Now the um you know kind of v Right? Now the um you know kind of v Right? Now the um you know kind of v omni many people feel as uh you know omni many people feel as uh you know omni many people feel as uh you know generating kind of pretty videos but I generating kind of pretty videos but I generating kind of pretty videos but I think our vision it's it's much more think our vision it's it's much more think our vision it's it's much more than that. It's a missing foundational than that. It's a missing foundational than that. It's a missing foundational model that's absolutely required if you model that's absolutely required if you model that's absolutely required if you want to make the AGI that match to want to make the AGI that match to want to make the AGI that match to humans not just a jacked one. humans not just a jacked one. humans not just a jacked one. >> Yeah. Um okay. So one one other thing >> Yeah. Um okay. So one one other thing >> Yeah. Um okay. So one one other thing you know you you mentioned on the vision you know you you mentioned on the vision you know you you mentioned on the vision side um and I'm kind of curious how sort side um and I'm kind of curious how sort side um and I'm kind of curious how sort of uh parallel you know in terms of your of uh parallel you know in terms of your of uh parallel you know in terms of your research careers um this development is research careers um this development is research careers um this development is like I think basically a lot of vision like I think basically a lot of vision like I think basically a lot of vision people have crossed over into more model people have crossed over into more model people have crossed over into more model people um a lot of vision people also people um a lot of vision people also people um a lot of vision people also become generative video and image people become generative video and image people become generative video and image people and is it just as simple as you know and is it just as simple as you know and is it just as simple as you know reversing uh image to text and then now reversing uh image to text and then now reversing uh image to text and then now it's text to image like it's text to image like it's text to image like >> [laughter] >> [laughter] >> [laughter] >> is is that if I mean that effectively >> is is that if I mean that effectively >> is is that if I mean that effectively was the diffusion process. Um was the diffusion process. Um was the diffusion process. Um I I just you know I I just see the I I just you know I I just see the I I just you know I I just see the career paths of the people that I talked career paths of the people that I talked career paths of the people that I talked to and and see and I I I see this to and and see and I I I see this to and and see and I I I see this overall trend of research directions and overall trend of research directions and overall trend of research directions and I just wanted you to guys to sort of I just wanted you to guys to sort of I just wanted you to guys to sort of reflect on on that.

reflect on on that. reflect on on that. >> I mean I certainly went that way right I >> I mean I certainly went that way right I >> I mean I certainly went that way right I I started long time ago uh doing I started long time ago uh doing I started long time ago uh doing computer vision sort of object detection computer vision sort of object detection computer vision sort of object detection recognition things like that. Uh I think recognition things like that. Uh I think recognition things like that. Uh I think just that's just simpler problem right just that's just simpler problem right just that's just simpler problem right just generation is just harder like it's just generation is just harder like it's just generation is just harder like it's a it's a different kind of mapping right a it's a different kind of mapping right a it's a different kind of mapping right you map from the the inverse mapping is you map from the the inverse mapping is you map from the the inverse mapping is not as simple as just inverting the the not as simple as just inverting the the not as simple as just inverting the the kind of network you use right it's it's kind of network you use right it's it's kind of network you use right it's it's a it's it's more ambiguous right to go a it's it's more ambiguous right to go a it's it's more ambiguous right to go from cat to image of a cat and in some from cat to image of a cat and in some from cat to image of a cat and in some ways it's also a loop because your ways it's also a loop because your ways it's also a loop because your vision work creates the synthetic labels vision work creates the synthetic labels vision work creates the synthetic labels that then continues that then continues that then continues >> I mean sure [laughter] >> I mean sure [laughter] >> I mean sure [laughter] I don't know I don't know I try to I don't know I don't know I try to I don't know I don't know I try to validate my my sort of theories about validate my my sort of theories about validate my my sort of theories about how fields develop how how careers has how fields develop how how careers has how fields develop how how careers has progressed through this progressed through this progressed through this >> I mean for like the the the better the >> I mean for like the the the better the >> I mean for like the the the better the understanding side gets like we have understanding side gets like we have understanding side gets like we have seen that the generation side also gets seen that the generation side also gets seen that the generation side also gets better right so like like better right so like like better right so like like >> it's completely bootstrapping yeah it's >> it's completely bootstrapping yeah it's >> it's completely bootstrapping yeah it's >> and so like like like there's definitely >> and so like like like there's definitely >> and so like like like there's definitely they're there to that thesis and I think they're there to that thesis and I think they're there to that thesis and I think yeah I think a lot of people have kind yeah I think a lot of people have kind yeah I think a lot of people have kind of like I I definitely worked with a lot of like I I definitely worked with a lot of like I I definitely worked with a lot of um image understanding people who of um image understanding people who of um image understanding people who became image generation people you know became image generation people you know became image generation people you know and then some of them have moved on to and then some of them have moved on to and then some of them have moved on to video because it's kind of like the next video because it's kind of like the next video because it's kind of like the next thing where you have so many more thing where you have so many more thing where you have so many more dimensions to work with so yeah I'm dimensions to work with so yeah I'm dimensions to work with so yeah I'm curious about you spec as your curious about you spec as your curious about you spec as your >> so I definitely like recommend start >> so I definitely like recommend start >> so I definitely like recommend start with understanding recognition because with understanding recognition because with understanding recognition because that's basically discriminator and then that's basically discriminator and then that's basically discriminator and then that's going to lead to better that's going to lead to better that's going to lead to better generation and that's what the bridge is generation and that's what the bridge is generation and that's what the bridge is basically reinforcement learning so my basically reinforcement learning so my basically reinforcement learning so my um my kind of journey is I initially um my kind of journey is I initially um my kind of journey is I initially kind of worked on the algorithmic kind of worked on the algorithmic kind of worked on the algorithmic research in the gent model against some research in the gent model against some research in the gent model against some like you know amnest kind of generation like you know amnest kind of generation like you know amnest kind of generation and then I worked on like RL and and then I worked on like RL and and then I worked on like RL and robotics um and then like six years ago

robotics um and then like six years ago robotics um and then like six years ago I was like leading like a moonshot on I was like leading like a moonshot on I was like leading like a moonshot on the dexterity it was pretty early but I the dexterity it was pretty early but I the dexterity it was pretty early but I see now everyone's kind of doing see now everyone's kind of doing see now everyone's kind of doing uh four years ago I basically kind of uh four years ago I basically kind of uh four years ago I basically kind of figured out that this like symbolic AGI figured out that this like symbolic AGI figured out that this like symbolic AGI is going to accelerate much faster than is going to accelerate much faster than is going to accelerate much faster than the kind of physical AGI kind of the kind of physical AGI kind of the kind of physical AGI kind of counterpart. So uh I decided to kind of counterpart. So uh I decided to kind of counterpart. So uh I decided to kind of like language models and then those like language models and then those like language models and then those things. Um and then recently kind of things. Um and then recently kind of things. Um and then recently kind of work with Doomi and then like omni team work with Doomi and then like omni team work with Doomi and then like omni team I quite enjoy kind of collaboration I quite enjoy kind of collaboration I quite enjoy kind of collaboration there. the what I quite enjoy uh what I there. the what I quite enjoy uh what I there. the what I quite enjoy uh what I recommend definitely to the researcher recommend definitely to the researcher recommend definitely to the researcher is to uh definitely kind of explore or is to uh definitely kind of explore or is to uh definitely kind of explore or at least like get exposure to what the at least like get exposure to what the at least like get exposure to what the top people in each of the community are top people in each of the community are top people in each of the community are like looking at how they kind of think like looking at how they kind of think like looking at how they kind of think about problems. So when I look at the about problems. So when I look at the about problems. So when I look at the video model to me it kind of reminds me video model to me it kind of reminds me video model to me it kind of reminds me like pretty early on sort of like like pretty early on sort of like like pretty early on sort of like language model where like very early language model where like very early language model where like very early language model was a kind of creative language model was a kind of creative language model was a kind of creative sort of demo right you kind of like try sort of demo right you kind of like try sort of demo right you kind of like try to write like a story like mobile and to write like a story like mobile and to write like a story like mobile and then like you know GBD2 and then those then like you know GBD2 and then those then like you know GBD2 and then those kind of days like LTM kind of days right kind of days like LTM kind of days right kind of days like LTM kind of days right and then you know uh instruction tuning and then you know uh instruction tuning and then you know uh instruction tuning you actually kind of make it usable as a you actually kind of make it usable as a you actually kind of make it usable as a chatbot but then at the chatbot stage it chatbot but then at the chatbot stage it chatbot but then at the chatbot stage it still had so much hallucinations and still had so much hallucinations and still had so much hallucinations and instruction for wasn't good enough so it instruction for wasn't good enough so it instruction for wasn't good enough so it couldn't use for reasoning and when I couldn't use for reasoning and when I couldn't use for reasoning and when I got good enough um in pre-training and got good enough um in pre-training and got good enough um in pre-training and post- trainining for reasoning then you post- trainining for reasoning then you post- trainining for reasoning then you know this kind of test time scaling the know this kind of test time scaling the know this kind of test time scaling the RL really took off to like many of the RL really took off to like many of the RL really took off to like many of the kind of best performing models and right kind of best performing models and right kind of best performing models and right now I think the video model is as we now I think the video model is as we now I think the video model is as we mentioned it's it is a complimentary mentioned it's it is a complimentary mentioned it's it is a complimentary foundational model and I can imagine foundational model and I can imagine foundational model and I can imagine it's going to follow a similar path it's it's going to follow a similar path it's it's going to follow a similar path it's going to be very uh it's going to

going to be very uh it's going to going to be very uh it's going to improve a lot instruction following a improve a lot instruction following a improve a lot instruction following a lot of uh this it's going to improve a lot of uh this it's going to improve a lot of uh this it's going to improve a lot in reducing coordinations to extend lot in reducing coordinations to extend lot in reducing coordinations to extend that it become a very reliable world that it become a very reliable world that it become a very reliable world model so we can kind of like intermixed model so we can kind of like intermixed model so we can kind of like intermixed video like space-time simulation was a video like space-time simulation was a video like space-time simulation was a text simulation to solve like arbitrary text simulation to solve like arbitrary text simulation to solve like arbitrary AGI problems. Also like I think the AGI problems. Also like I think the AGI problems. Also like I think the difference still is between sort of text difference still is between sort of text difference still is between sort of text models and like image video models is models and like image video models is models and like image video models is that like we haven't quite unified that like we haven't quite unified that like we haven't quite unified understanding and generation in in understanding and generation in in understanding and generation in in multimedia I'd say yet like I mean I multimedia I'd say yet like I mean I multimedia I'd say yet like I mean I think I think without going to the think I think without going to the think I think without going to the details of course there's like it details of course there's like it details of course there's like it depends on on at which level you're depends on on at which level you're depends on on at which level you're thinking about this but generally like thinking about this but generally like thinking about this but generally like there's not that many as far as I know there's not that many as far as I know there's not that many as far as I know models sot kind of you know frontier models sot kind of you know frontier models sot kind of you know frontier models that are genuinely models that are genuinely models that are genuinely kind of good at both understanding and kind of good at both understanding and kind of good at both understanding and generation of of let's videos, right? generation of of let's videos, right? generation of of let's videos, right? Like it's a it's a it's an interesting Like it's a it's a it's an interesting Like it's a it's a it's an interesting challenge. I'm not saying that we should challenge. I'm not saying that we should challenge. I'm not saying that we should do this. Uh but but I think uh it kind do this. Uh but but I think uh it kind do this. Uh but but I think uh it kind of stands to reason that like you know of stands to reason that like you know of stands to reason that like you know understanding and generation are two understanding and generation are two understanding and generation are two sides of the same coin. So they they sides of the same coin. So they they sides of the same coin. So they they kind of should be in the same model in kind of should be in the same model in kind of should be in the same model in some ways. Uh but we don't necessarily some ways. Uh but we don't necessarily some ways. Uh but we don't necessarily always do that. So yeah.

always do that. So yeah. always do that. So yeah. >> Uh you mentioned audio as well, right? >> Uh you mentioned audio as well, right? >> Uh you mentioned audio as well, right? Yeah. Uh is that as hard as video or Yeah. Uh is that as hard as video or Yeah. Uh is that as hard as video or qualitatively different? If if so, in qualitatively different? If if so, in qualitatively different? If if so, in what way? Uh, one of the interesting what way? Uh, one of the interesting what way? Uh, one of the interesting directions three years ago was people directions three years ago was people directions three years ago was people using um, I guess diffusion to do audio using um, I guess diffusion to do audio using um, I guess diffusion to do audio uh, as in like the the sort of refusion uh, as in like the the sort of refusion uh, as in like the the sort of refusion approach. I don't know if you you guys approach. I don't know if you you guys approach. I don't know if you you guys saw that. Um, and I just think it's like saw that. Um, and I just think it's like saw that. Um, and I just think it's like very interesting if a modality that we very interesting if a modality that we very interesting if a modality that we perceive which is audio is different perceive which is audio is different perceive which is audio is different than video actually two machines is than video actually two machines is than video actually two machines is exactly the same like there's they see exactly the same like there's they see exactly the same like there's they see no difference. no difference. no difference. I mean I think on a technical level I mean I think on a technical level I mean I think on a technical level there are some differences but I think there are some differences but I think there are some differences but I think they're like relatively minor. I think they're like relatively minor. I think they're like relatively minor. I think from my perspective audio came into into from my perspective audio came into into from my perspective audio came into into my life when we shipped V3 which was I my life when we shipped V3 which was I my life when we shipped V3 which was I believe the first model that did like a believe the first model that did like a believe the first model that did like a joint joint joint >> with the slicing of the >> with the slicing of the >> with the slicing of the >> Yeah. Yeah. the gold bars or whatever. >> Yeah. Yeah. the gold bars or whatever. >> Yeah. Yeah. the gold bars or whatever. Um it it was the first model that did Um it it was the first model that did Um it it was the first model that did this sort of joint audiovisisual this sort of joint audiovisisual this sort of joint audiovisisual generation. Yes. uh like in the in the I generation. Yes. uh like in the in the I generation. Yes. uh like in the in the I mean there are there were other models mean there are there were other models mean there are there were other models that did kind of you know kind of kind that did kind of you know kind of kind that did kind of you know kind of kind of agentic hacking under the hood but of agentic hacking under the hood but of agentic hacking under the hood but this one was truly sort of you know this one was truly sort of you know this one was truly sort of you know generating everything at once and we the generating everything at once and we the generating everything at once and we the reason we did that is because we felt reason we did that is because we felt reason we did that is because we felt and I think was the right choice we felt and I think was the right choice we felt and I think was the right choice we felt that like uh it only makes sense to that like uh it only makes sense to that like uh it only makes sense to generate them at the same time because generate them at the same time because generate them at the same time because there sort of kind of like from a there sort of kind of like from a there sort of kind of like from a machine learning perspective there's one machine learning perspective there's one machine learning perspective there's one latent kind of you know causal kind of latent kind of you know causal kind of latent kind of you know causal kind of you know generative process right like you know generative process right like you know generative process right like there's something that generates you there's something that generates you there's something that generates you speaking it's not the pixels and then speaking it's not the pixels and then speaking it's not the pixels and then the the audio or somehow somehow

the the audio or somehow somehow the the audio or somehow somehow generated by some other process like the generated by some other process like the generated by some other process like the lips have to move in sync with with the lips have to move in sync with with the lips have to move in sync with with the with the audio, right? So, I think that with the audio, right? So, I think that with the audio, right? So, I think that that solved a lot of the issues that that solved a lot of the issues that that solved a lot of the issues that previous models had or the way that previous models had or the way that previous models had or the way that people did video generation before where people did video generation before where people did video generation before where it was like, okay, we generate the it was like, okay, we generate the it was like, okay, we generate the pixels and then we're going to hack pixels and then we're going to hack pixels and then we're going to hack something on top of it that like moves something on top of it that like moves something on top of it that like moves the lips with the audio that we the lips with the audio that we the lips with the audio that we generate. And that's was very bad. generate. And that's was very bad. generate. And that's was very bad. [laughter] [laughter] [laughter] And so, I think I think that was that's And so, I think I think that was that's And so, I think I think that was that's to me that's the the I mean after V3 to me that's the the I mean after V3 to me that's the the I mean after V3 like you know people were like what do like you know people were like what do like you know people were like what do you mean like there's no audio in your you mean like there's no audio in your you mean like there's no audio in your model? like that makes no sense like model? like that makes no sense like model? like that makes no sense like once it's there like you you have to once it's there like you you have to once it's there like you you have to have it. So I think that was that was have it. So I think that was that was have it. So I think that was that was the right choice and doing it to one the right choice and doing it to one the right choice and doing it to one single generative model I think was was single generative model I think was was single generative model I think was was the right choice. the right choice. the right choice. >> One thing I kind of want to also can ask >> One thing I kind of want to also can ask >> One thing I kind of want to also can ask you guys an opinion as well once one you guys an opinion as well once one you guys an opinion as well once one difference I find the audio and then difference I find the audio and then difference I find the audio and then against the image and video is like the against the image and video is like the against the image and video is like the audio information is less verbalized. I audio information is less verbalized. I audio information is less verbalized. I mean of course the TTS and stuff is mean of course the TTS and stuff is mean of course the TTS and stuff is trivial right but the when you get her trivial right but the when you get her trivial right but the when you get her outside like how to describe music how outside like how to describe music how outside like how to describe music how do you describe this like this person's do you describe this like this person's do you describe this like this person's tone kind of pitch I feel the sort of tone kind of pitch I feel the sort of tone kind of pitch I feel the sort of the verbalization is insufficient and the verbalization is insufficient and the verbalization is insufficient and the interesting thing is that you kind the interesting thing is that you kind the interesting thing is that you kind of see that in two other things like of see that in two other things like of see that in two other things like taste taste sense and also uh say um taste taste sense and also uh say um taste taste sense and also uh say um touch touch touch >> like smell and then the another >> like smell and then the another >> like smell and then the another interesting thing is the skin color so interesting thing is the skin color so interesting thing is the skin color so skin color the the language is pretty skin color the the language is pretty skin color the the language is pretty limited to describe the skin color and limited to describe the skin color and limited to describe the skin color and the reason is that we're extremely uh the reason is that we're extremely uh the reason is that we're extremely uh sensitive to the small difference sensitive to the small difference sensitive to the small difference perturvations or not skin color because perturvations or not skin color because perturvations or not skin color because that basically shows us is this person that basically shows us is this person that basically shows us is this person going to kill me or is can I befriend going to kill me or is can I befriend going to kill me or is can I befriend this person kind of those kind of this person kind of those kind of this person kind of those kind of information and then I feel the smell

information and then I feel the smell information and then I feel the smell tastes um skin color and like sound kind tastes um skin color and like sound kind tastes um skin color and like sound kind of stuff is very very tied into of stuff is very very tied into of stuff is very very tied into primitive a like survival kind of stuff primitive a like survival kind of stuff primitive a like survival kind of stuff and so our sort of sensory system is so and so our sort of sensory system is so and so our sort of sensory system is so sensitive that it's intractable to Um, sensitive that it's intractable to Um, sensitive that it's intractable to Um, so for example, I asked like one the so for example, I asked like one the so for example, I asked like one the wine sort of taster and then like wine sort of taster and then like wine sort of taster and then like professional and then he basically said professional and then he basically said professional and then he basically said he kind of use like a language from like he kind of use like a language from like he kind of use like a language from like a dating, you know, describing like a, a dating, you know, describing like a, a dating, you know, describing like a, you know, partner as a way to describe you know, partner as a way to describe you know, partner as a way to describe the taste because there's no sufficient the taste because there's no sufficient the taste because there's no sufficient vocab to describe. Um, so I'm kind of vocab to describe. Um, so I'm kind of vocab to describe. Um, so I'm kind of curious. Yeah. Do you guys feel that? curious. Yeah. Do you guys feel that? curious. Yeah. Do you guys feel that? >> I think well to some extent I think the >> I think well to some extent I think the >> I think well to some extent I think the same is true for visual information, same is true for visual information, same is true for visual information, right? when you think about like a right? when you think about like a right? when you think about like a certain style or a certain aesthetic, certain style or a certain aesthetic, certain style or a certain aesthetic, right? Like like there are some people right? Like like there are some people right? Like like there are some people who just have a much more kind of who just have a much more kind of who just have a much more kind of developed like whether it's palette or developed like whether it's palette or developed like whether it's palette or kind of visual taste and aesthetic, kind of visual taste and aesthetic, kind of visual taste and aesthetic, right? Like I I think language just right? Like I I think language just right? Like I I think language just tends to be a bit of a limiting factor tends to be a bit of a limiting factor tends to be a bit of a limiting factor when you are trying to describe any of when you are trying to describe any of when you are trying to describe any of these things that like we experience these things that like we experience these things that like we experience with sensory information. And to your with sensory information. And to your with sensory information. And to your point earlier, I think that is the kind point earlier, I think that is the kind point earlier, I think that is the kind of the reason why we are investing in of the reason why we are investing in of the reason why we are investing in world models and why we are pushing on world models and why we are pushing on world models and why we are pushing on kind of the like perception and like kind of the like perception and like kind of the like perception and like generation side of things because it it generation side of things because it it generation side of things because it it is such a large part of how we as humans is such a large part of how we as humans is such a large part of how we as humans navigate the world. It's a large part of navigate the world. It's a large part of navigate the world. It's a large part of how like embodied AI navigates the how like embodied AI navigates the how like embodied AI navigates the world. Um, and and I do I do think world. Um, and and I do I do think world. Um, and and I do I do think language like does have a lot of it's language like does have a lot of it's language like does have a lot of it's it's gotten us very far and it can it's gotten us very far and it can it's gotten us very far and it can probably get us really far, but it it

probably get us really far, but it it probably get us really far, but it it feels limiting in a lot of these kind of feels limiting in a lot of these kind of feels limiting in a lot of these kind of areas. And yeah, I don't I don't really areas. And yeah, I don't I don't really areas. And yeah, I don't I don't really know how to describe, you know, like know how to describe, you know, like know how to describe, you know, like sense and taste. Um, but yeah, I'm sense and taste. Um, but yeah, I'm sense and taste. Um, but yeah, I'm curious to me. curious to me. curious to me. >> Um, I I yeah, I don't know that I have >> Um, I I yeah, I don't know that I have >> Um, I I yeah, I don't know that I have thought that deeply about this yet. So, thought that deeply about this yet. So, thought that deeply about this yet. So, uh, yeah, I mean yeah, I don't have a uh, yeah, I mean yeah, I don't have a uh, yeah, I mean yeah, I don't have a good answer about audio. I mean like I good answer about audio. I mean like I good answer about audio. I mean like I don't know the limit because I'm don't know the limit because I'm don't know the limit because I'm thinking about like well what is what is thinking about like well what is what is thinking about like well what is what is Omni bad at in terms of audio but Omni bad at in terms of audio but Omni bad at in terms of audio but they're all like solvable problems I they're all like solvable problems I they're all like solvable problems I find uh so like with more data or better find uh so like with more data or better find uh so like with more data or better data or whatever it is so I don't know data or whatever it is so I don't know data or whatever it is so I don't know like that we have pushed the frontier so like that we have pushed the frontier so like that we have pushed the frontier so much that like we are have hit some sort much that like we are have hit some sort much that like we are have hit some sort of limits that are rooted in of limits that are rooted in of limits that are rooted in evolutionary uh kind of you know limits evolutionary uh kind of you know limits evolutionary uh kind of you know limits imposed by humans. I don't know. He's imposed by humans. I don't know. He's imposed by humans. I don't know. He's feeling the limits of captioning which feeling the limits of captioning which feeling the limits of captioning which is the the thing I was is the the thing I was is the the thing I was >> Yeah, exactly. [laughter] There there's >> Yeah, exactly. [laughter] There there's >> Yeah, exactly. [laughter] There there's a lot of information in the world and it a lot of information in the world and it a lot of information in the world and it connects to basically why we do work connects to basically why we do work connects to basically why we do work modeling you mentioned. You just need modeling you mentioned. You just need modeling you mentioned. You just need srefs sref476 srefs sref476 srefs sref476 and then that's your that's what your and then that's your that's what your and then that's your that's what your journey does, right? I guess maybe I journey does, right? I guess maybe I journey does, right? I guess maybe I can't describe this vibe but can't describe this vibe but can't describe this vibe but >> well well I think that that's kind of >> well well I think that that's kind of >> well well I think that that's kind of the point of providing some of these the point of providing some of these the point of providing some of these references, right? Because because like references, right? Because because like references, right? Because because like even just describing how someone talks even just describing how someone talks even just describing how someone talks and like their tone and and like procity and like their tone and and like procity and like their tone and and like procity and all of these things like I think I and all of these things like I think I and all of these things like I think I think some of these terms even like I think some of these terms even like I think some of these terms even like I didn't used to know what they mean, didn't used to know what they mean, didn't used to know what they mean, right? Well, now right? Well, now right? Well, now >> yes. Dispuencuencies ex like like there >> yes. Dispuencuencies ex like like there >> yes. Dispuencuencies ex like like there there's kind of an entire vocabulary there's kind of an entire vocabulary there's kind of an entire vocabulary that even if you're not kind of steeped that even if you're not kind of steeped that even if you're not kind of steeped in a domain, which is true for actually in a domain, which is true for actually in a domain, which is true for actually like most human domains that like you like most human domains that like you like most human domains that like you don't even know what it means. Um and don't even know what it means. Um and don't even know what it means. Um and sometimes it's also a question of like

sometimes it's also a question of like sometimes it's also a question of like if we haven't focused on those things, if we haven't focused on those things, if we haven't focused on those things, you know, with the large language models you know, with the large language models you know, with the large language models that they may also have gaps in those that they may also have gaps in those that they may also have gaps in those areas, right? And then we feel them on areas, right? And then we feel them on areas, right? And then we feel them on the other side with generation because the other side with generation because the other side with generation because we're like fundamentally relying on on we're like fundamentally relying on on we're like fundamentally relying on on the language models understanding of the the language models understanding of the the language models understanding of the world to then be able to like represent world to then be able to like represent world to then be able to like represent it. Um, so I yeah, it all kind of goes it. Um, so I yeah, it all kind of goes it. Um, so I yeah, it all kind of goes back to your question about like the the back to your question about like the the back to your question about like the the language as an intermediary. Um, but language as an intermediary. Um, but language as an intermediary. Um, but yeah, I think to De's point like some of yeah, I think to De's point like some of yeah, I think to De's point like some of these might just be like focus areas and these might just be like focus areas and these might just be like focus areas and things that we haven't necessarily things that we haven't necessarily things that we haven't necessarily pushed on as much as we can and like as pushed on as much as we can and like as pushed on as much as we can and like as we will we will discover what the actual we will we will discover what the actual we will we will discover what the actual ceiling is. ceiling is. ceiling is. >> Yeah, as a podcaster I think a lot about >> Yeah, as a podcaster I think a lot about >> Yeah, as a podcaster I think a lot about sound. sound. sound. >> Um, and and I I'll just offer a couple >> Um, and and I I'll just offer a couple >> Um, and and I I'll just offer a couple things for discussion in case in case it things for discussion in case in case it things for discussion in case in case it triggers anything with you guys. Um I triggers anything with you guys. Um I triggers anything with you guys. Um I have three domains of rough audio which have three domains of rough audio which have three domains of rough audio which is like a music voice SFX you know is is like a music voice SFX you know is is like a music voice SFX you know is that rough okay covers everything and that rough okay covers everything and that rough okay covers everything and then also even within voice let's just then also even within voice let's just then also even within voice let's just let's just focus on voice forget the let's just focus on voice forget the let's just focus on voice forget the other two um room sound like the the other two um room sound like the the other two um room sound like the the echoiness of like big room small room in echoiness of like big room small room in echoiness of like big room small room in person in a car over a phone all these person in a car over a phone all these person in a car over a phone all these like are labelable but we experience like are labelable but we experience like are labelable but we experience them very differently and I I often them very differently and I I often them very differently and I I often think like one of the tells of a AI think like one of the tells of a AI think like one of the tells of a AI video is that it is studio quality video is that it is studio quality video is that it is studio quality because it was recorded in a studio because it was recorded in a studio because it was recorded in a studio video because that's your training data video because that's your training data video because that's your training data and like and and to me that's one thing and like and and to me that's one thing and like and and to me that's one thing actually like the most interesting thing actually like the most interesting thing actually like the most interesting thing is just uh when I tell this is how I is just uh when I tell this is how I is just uh when I tell this is how I convince people who are kind of convince people who are kind of convince people who are kind of skeptical about the need for world skeptical about the need for world skeptical about the need for world models because you need it even for models because you need it even for models because you need it even for audio about well I'm further away from audio about well I'm further away from audio about well I'm further away from you so I should sound a little bit you so I should sound a little bit you so I should sound a little bit softer or more diffused and like the the

softer or more diffused and like the the softer or more diffused and like the the video models need to pick that up video models need to pick that up video models need to pick that up because if they're going to do immersive because if they're going to do immersive because if they're going to do immersive video and audio you need that video and audio you need that video and audio you need that >> I I I love that example of basically >> I I I love that example of basically >> I I I love that example of basically like studio quality or not in a way like like studio quality or not in a way like like studio quality or not in a way like we don't have enough language to really we don't have enough language to really we don't have enough language to really describe like like this kind of echoing describe like like this kind of echoing describe like like this kind of echoing or like some kind of noise kind of or like some kind of noise kind of or like some kind of noise kind of happening we just like don't have happening we just like don't have happening we just like don't have precise enough and uh if you um you know precise enough and uh if you um you know precise enough and uh if you um you know basically the reason that I think it's basically the reason that I think it's basically the reason that I think it's quite important to have like relatively quite important to have like relatively quite important to have like relatively information rich like kind of captioning information rich like kind of captioning information rich like kind of captioning is that we kind of rely on the natural is that we kind of rely on the natural is that we kind of rely on the natural language as a representation but if you language as a representation but if you language as a representation but if you basically don't have enough uh basically don't have enough uh basically don't have enough uh representation that basically means the representation that basically means the representation that basically means the condition on the language the generation condition on the language the generation condition on the language the generation is very multimodal and if you anything is very multimodal and if you anything is very multimodal and if you anything can learn from the BAE kind of like you can learn from the BAE kind of like you can learn from the BAE kind of like you know very old you know GMBA kind of know very old you know GMBA kind of know very old you know GMBA kind of research the idea is we really want to research the idea is we really want to research the idea is we really want to capture most of the stoasticity in the capture most of the stoasticity in the capture most of the stoasticity in the later representation and then the the X later representation and then the the X later representation and then the the X given the Z should be kind of like given the Z should be kind of like given the Z should be kind of like deterministic so yeah deterministic so yeah deterministic so yeah >> yeah yeah um well I hope I hope there's >> yeah yeah um well I hope I hope there's >> yeah yeah um well I hope I hope there's more uh progress there and I'm sure you more uh progress there and I'm sure you more uh progress there and I'm sure you guys are doing guys are doing guys are doing >> I even actually like facial expressions >> I even actually like facial expressions >> I even actually like facial expressions right and maybe this gets to your point right and maybe this gets to your point right and maybe this gets to your point about like things that we're very about like things that we're very about like things that we're very sensitive to right I think you can tell sensitive to right I think you can tell sensitive to right I think you can tell a lot of AI content also just by from a lot of AI content also just by from a lot of AI content also just by from like people's facial expressions like people's facial expressions like people's facial expressions stressful.

stressful. stressful. [laughter] [laughter] [laughter] >> Yes. >> Yes. >> Yes. >> Yes. And we try not to contribute to it, >> Yes. And we try not to contribute to it, >> Yes. And we try not to contribute to it, but you know, um and or or like skin but you know, um and or or like skin but you know, um and or or like skin textures, right? Like like the things textures, right? Like like the things textures, right? Like like the things that kind of make things look real in that kind of make things look real in that kind of make things look real in real life. Like I you know, I can tell real life. Like I you know, I can tell real life. Like I you know, I can tell from the way you're nodding or from the from the way you're nodding or from the from the way you're nodding or from the way like your micro expressions are kind way like your micro expressions are kind way like your micro expressions are kind of changing of like how you're reacting of changing of like how you're reacting of changing of like how you're reacting to what I'm saying. Like we haven't to what I'm saying. Like we haven't to what I'm saying. Like we haven't quite crossed that chasm. I think like quite crossed that chasm. I think like quite crossed that chasm. I think like we're we're so much better than we were we're we're so much better than we were we're we're so much better than we were a year ago. a year ago. a year ago. >> Yeah. Um, but there's so much more >> Yeah. Um, but there's so much more >> Yeah. Um, but there's so much more headroom kind of in a lot of those headroom kind of in a lot of those headroom kind of in a lot of those things that like we as humans are super things that like we as humans are super things that like we as humans are super sensitive to. And like I think image sensitive to. And like I think image sensitive to. And like I think image arguably probably is there because arguably probably is there because arguably probably is there because there's there's a lot of kind of images there's there's a lot of kind of images there's there's a lot of kind of images that I will see that like really do look that I will see that like really do look that I will see that like really do look indistinguishable from reality and I indistinguishable from reality and I indistinguishable from reality and I can't tell if they're generated or not. can't tell if they're generated or not. can't tell if they're generated or not. >> Better than reality >> Better than reality >> Better than reality >> um or well that's a different >> um or well that's a different >> um or well that's a different >> No, I I think that one of the parad >> No, I I think that one of the parad >> No, I I think that one of the parad [laughter] better than what I would take [laughter] better than what I would take [laughter] better than what I would take on my vacation as a photo. Yes. One of on my vacation as a photo. Yes. One of on my vacation as a photo. Yes. One of the one of the fun experiments that we the one of the fun experiments that we the one of the fun experiments that we did a while ago in the team is is like did a while ago in the team is is like did a while ago in the team is is like can we generate videos that are better can we generate videos that are better can we generate videos that are better than than real videos, right? So you than than real videos, right? So you than than real videos, right? So you just take the same caption from like oh just take the same caption from like oh just take the same caption from like oh yeah some video and then yeah some video and then yeah some video and then >> recycle it. Yeah. Just just try to like >> recycle it. Yeah. Just just try to like >> recycle it. Yeah. Just just try to like describe a real video and then generate describe a real video and then generate describe a real video and then generate the equivalent version with omni and the equivalent version with omni and the equivalent version with omni and then do a human eval. How does how does then do a human eval. How does how does then do a human eval. How does how does it do? And then humans largely prefer AI it do? And then humans largely prefer AI it do? And then humans largely prefer AI generated generated generated >> margin.

>> margin. >> margin. >> But because it's because it's the RL >> But because it's because it's the RL >> But because it's because it's the RL process, process, process, >> that's the RL process working. >> that's the RL process working. >> that's the RL process working. >> It's however you want to rationalize it. >> It's however you want to rationalize it. >> It's however you want to rationalize it. It's not necessarily the old process. It's not necessarily the old process. It's not necessarily the old process. It's just like I think it's just I'm not It's just like I think it's just I'm not It's just like I think it's just I'm not saying this is a good result. I'm just saying this is a good result. I'm just saying this is a good result. I'm just saying is we have optimized in a way saying is we have optimized in a way saying is we have optimized in a way that like kind of potentially sort of, that like kind of potentially sort of, that like kind of potentially sort of, you know, triggers something in the you know, triggers something in the you know, triggers something in the human brain that like, oh, it looks it human brain that like, oh, it looks it human brain that like, oh, it looks it looks all a lot of the videos just look looks all a lot of the videos just look looks all a lot of the videos just look better. Like I'm not Yeah. Yeah. Yeah. better. Like I'm not Yeah. Yeah. Yeah. better. Like I'm not Yeah. Yeah. Yeah. on on inspection on on deeper inspection on on inspection on on deeper inspection on on inspection on on deeper inspection they they would not actually be more they they would not actually be more they they would not actually be more useful or whatever but like if you just useful or whatever but like if you just useful or whatever but like if you just say side by side random YouTube video say side by side random YouTube video say side by side random YouTube video versus versus versus >> generated version of it will you will >> generated version of it will you will >> generated version of it will you will just have a it will just look better just have a it will just look better just have a it will just look better because it's more it's a sharper more because it's more it's a sharper more because it's more it's a sharper more HDR uh you know the skin tone is is is HDR uh you know the skin tone is is is HDR uh you know the skin tone is is is better it's not again it's not more better it's not again it's not more better it's not again it's not more realistic realistic realistic >> uh it doesn't solve your problem >> uh it doesn't solve your problem >> uh it doesn't solve your problem necessarily but it it looks better necessarily but it it looks better necessarily but it it looks better >> I I since also depend on the sensitivity >> I I since also depend on the sensitivity >> I I since also depend on the sensitivity of the people. Uh I was born raised in of the people. Uh I was born raised in of the people. Uh I was born raised in Japan and I think one thing I kind of Japan and I think one thing I kind of Japan and I think one thing I kind of know is like they're extremely extremely know is like they're extremely extremely know is like they're extremely extremely like sensitive about like you know like sensitive about like you know like sensitive about like you know that's why you know like architecture that's why you know like architecture that's why you know like architecture like food and stuff like they have.

like food and stuff like they have. like food and stuff like they have. >> Um so I talked to like a mangar like >> Um so I talked to like a mangar like >> Um so I talked to like a mangar like like artist there and he's like he's like artist there and he's like he's like artist there and he's like he's kind of disgusted by like the generation kind of disgusted by like the generation kind of disgusted by like the generation AI and one kind of thing he mentioned is AI and one kind of thing he mentioned is AI and one kind of thing he mentioned is like the eye gaze like the eye gaze like the eye gaze >> eye gaze that slight difference >> eye gaze that slight difference >> eye gaze that slight difference >> makes me makes him kind of feel creepy >> makes me makes him kind of feel creepy >> makes me makes him kind of feel creepy about like unnatural about like unnatural about like unnatural >> like if you're looking a little bit off. >> like if you're looking a little bit off. >> like if you're looking a little bit off. >> Yeah. It's just uh Yeah. just like uh it >> Yeah. It's just uh Yeah. just like uh it >> Yeah. It's just uh Yeah. just like uh it looks too fake. Yeah. To the point. So looks too fake. Yeah. To the point. So looks too fake. Yeah. To the point. So So I think it does depend on the So I think it does depend on the So I think it does depend on the sensitivity and sensitivity and sensitivity and >> Yeah. Yeah. All I'm saying is like you >> Yeah. Yeah. All I'm saying is like you >> Yeah. Yeah. All I'm saying is like you know human preferences are like a not know human preferences are like a not know human preferences are like a not particularly like uh reliable barometer particularly like uh reliable barometer particularly like uh reliable barometer of like what you should be optimizing of like what you should be optimizing of like what you should be optimizing for like if you just ask people do you for like if you just ask people do you for like if you just ask people do you like this or not you not necessarily get like this or not you not necessarily get like this or not you not necessarily get what you wanted. what you wanted. what you wanted. >> Yeah. Let let me just kind of add one >> Yeah. Let let me just kind of add one >> Yeah. Let let me just kind of add one thing but like four years ago there was thing but like four years ago there was thing but like four years ago there was a like debate that if the prompt a like debate that if the prompt a like debate that if the prompt engineering is going to disappear and uh engineering is going to disappear and uh engineering is going to disappear and uh my my like you know some very powerful my my like you know some very powerful my my like you know some very powerful people say you know it's going to people say you know it's going to people say you know it's going to disappear but I basically said like it disappear but I basically said like it disappear but I basically said like it shouldn't because the prompt engineering shouldn't because the prompt engineering shouldn't because the prompt engineering like sort of you know specifying that is like sort of you know specifying that is like sort of you know specifying that is like the the only way you can sort of like the the only way you can sort of like the the only way you can sort of control the output sort of you know when control the output sort of you know when control the output sort of you know when you have like sort of control by the AI you have like sort of control by the AI you have like sort of control by the AI and what allows you to prompt engineer and what allows you to prompt engineer and what allows you to prompt engineer is really that sensitivity. So sure is really that sensitivity. So sure is really that sensitivity. So sure maybe like right now the AI can do a lot maybe like right now the AI can do a lot maybe like right now the AI can do a lot of autoprompting and that and it can of autoprompting and that and it can of autoprompting and that and it can generate something that's sufficient but generate something that's sufficient but generate something that's sufficient but uh if it's like that never be satisfied uh if it's like that never be satisfied uh if it's like that never be satisfied like never be satisfied with the AI's like never be satisfied with the AI's like never be satisfied with the AI's generated content always fine-tune your generated content always fine-tune your generated content always fine-tune your sensitivity and always kind of keep sensitivity and always kind of keep sensitivity and always kind of keep prompting the differences. I I think to prompting the differences. I I think to prompting the differences. I I think to the there's also a big difference the there's also a big difference the there's also a big difference between like the average human untrained

between like the average human untrained between like the average human untrained eye which I I would put myself in that eye which I I would put myself in that eye which I I would put myself in that bucket you know like I have I have some bucket you know like I have I have some bucket you know like I have I have some aesthetic sensibilities and I've done aesthetic sensibilities and I've done aesthetic sensibilities and I've done this long enough that you know like I this long enough that you know like I this long enough that you know like I have I have a preference um but you know have I have a preference um but you know have I have a preference um but you know like your example of a manga artist like like your example of a manga artist like like your example of a manga artist like that's somebody who has honed a craft that's somebody who has honed a craft that's somebody who has honed a craft like over possibly many decades. Um, and like over possibly many decades. Um, and like over possibly many decades. Um, and anybody who does that, whether it's like anybody who does that, whether it's like anybody who does that, whether it's like design, architecture, right? Like you design, architecture, right? Like you design, architecture, right? Like you you you just have a very different level you you just have a very different level you you just have a very different level of like expertise and you see things of like expertise and you see things of like expertise and you see things that like the average human will not that like the average human will not that like the average human will not see. But Doom is right. Like when we see. But Doom is right. Like when we see. But Doom is right. Like when we look at if you were to just, you know, look at if you were to just, you know, look at if you were to just, you know, um, poll 10 people on the street, they um, poll 10 people on the street, they um, poll 10 people on the street, they would probably prefer the like overly would probably prefer the like overly would probably prefer the like overly smooth like very saturated kind of smooth like very saturated kind of smooth like very saturated kind of >> It's called the Instagram filter. >> It's called the Instagram filter. >> It's called the Instagram filter. >> It is. It is the Yeah. [laughter] Um, >> It is. It is the Yeah. [laughter] Um, >> It is. It is the Yeah. [laughter] Um, and you know, and and so there's also a and you know, and and so there's also a and you know, and and so there's also a little bit of a question of like what little bit of a question of like what little bit of a question of like what does your default aesthetic look like if does your default aesthetic look like if does your default aesthetic look like if you don't specify? But then to Shane's you don't specify? But then to Shane's you don't specify? But then to Shane's point, one of the things we always try point, one of the things we always try point, one of the things we always try to get these models better at is to get these models better at is to get these models better at is instruction follow so that like when you instruction follow so that like when you instruction follow so that like when you want to get them to a different outcome want to get them to a different outcome want to get them to a different outcome like you should be able to whether like you should be able to whether like you should be able to whether that's through language or whether that's through language or whether that's through language or whether that's through references because that's through references because that's through references because language is sometimes too limiting. Um, language is sometimes too limiting. Um, language is sometimes too limiting. Um, and so like these models continue to get and so like these models continue to get and so like these models continue to get better at it but they so much at work.

better at it but they so much at work. better at it but they so much at work. Do do you feel pressure as a as a Do do you feel pressure as a as a Do do you feel pressure as a as a product director to set the default for product director to set the default for product director to set the default for the world like I mean [laughter] the world like I mean [laughter] the world like I mean [laughter] >> kind of >> kind of >> kind of >> maybe I should I don't know I haven't >> maybe I should I don't know I haven't >> maybe I should I don't know I haven't thought about this thought about this thought about this >> you know you know it's like someone has >> you know you know it's like someone has >> you know you know it's like someone has to have a default the default has to to have a default the default has to to have a default the default has to exist exist exist >> actually I will say like we have thought >> actually I will say like we have thought >> actually I will say like we have thought about this um and I I think one of the about this um and I I think one of the about this um and I I think one of the so for example actually like if you look so for example actually like if you look so for example actually like if you look at nanobanana generations we had like an at nanobanana generations we had like an at nanobanana generations we had like an explosion of nanobanana infographics explosion of nanobanana infographics explosion of nanobanana infographics when nanobanana pro came out when nanobanana pro came out when nanobanana pro came out >> I tried it yeah >> I tried it yeah >> I tried it yeah >> um yeah >> um yeah >> um yeah I think Nurb's papers were like all you I think Nurb's papers were like all you I think Nurb's papers were like all you know so so many had like infographics know so so many had like infographics know so so many had like infographics generated. Can you run your uh generated. Can you run your uh generated. Can you run your uh watermarking on it and see how many watermarking on it and see how many watermarking on it and see how many >> uh we pro we probably could we have we >> uh we pro we probably could we have we >> uh we pro we probably could we have we haven't done that but I saw so like my haven't done that but I saw so like my haven't done that but I saw so like my Twitter was maybe this is just also like Twitter was maybe this is just also like Twitter was maybe this is just also like the bias of my algorithm but they were the bias of my algorithm but they were the bias of my algorithm but they were everywhere um and it was actually very everywhere um and it was actually very everywhere um and it was actually very painful because um I think our default painful because um I think our default painful because um I think our default aesthetic was a little bit too it was aesthetic was a little bit too it was aesthetic was a little bit too it was too cluttered like I think that the the too cluttered like I think that the the too cluttered like I think that the the model is like a bit of an overeager model is like a bit of an overeager model is like a bit of an overeager student that just like learned you know student that just like learned you know student that just like learned you know it was like oh I know all these like I it was like oh I know all these like I it was like oh I know all these like I know all this information about this know all this information about this know all this information about this concept let me like shove into the same concept let me like shove into the same concept let me like shove into the same image. Japanese infographics 5x that image. Japanese infographics 5x that image. Japanese infographics 5x that [laughter] [laughter] [laughter] >> or maybe it was you know um but it just >> or maybe it was you know um but it just >> or maybe it was you know um but it just and and and and and and >> wait so same prompt same content if it's >> wait so same prompt same content if it's >> wait so same prompt same content if it's in Japanese it's in Japanese it's in Japanese it's >> density density >> density density >> density density >> oh wow >> oh wow >> oh wow >> because that's the style in Japan >> because that's the style in Japan >> because that's the style in Japan >> yeah some like very you know bureaucrat >> yeah some like very you know bureaucrat >> yeah some like very you know bureaucrat and [laughter] there's a famous word for and [laughter] there's a famous word for and [laughter] there's a famous word for it yeah it yeah it yeah >> no but we do do go through this process >> no but we do do go through this process >> no but we do do go through this process with Omni we did it together right like with Omni we did it together right like with Omni we did it together right like where like we had like a bunch of like

where like we had like a bunch of like where like we had like a bunch of like we like at the very end okay like this we like at the very end okay like this we like at the very end okay like this is we did some tuning and like okay what is we did some tuning and like okay what is we did some tuning and like okay what kind of style do we prefer right like kind of style do we prefer right like kind of style do we prefer right like you know you know you know >> is it more muted more saturated >> is it more muted more saturated >> is it more muted more saturated >> we had a lot of saturation >> we had a lot of saturation >> we had a lot of saturation >> yeah there was there were I think Nicole >> yeah there was there were I think Nicole >> yeah there was there were I think Nicole just has PTSD so has forgotten about it just has PTSD so has forgotten about it just has PTSD so has forgotten about it but she was very much involved in this but she was very much involved in this but she was very much involved in this of like okay which which kind of color of like okay which which kind of color of like okay which which kind of color palette do we basically prefer right and palette do we basically prefer right and palette do we basically prefer right and it's you know it's it's it's not it's you know it's it's it's not it's you know it's it's it's not something that like you have to make a a something that like you have to make a a something that like you have to make a a trade-off there like uh trade-off there like uh trade-off there like uh >> and and and it's because it ends up >> and and and it's because it ends up >> and and and it's because it ends up being us right like actually it is true being us right like actually it is true being us right like actually it is true like it it ends up being the modeling like it it ends up being the modeling like it it ends up being the modeling teams and you could ask the question teams and you could ask the question teams and you could ask the question legitimately of like are we the best legitimately of like are we the best legitimately of like are we the best people to do that or should we actually people to do that or should we actually people to do that or should we actually work with someone who like has a really work with someone who like has a really work with someone who like has a really creative point of view and is more of creative point of view and is more of creative point of view and is more of like you know an art director and like like you know an art director and like like you know an art director and like has like and we kind of go back and has like and we kind of go back and has like and we kind of go back and forth on this um forth on this um forth on this um >> we have the trusted testers I'm on >> we have the trusted testers I'm on >> we have the trusted testers I'm on >> we do we have trusted testers who give >> we do we have trusted testers who give >> we do we have trusted testers who give us a lot of feedback and we take that us a lot of feedback and we take that us a lot of feedback and we take that serious serious serious >> very well organized by the way to have >> very well organized by the way to have >> very well organized by the way to have these like weekly calls and stuff like these like weekly calls and stuff like these like weekly calls and stuff like it's it's amazing it's it's amazing it's it's amazing >> um Logan's team does a lot of that so >> um Logan's team does a lot of that so >> um Logan's team does a lot of that so kudo kuda kudos kudos to Logan um who kudo kuda kudos kudos to Logan um who kudo kuda kudos kudos to Logan um who couldn't be here today um and we have a couldn't be here today um and we have a couldn't be here today um and we have a lot of people actually internally at lot of people actually internally at lot of people actually internally at Google like Fulfur who give us like a Google like Fulfur who give us like a Google like Fulfur who give us like a ton of No, no, no. Truly like who give ton of No, no, no. Truly like who give ton of No, no, no. Truly like who give us a ton of feedback on like when we us a ton of feedback on like when we us a ton of feedback on like when we when we release new checkpoints and like when we release new checkpoints and like when we release new checkpoints and like sometimes it will be stuff that we like sometimes it will be stuff that we like sometimes it will be stuff that we like don't see right like we would be like oh don't see right like we would be like oh don't see right like we would be like oh yeah this optimization seems okay and yeah this optimization seems okay and yeah this optimization seems okay and then they would come back what have you then they would come back what have you then they would come back what have you done like you completely ruined my grass done like you completely ruined my grass done like you completely ruined my grass you know because now the detail is all you know because now the detail is all you know because now the detail is all blurry.

blurry. blurry. >> I think he just noticed not not a super >> I think he just noticed not not a super >> I think he just noticed not not a super secret at this point but like that our secret at this point but like that our secret at this point but like that our model tends to put rings wedding rings model tends to put rings wedding rings model tends to put rings wedding rings on on on hand. That's yeah on on on hand. That's yeah on on on hand. That's yeah >> very strange. I had never noticed that >> very strange. I had never noticed that >> very strange. I had never noticed that but he's like he I just saw it and but he's like he I just saw it and but he's like he I just saw it and there's a faux fur channel basically. there's a faux fur channel basically. there's a faux fur channel basically. >> Uh he posted I was like why is there >> Uh he posted I was like why is there >> Uh he posted I was like why is there wedding ring in every hand? I'm like wedding ring in every hand? I'm like wedding ring in every hand? I'm like that's strange. that's strange. that's strange. >> That sounds very common reward hacking. >> That sounds very common reward hacking. >> That sounds very common reward hacking. >> Yeah. Yeah. Yeah. So but you know >> Yeah. Yeah. Yeah. So but you know >> Yeah. Yeah. Yeah. So but you know something that we would not have we something that we would not have we something that we would not have we would not have noticed necessarily while would not have noticed necessarily while would not have noticed necessarily while while developing this right is an oral while developing this right is an oral while developing this right is an oral artifact or artifact or artifact or >> I I don't know you do have like a lot of >> I I don't know you do have like a lot of >> I I don't know you do have like a lot of preference based and then you know you preference based and then you know you preference based and then you know you may can prefer that sperious correlation may can prefer that sperious correlation may can prefer that sperious correlation reward hacking it can happen like in reward hacking it can happen like in reward hacking it can happen like in many weird ways. Yeah many weird ways. Yeah many weird ways. Yeah >> it does. It is >> it does. It is >> it does. It is >> uh this was related to another topic >> uh this was related to another topic >> uh this was related to another topic that again I I try to use these that again I I try to use these that again I I try to use these mainstage things as introductions or mainstage things as introductions or mainstage things as introductions or ties in. Uh we have the eval track we ties in. Uh we have the eval track we ties in. Uh we have the eval track we have character AI and YouTube talking have character AI and YouTube talking have character AI and YouTube talking about how they evaluate videos. Um how about how they evaluate videos. Um how about how they evaluate videos. Um how do you evaluate videos do you evaluate videos do you evaluate videos >> apart from furer [laughter] >> apart from furer [laughter] >> apart from furer [laughter] >> not everyone has a fauxur but also you >> not everyone has a fauxur but also you >> not everyone has a fauxur but also you know I think there needs to be something know I think there needs to be something know I think there needs to be something more quantitative more quantitative more quantitative >> well I mean it's you improve Gemini >> well I mean it's you improve Gemini >> well I mean it's you improve Gemini to improve the evaluation for video. Um to improve the evaluation for video. Um to improve the evaluation for video. Um that's that's no no that's that's that's that's no no that's that's that's that's no no that's that's definitely one way uh it's actually very definitely one way uh it's actually very definitely one way uh it's actually very hard.

hard. hard. >> It's very hard. It's very hard um to get >> It's very hard. It's very hard um to get >> It's very hard. It's very hard um to get like you know audators to evaluate like you know audators to evaluate like you know audators to evaluate things in a video like including things in a video like including things in a video like including especially things like aesthetics right especially things like aesthetics right especially things like aesthetics right like that it's like there are some like that it's like there are some like that it's like there are some things that are a little bit more things that are a little bit more things that are a little bit more objective like especially when we talk objective like especially when we talk objective like especially when we talk like let's say we talk about images and like let's say we talk about images and like let's say we talk about images and we look at like infographics text we look at like infographics text we look at like infographics text rendering that's actually fine right rendering that's actually fine right rendering that's actually fine right because like you can kind of OCR things because like you can kind of OCR things because like you can kind of OCR things out and then you can look at like okay out and then you can look at like okay out and then you can look at like okay this letter is like messed up and then this letter is like messed up and then this letter is like messed up and then the whole thing is actually useless the whole thing is actually useless the whole thing is actually useless because if like literally if a letter is because if like literally if a letter is because if like literally if a letter is off in render text you just can't use off in render text you just can't use off in render text you just can't use that asset. Right. So th those things that asset. Right. So th those things that asset. Right. So th those things are like a little bit more auto ratable. are like a little bit more auto ratable. are like a little bit more auto ratable. Um from what we found we do rely a lot Um from what we found we do rely a lot Um from what we found we do rely a lot on humans looking at things and so we do on humans looking at things and so we do on humans looking at things and so we do do a lot of human evals. We do a lot of do a lot of human evals. We do a lot of do a lot of human evals. We do a lot of human evals. human evals. human evals. >> Do a lot of human ev and every time Jane >> Do a lot of human ev and every time Jane >> Do a lot of human ev and every time Jane is like um and every time we have a new is like um and every time we have a new is like um and every time we have a new model we like want to do more things and model we like want to do more things and model we like want to do more things and we want to like gem in more capabilities we want to like gem in more capabilities we want to like gem in more capabilities and then we have like more emails that and then we have like more emails that and then we have like more emails that we have to run. Um, and then at some we have to run. Um, and then at some we have to run. Um, and then at some point you do get two models that are point you do get two models that are point you do get two models that are like kind of close to each other and like kind of close to each other and like kind of close to each other and then like we literally make decisions then like we literally make decisions then like we literally make decisions based on like looking at outputs side by based on like looking at outputs side by based on like looking at outputs side by side. Sometimes like in a room like I've side. Sometimes like in a room like I've side. Sometimes like in a room like I've been in rooms where there's like 10 of been in rooms where there's like 10 of been in rooms where there's like 10 of us and we're just like looking at video us and we're just like looking at video us and we're just like looking at video side by side and we're like do you side by side and we're like do you side by side and we're like do you prefer this or do you prefer that? like prefer this or do you prefer that? like prefer this or do you prefer that? like oh wow it's oh wow it's oh wow it's >> I mean but it is it is genuinely very >> I mean but it is it is genuinely very >> I mean but it is it is genuinely very complicated the more capabilities you complicated the more capabilities you complicated the more capabilities you add like you know even just the one add like you know even just the one add like you know even just the one capability but it's like almost AGI capability but it's like almost AGI capability but it's like almost AGI complete capabilities like video editing complete capabilities like video editing complete capabilities like video editing right like think about video editing as right like think about video editing as right like think about video editing as a and like editing with audio and

a and like editing with audio and a and like editing with audio and >> my editor will be very happy to hear >> my editor will be very happy to hear >> my editor will be very happy to hear this this this >> edit the hardest problem in g media >> edit the hardest problem in g media >> edit the hardest problem in g media >> I mean I don't know if it's the hardest >> I mean I don't know if it's the hardest >> I mean I don't know if it's the hardest but it's definitely there right like uh but it's definitely there right like uh but it's definitely there right like uh in terms of like complexity of of in terms of like complexity of of in terms of like complexity of of evaluation like free form video editing evaluation like free form video editing evaluation like free form video editing is you can do anything like is you can do anything like is you can do anything like >> yes uh and like I I spent a lot of money >> yes uh and like I I spent a lot of money >> yes uh and like I I spent a lot of money on that and it's very hard to tell me on that and it's very hard to tell me on that and it's very hard to tell me >> like adding those we don't have like add >> like adding those we don't have like add >> like adding those we don't have like add a sloth eval right like uh that we a sloth eval right like uh that we a sloth eval right like uh that we >> well now we should >> well now we should >> well now we should >> now we should yeah yeah yeah but like >> now we should yeah yeah yeah but like >> now we should yeah yeah yeah but like things like that like it's it's it's not things like that like it's it's it's not things like that like it's it's it's not that easy to track that easy to track that easy to track >> I think I'm just surprised at the sample >> I think I'm just surprised at the sample >> I think I'm just surprised at the sample size that you have right like to to test size that you have right like to to test size that you have right like to to test the entire surface of your models you the entire surface of your models you the entire surface of your models you still rely on a magnitude of hundreds still rely on a magnitude of hundreds still rely on a magnitude of hundreds >> no no no so we do like yeah well we do >> no no no so we do like yeah well we do >> no no no so we do like yeah well we do we do a ton of human evals on like on we do a ton of human evals on like on we do a ton of human evals on like on like you know thousands of things. Um I like you know thousands of things. Um I like you know thousands of things. Um I I think there's also like an element of I think there's also like an element of I think there's also like an element of you know we can talk about things like you know we can talk about things like you know we can talk about things like live experiments right like which which live experiments right like which which live experiments right like which which is also where you get signal on like is also where you get signal on like is also where you get signal on like like some of these more minute like some of these more minute like some of these more minute differences at like much larger scale differences at like much larger scale differences at like much larger scale then there's auto raers which is then there's auto raers which is then there's auto raers which is definitely kind of a more it's a very definitely kind of a more it's a very definitely kind of a more it's a very well defined space I think for LLMs much well defined space I think for LLMs much well defined space I think for LLMs much more nent for media models and then like more nent for media models and then like more nent for media models and then like sometimes you still do rely on human sometimes you still do rely on human sometimes you still do rely on human judgment and we do rely on things like judgment and we do rely on things like judgment and we do rely on things like feedback from people who just like have feedback from people who just like have feedback from people who just like have a very owned like aesthetic and and a very owned like aesthetic and and a very owned like aesthetic and and people who just like use these models in people who just like use these models in people who just like use these models in their workflows dayto-day, right?

their workflows dayto-day, right? their workflows dayto-day, right? Because we could also like you could Because we could also like you could Because we could also like you could have a model that does really well on have a model that does really well on have a model that does really well on some slice of human evals, but then it some slice of human evals, but then it some slice of human evals, but then it like really breaks a workflow for like really breaks a workflow for like really breaks a workflow for somebody. And so this is why we do like somebody. And so this is why we do like somebody. And so this is why we do like early access programs and we try to get early access programs and we try to get early access programs and we try to get feedback and then we like try to feedback and then we like try to feedback and then we like try to incorporate it before we release incorporate it before we release incorporate it before we release something more broadly. I feel like something more broadly. I feel like something more broadly. I feel like Shane had a hot take based on his Shane had a hot take based on his Shane had a hot take based on his >> expression always when we were talking >> expression always when we were talking >> expression always when we were talking about this about this about this >> every kind of human sort of you know >> every kind of human sort of you know >> every kind of human sort of you know work should be gradually kind of work should be gradually kind of work should be gradually kind of amortized and then the interesting thing amortized and then the interesting thing amortized and then the interesting thing is the video understanding especially is the video understanding especially is the video understanding especially like against like AI gener like like against like AI gener like like against like AI gener like detecting air stuff is extremely detecting air stuff is extremely detecting air stuff is extremely interesting uh visual task interesting uh visual task interesting uh visual task >> and then like some of it kind of >> and then like some of it kind of >> and then like some of it kind of aesthetics or this kind of visual aesthetics or this kind of visual aesthetics or this kind of visual quality but for some of the kind of quality but for some of the kind of quality but for some of the kind of cases like semantically doesn't make cases like semantically doesn't make cases like semantically doesn't make sense for example you're taking like sense for example you're taking like sense for example you're taking like some like a famous scene from a movie some like a famous scene from a movie some like a famous scene from a movie and try to sort of um construct that and and try to sort of um construct that and and try to sort of um construct that and then if you kind of generate it uh it then if you kind of generate it uh it then if you kind of generate it uh it can generate something there but at some can generate something there but at some can generate something there but at some point some of the semantic information point some of the semantic information point some of the semantic information doesn't make sense like it's actually doesn't make sense like it's actually doesn't make sense like it's actually inconsistent. So can the AI actually inconsistent. So can the AI actually inconsistent. So can the AI actually detect that? So when I evaluate the AI detect that? So when I evaluate the AI detect that? So when I evaluate the AI videos like oh I feel I'm so smart you videos like oh I feel I'm so smart you videos like oh I feel I'm so smart you know like like AI is still kind of know like like AI is still kind of know like like AI is still kind of behind but we should make like a lot of behind but we should make like a lot of behind but we should make like a lot of effort. I think the video understanding effort. I think the video understanding effort. I think the video understanding is extremely uh important intelligence is extremely uh important intelligence is extremely uh important intelligence task uh beyond just the pure aesthetics task uh beyond just the pure aesthetics task uh beyond just the pure aesthetics or the preference. Um and yeah we we or the preference. Um and yeah we we or the preference. Um and yeah we we should always try to amatize the human should always try to amatize the human should always try to amatize the human >> human label. Yeah.

>> human label. Yeah. >> human label. Yeah. >> Yeah. Um, what data do you need? A lot >> Yeah. Um, what data do you need? A lot >> Yeah. Um, what data do you need? A lot of people I talked to wanted to get in of people I talked to wanted to get in of people I talked to wanted to get in front of you actually. Uh, they I mean front of you actually. Uh, they I mean front of you actually. Uh, they I mean they want to be nice about it. They have they want to be nice about it. They have they want to be nice about it. They have a lot of video data. They have gaming a lot of video data. They have gaming a lot of video data. They have gaming data. They have real world video data. data. They have real world video data. data. They have real world video data. They have images. They have labelers. They have images. They have labelers. They have images. They have labelers. What do you want? What do you want? What do you want? >> Are you like offering? >> Are you like offering? >> Are you like offering? >> I'm just like this is your request for >> I'm just like this is your request for >> I'm just like this is your request for like Okay. Okay. We get I'm sure you get like Okay. Okay. We get I'm sure you get like Okay. Okay. We get I'm sure you get a lot of pitches, right? You get a lot a lot of pitches, right? You get a lot a lot of pitches, right? You get a lot of people want to talk to you. what's of people want to talk to you. what's of people want to talk to you. what's like I think actually it's the signal is like I think actually it's the signal is like I think actually it's the signal is this problem this sorting out signal this problem this sorting out signal this problem this sorting out signal from noise is the main problem so from noise is the main problem so from noise is the main problem so creating a nice API of like okay if you creating a nice API of like okay if you creating a nice API of like okay if you actually do a b and c we are interested actually do a b and c we are interested actually do a b and c we are interested in that in that in that >> um >> um >> um loaded question there so uh I don't know loaded question there so uh I don't know loaded question there so uh I don't know that there's like an easy like you know that there's like an easy like you know that there's like an easy like you know did you do I think we we do already have did you do I think we we do already have did you do I think we we do already have a lot of data I think it's it's a lot of data I think it's it's a lot of data I think it's it's >> hard to talk about this >> hard to talk about this >> hard to talk about this >> you know you want to talk about the >> you know you want to talk about the >> you know you want to talk about the public I don't want to get you in public I don't want to get you in public I don't want to get you in trouble Yeah, trouble Yeah, trouble Yeah, >> but like I think >> but like I think >> but like I think >> No, no. What I just want to say is like >> No, no. What I just want to say is like >> No, no. What I just want to say is like hard to talk about this in a sort of you hard to talk about this in a sort of you hard to talk about this in a sort of you know without trying to without I have to know without trying to without I have to know without trying to without I have to think about the what I am revealing think about the what I am revealing think about the what I am revealing about our project and what where we're about our project and what where we're about our project and what where we're going. Um generally high quality data I going. Um generally high quality data I going. Um generally high quality data I think maybe maybe let's just put it this think maybe maybe let's just put it this think maybe maybe let's just put it this way right it's not not the secret way right it's not not the secret way right it's not not the secret >> embodied I'm sorry >> embodied I'm sorry >> embodied I'm sorry >> embodied data >> embodied data >> embodied data >> I mean >> I mean >> I mean >> yeah sure I mean we have sort of >> yeah sure I mean we have sort of >> yeah sure I mean we have sort of announced I think publicly right that we announced I think publicly right that we announced I think publicly right that we we have some sort of robotics we have some sort of robotics we have some sort of robotics collaboration right like so I think it's collaboration right like so I think it's collaboration right like so I think it's like a like or or but you because we

like a like or or but you because we like a like or or but you because we have a robotics team at GDM so you know have a robotics team at GDM so you know have a robotics team at GDM so you know they're always interested in things like they're always interested in things like they're always interested in things like that um I mean for Omni specifically I that um I mean for Omni specifically I that um I mean for Omni specifically I think we're just quite interested just think we're just quite interested just think we're just quite interested just high quality data right like you know it high quality data right like you know it high quality data right like you know it it's not some sort of not necessarily it's not some sort of not necessarily it's not some sort of not necessarily like oh random YouTube video but like like oh random YouTube video but like like oh random YouTube video but like you know some a some more professional you know some a some more professional you know some a some more professional shop things like that right the things shop things like that right the things shop things like that right the things that those are those are things that that those are those are things that that those are those are things that we're always on the lookout for like uh we're always on the lookout for like uh we're always on the lookout for like uh and yeah and yeah and yeah >> and I think for you know maybe this is >> and I think for you know maybe this is >> and I think for you know maybe this is easier to some extent to answer for like easier to some extent to answer for like easier to some extent to answer for like some of the agentic work as well like some of the agentic work as well like some of the agentic work as well like like like actual kind of like what are like like actual kind of like what are like like actual kind of like what are the tests that people are trying to do the tests that people are trying to do the tests that people are trying to do right these things are actually kind of right these things are actually kind of right these things are actually kind of difficult to manufacture if you're doing difficult to manufacture if you're doing difficult to manufacture if you're doing it yourself or if you're like doing it it yourself or if you're like doing it it yourself or if you're like doing it with a vendor, like what is the actual with a vendor, like what is the actual with a vendor, like what is the actual like if you're creating a marketing like if you're creating a marketing like if you're creating a marketing campaign, like what does that look like, campaign, like what does that look like, campaign, like what does that look like, right? Like do do you start from here's right? Like do do you start from here's right? Like do do you start from here's like a picture of my new product and like a picture of my new product and like a picture of my new product and then I want to turn that into a video ad then I want to turn that into a video ad then I want to turn that into a video ad and I want to turn that into a bunch of and I want to turn that into a bunch of and I want to turn that into a bunch of assets that like fit fit all these assets that like fit fit all these assets that like fit fit all these different ad formats that I need to push different ad formats that I need to push different ad formats that I need to push onto various platforms to promote and onto various platforms to promote and onto various platforms to promote and then like so you kind of go from this to then like so you kind of go from this to then like so you kind of go from this to that and like what is that kind of that and like what is that kind of that and like what is that kind of trajectory of tasks that you're that trajectory of tasks that you're that trajectory of tasks that you're that you're like you know experiencing along you're like you know experiencing along you're like you know experiencing along the way like that is really useful and the way like that is really useful and the way like that is really useful and that is actually kind of difficult to that is actually kind of difficult to that is actually kind of difficult to get right u because like we don't always get right u because like we don't always get right u because like we don't always have the right firstparty surface where have the right firstparty surface where have the right firstparty surface where people are actually doing some of these people are actually doing some of these people are actually doing some of these things or like you might work with things or like you might work with things or like you might work with someone who's a vendor but they don't someone who's a vendor but they don't someone who's a vendor but they don't also don't have that product surface also don't have that product surface also don't have that product surface right like like a lot of this kind of right like like a lot of this kind of right like like a lot of this kind of information lives in the places where information lives in the places where information lives in the places where people are doing these tasks and so

people are doing these tasks and so people are doing these tasks and so that's kind of difficult to get like if that's kind of difficult to get like if that's kind of difficult to get like if anyone's figured that out you should anyone's figured that out you should anyone's figured that out you should reach out to us reach out to us reach out to us >> every channel of thought yeah every >> every channel of thought yeah every >> every channel of thought yeah every [laughter] thought [laughter] thought [laughter] thought >> every thought yeah and maybe the data >> every thought yeah and maybe the data >> every thought yeah and maybe the data the Chinese lab is using the Chinese lab is using the Chinese lab is using >> yes yeah uh you know >> yes yeah uh you know >> yes yeah uh you know as a media person myself, right? Like as a media person myself, right? Like as a media person myself, right? Like there's so many podcasters and people in there's so many podcasters and people in there's so many podcasters and people in in marketing departments and all these in marketing departments and all these in marketing departments and all these like they would happy to be your data like they would happy to be your data like they would happy to be your data like you know just like put a BCI on my like you know just like put a BCI on my like you know just like put a BCI on my head head head >> and podcast [laughter] watch my things >> and podcast [laughter] watch my things >> and podcast [laughter] watch my things uh because like you know there's just uh because like you know there's just uh because like you know there's just endless amount of work to do like endless amount of work to do like endless amount of work to do like there's so much work and this is all there's so much work and this is all there's so much work and this is all like this needs to somewhat be commodity like this needs to somewhat be commodity like this needs to somewhat be commodity like obviously you can be an art like an like obviously you can be an art like an like obviously you can be an art like an artisan like you can be Hollywood for artisan like you can be Hollywood for artisan like you can be Hollywood for like the really high quality stuff but like the really high quality stuff but like the really high quality stuff but actually a lot of work is commodity and actually a lot of work is commodity and actually a lot of work is commodity and like should be modelable and we want you like should be modelable and we want you like should be modelable and we want you to do to do to do >> [laughter] >> [laughter] >> [laughter] >> And but we we want the high quality to >> And but we we want the high quality to >> And but we we want the high quality to Demi's point right like we do want we Demi's point right like we do want we Demi's point right like we do want we want the high quality. want the high quality. want the high quality. >> We want commodity. Yeah. Yes. Yes. You >> We want commodity. Yeah. Yes. Yes. You >> We want commodity. Yeah. Yes. Yes. You want on both sides. want on both sides. want on both sides. >> Um I I just >> Um I I just >> Um I I just >> Thank you for the solicitation.

>> Thank you for the solicitation. >> Thank you for the solicitation. [laughter] [laughter] [laughter] >> Uh I you know we we we also I also added >> Uh I you know we we we also I also added >> Uh I you know we we we also I also added a data quality track. I I think that uh a data quality track. I I think that uh a data quality track. I I think that uh people want to understand like what uh people want to understand like what uh people want to understand like what uh at AI like how to raise the bar, right? at AI like how to raise the bar, right? at AI like how to raise the bar, right? like like the and a lot of it is just like like the and a lot of it is just like like the and a lot of it is just educating the market and educating educating the market and educating educating the market and educating researchers and engineers and founders researchers and engineers and founders researchers and engineers and founders on like this is where we're going a lot on like this is where we're going a lot on like this is where we're going a lot of this is stop doing that do this do of this is stop doing that do this do of this is stop doing that do this do this instead and I'm like people will this instead and I'm like people will this instead and I'm like people will listen listen listen yeah I don't know uh to that extent you yeah I don't know uh to that extent you yeah I don't know uh to that extent you know know know >> but I think to that to that point like >> but I think to that to that point like >> but I think to that to that point like there's a lot of again just like craft there's a lot of again just like craft there's a lot of again just like craft that goes into this right and there's a that goes into this right and there's a that goes into this right and there's a lot of process like you even to the lot of process like you even to the lot of process like you even to the marketing campaign example you don't marketing campaign example you don't marketing campaign example you don't create that in like five minutes right create that in like five minutes right create that in like five minutes right you like go you go through a process and you like go you go through a process and you like go you go through a process and you iterate and you like pick something you iterate and you like pick something you iterate and you like pick something over something else because you liked it over something else because you liked it over something else because you liked it for whatever reason like maybe the eye for whatever reason like maybe the eye for whatever reason like maybe the eye gaze was correct right like we just we gaze was correct right like we just we gaze was correct right like we just we don't know these things right because don't know these things right because don't know these things right because none of us are marketing directors and none of us are marketing directors and none of us are marketing directors and like the models don't know these things like the models don't know these things like the models don't know these things >> I even kind of say this for the natural >> I even kind of say this for the natural >> I even kind of say this for the natural like a language as well like I I always like a language as well like I I always like a language as well like I I always kind of say 99% of information is inside kind of say 99% of information is inside kind of say 99% of information is inside people you can only extract it through people you can only extract it through people you can only extract it through active dialogue and befriending them so active dialogue and befriending them so active dialogue and befriending them so most of the stuff on the internet is most of the stuff on the internet is most of the stuff on the internet is like sort of the outcome the output of like sort of the outcome the output of like sort of the outcome the output of that yes but you know what are what are that yes but you know what are what are that yes but you know what are what are all the trajectories you know how did all the trajectories you know how did all the trajectories you know how did this person have this inspiration to this person have this inspiration to this person have this inspiration to write this paper write this paper write this paper >> what is the starting point what is the >> what is the starting point what is the >> what is the starting point what is the inspiration what are the dialogue that inspiration what are the dialogue that inspiration what are the dialogue that sparked it those kind of stuff is kind sparked it those kind of stuff is kind sparked it those kind of stuff is kind of inside people so even you know those of inside people so even you know those of inside people so even you know those kind of like even the language space is kind of like even the language space is kind of like even the language space is kind of that I think the creative is kind of that I think the creative is kind of that I think the creative is kind of similar as well there's a lot of kind of similar as well there's a lot of kind of similar as well there's a lot of dark knowledge dark knowledge dark knowledge >> yeah it's like when you write a novel >> yeah it's like when you write a novel >> yeah it's like when you write a novel right like a novel speaks to you because

right like a novel speaks to you because right like a novel speaks to you because like usually there's some sort of like a like usually there's some sort of like a like usually there's some sort of like a personal connection that you feel to personal connection that you feel to personal connection that you feel to like the story or the trajectory or the like the story or the trajectory or the like the story or the trajectory or the characters right like if you read most characters right like if you read most characters right like if you read most of the stuff that's written by LLM's of the stuff that's written by LLM's of the stuff that's written by LLM's today like it's, you know, it's it's it today like it's, you know, it's it's it today like it's, you know, it's it's it starts it falls into these like default starts it falls into these like default starts it falls into these like default par patterns and like the language par patterns and like the language par patterns and like the language starts to feel really similar and all starts to feel really similar and all starts to feel really similar and all the descriptions sound really similar. the descriptions sound really similar. the descriptions sound really similar. You can kind of like quickly read it as You can kind of like quickly read it as You can kind of like quickly read it as like, oh, this is not that interesting like, oh, this is not that interesting like, oh, this is not that interesting because like I can't connect to it, because like I can't connect to it, because like I can't connect to it, right? Um, and again, that's that's kind right? Um, and again, that's that's kind right? Um, and again, that's that's kind of like a human expertise. of like a human expertise. of like a human expertise. >> One nice thing recently is the Google >> One nice thing recently is the Google >> One nice thing recently is the Google Cloud and the Google Deep Mind are kind Cloud and the Google Deep Mind are kind Cloud and the Google Deep Mind are kind of starting to invest a lot more in the of starting to invest a lot more in the of starting to invest a lot more in the FTEEs for the product engineers. And I FTEEs for the product engineers. And I FTEEs for the product engineers. And I also kind of saw some uh recruiting for also kind of saw some uh recruiting for also kind of saw some uh recruiting for the creative you know gem media kind of the creative you know gem media kind of the creative you know gem media kind of space as well. So I think those are kind space as well. So I think those are kind space as well. So I think those are kind of really the effort because we we kind of really the effort because we we kind of really the effort because we we kind of feel that you know what we can kind of feel that you know what we can kind of feel that you know what we can kind of do with a lot of public data there's of do with a lot of public data there's of do with a lot of public data there's limits but really you know partnering limits but really you know partnering limits but really you know partnering with that we can provide kind of better with that we can provide kind of better with that we can provide kind of better models and products and yeah we kind of models and products and yeah we kind of models and products and yeah we kind of feedback feedback feedback >> uh we have an FD track here for the >> uh we have an FD track here for the >> uh we have an FD track here for the first time every lab is announcing it. first time every lab is announcing it. first time every lab is announcing it. It's it's crazy. Um, one thing I'm It's it's crazy. Um, one thing I'm It's it's crazy. Um, one thing I'm actually very keen on doing and I push I actually very keen on doing and I push I actually very keen on doing and I push I push for this at Cognition as well is to push for this at Cognition as well is to push for this at Cognition as well is to turn the FDES not just into sales and turn the FDES not just into sales and turn the FDES not just into sales and solutions but also to EVAL's uh eval solutions but also to EVAL's uh eval solutions but also to EVAL's uh eval workers.

workers. workers. >> FD is not the sales FD is way way bigger >> FD is not the sales FD is way way bigger >> FD is not the sales FD is way way bigger than that. How do you frame FDs then? than that. How do you frame FDs then? than that. How do you frame FDs then? Because [laughter] I do think about it Because [laughter] I do think about it Because [laughter] I do think about it as sales like you're you know the more as sales like you're you know the more as sales like you're you know the more the more you customize the solution for the more you customize the solution for the more you customize the solution for >> so I define post training as anything >> so I define post training as anything >> so I define post training as anything between the pre-training and the final between the pre-training and the final between the pre-training and the final user experience anything anything is a user experience anything anything is a user experience anything anything is a post training post training post training >> and to me when I first sort of you know >> and to me when I first sort of you know >> and to me when I first sort of you know learned a lot about I mean FD kind of I learned a lot about I mean FD kind of I learned a lot about I mean FD kind of I guess originally you know came from like guess originally you know came from like guess originally you know came from like path here and then that so I guess the path here and then that so I guess the path here and then that so I guess the kind of history is different but yeah I kind of history is different but yeah I kind of history is different but yeah I think the key is really that um you know think the key is really that um you know think the key is really that um you know the key is like not only to kind of work the key is like not only to kind of work the key is like not only to kind of work uh with them and ensure that they kind uh with them and ensure that they kind uh with them and ensure that they kind of know how to of know how to of know how to but also to sort of code like derive but also to sort of code like derive but also to sort of code like derive kind of insights that can basically kind kind of insights that can basically kind kind of insights that can basically kind of help both parties. They can put the of help both parties. They can put the of help both parties. They can put the like a lot of harness how they use the like a lot of harness how they use the like a lot of harness how they use the model. We can improve like very model. We can improve like very model. We can improve like very upstream. So how to get the customer upstream. So how to get the customer upstream. So how to get the customer feedback to the modeling I feel is the feedback to the modeling I feel is the feedback to the modeling I feel is the kind of more the the role I I kind of kind of more the the role I I kind of kind of more the the role I I kind of want for the fds. Yeah. want for the fds. Yeah. want for the fds. Yeah. >> Yeah. Yeah. and and even for sorry just >> Yeah. Yeah. and and even for sorry just >> Yeah. Yeah. and and even for sorry just on that like if you want to talk to us on that like if you want to talk to us on that like if you want to talk to us or at least me um I I'm not going to or at least me um I I'm not going to or at least me um I I'm not going to offer up your time um but I it's really offer up your time um but I it's really offer up your time um but I it's really helpful for us to actually talk to helpful for us to actually talk to helpful for us to actually talk to people who are using our models and like people who are using our models and like people who are using our models and like understand where they're struggling uh understand where they're struggling uh understand where they're struggling uh because again that just like it's it's because again that just like it's it's because again that just like it's it's the real world task that you're actually the real world task that you're actually the real world task that you're actually trying to use them for right like I will trying to use them for right like I will trying to use them for right like I will talk to people who do kind of interior talk to people who do kind of interior talk to people who do kind of interior inter interior design with some of our inter interior design with some of our inter interior design with some of our image models um you know and they will image models um you know and they will image models um you know and they will say hey like I really want to take this say hey like I really want to take this say hey like I really want to take this pattern pattern, but then I want to pattern pattern, but then I want to pattern pattern, but then I want to scale it across like 10 different ruck scale it across like 10 different ruck scale it across like 10 different ruck sizes and sometimes I have like a very

sizes and sometimes I have like a very sizes and sometimes I have like a very custom ruck size and then the model custom ruck size and then the model custom ruck size and then the model fails at like replicating the pattern fails at like replicating the pattern fails at like replicating the pattern the same way or you know I want to do a the same way or you know I want to do a the same way or you know I want to do a try on for these earrings and then the try on for these earrings and then the try on for these earrings and then the earrings have a certain size and then earrings have a certain size and then earrings have a certain size and then like my head has a certain size like it like my head has a certain size like it like my head has a certain size like it has to make sense if you're actually has to make sense if you're actually has to make sense if you're actually trying to try things on and like the trying to try things on and like the trying to try things on and like the models kind of fail at a bunch of these models kind of fail at a bunch of these models kind of fail at a bunch of these things that like actually happen in the things that like actually happen in the things that like actually happen in the real world, right? Um and so that that's real world, right? Um and so that that's real world, right? Um and so that that's like useful for us because for some of like useful for us because for some of like useful for us because for some of these things like we don't think about these things like we don't think about these things like we don't think about because we don't you know we don't use because we don't you know we don't use because we don't you know we don't use the models for those tasks the models for those tasks the models for those tasks >> or like um you know I think to your >> or like um you know I think to your >> or like um you know I think to your point about ad campaigns or whatever point about ad campaigns or whatever point about ad campaigns or whatever like people have like notions of brand like people have like notions of brand like people have like notions of brand languages or whatever like which is languages or whatever like which is languages or whatever like which is >> yes >> yes >> yes >> like a a bunch of images or PDFs saying >> like a a bunch of images or PDFs saying >> like a a bunch of images or PDFs saying things you know it's a pretty kind of things you know it's a pretty kind of things you know it's a pretty kind of you know ambiguous question as well what you know ambiguous question as well what you know ambiguous question as well what is the IKEA brand language you know is is the IKEA brand language you know is is the IKEA brand language you know is it is it blue and yellow I mean that's it is it blue and yellow I mean that's it is it blue and yellow I mean that's that's not a very like that's not a very like that's not a very like >> but like what shade of blue you know. >> but like what shade of blue you know. >> but like what shade of blue you know. >> Yeah. Yeah. Yeah. So there there's like, >> Yeah. Yeah. Yeah. So there there's like, >> Yeah. Yeah. Yeah. So there there's like, you know, and the brands are pretty you know, and the brands are pretty you know, and the brands are pretty spec, you know, pretty, you know, like spec, you know, pretty, you know, like spec, you know, pretty, you know, like they they do care about the shade of they they do care about the shade of they they do care about the shade of blue. It's not shouldn't just be a blue. It's not shouldn't just be a blue. It's not shouldn't just be a random blue and a random yellow. That's random blue and a random yellow. That's random blue and a random yellow. That's not going to be IKEA, right? I'm just not going to be IKEA, right? I'm just not going to be IKEA, right? I'm just thinking about an example. But like this thinking about an example. But like this thinking about an example. But like this is the kind of stuff that, you know, is the kind of stuff that, you know, is the kind of stuff that, you know, it's not necessarily part of our like, it's not necessarily part of our like, it's not necessarily part of our like, you know, developing frontier models you know, developing frontier models you know, developing frontier models kind of, you know, necessarily mandate, kind of, you know, necessarily mandate, kind of, you know, necessarily mandate, but it's something that we do want to we but it's something that we do want to we but it's something that we do want to we do want to fundamentally like build do want to fundamentally like build do want to fundamentally like build products that people will use to solve products that people will use to solve products that people will use to solve concrete tasks, not just not just concrete tasks, not just not just concrete tasks, not just not just research artifacts, right? So I think research artifacts, right? So I think research artifacts, right? So I think it's useful to understand what people do it's useful to understand what people do it's useful to understand what people do care about. Uh well, I'm sure a lot of care about. Uh well, I'm sure a lot of care about. Uh well, I'm sure a lot of people are very grateful for your work people are very grateful for your work people are very grateful for your work and there's a lot more to do that you've and there's a lot more to do that you've and there's a lot more to do that you've made so much progress over the last like made so much progress over the last like made so much progress over the last like even just couple years of like Nano even just couple years of like Nano even just couple years of like Nano Banana and Theo and Omni and uh I don't

Banana and Theo and Omni and uh I don't Banana and Theo and Omni and uh I don't know what else you got cooking but we're know what else you got cooking but we're know what else you got cooking but we're very excited like you this is one of very excited like you this is one of very excited like you this is one of those things where like I was very those things where like I was very those things where like I was very disappointed you know when Sora shut disappointed you know when Sora shut disappointed you know when Sora shut down and and I think like there needs to down and and I think like there needs to down and and I think like there needs to be more general exploration of uh you be more general exploration of uh you be more general exploration of uh you know generative models and not just you know generative models and not just you know generative models and not just you know coding. [laughter] I think I think know coding. [laughter] I think I think know coding. [laughter] I think I think that is that is that is >> we obviously like this. >> we obviously like this. >> we obviously like this. >> We love coding. Love coding and and uh >> We love coding. Love coding and and uh >> We love coding. Love coding and and uh yes uh but thank you so much for your yes uh but thank you so much for your yes uh but thank you so much for your time. Uh it's been a real pleasure and I time. Uh it's been a real pleasure and I time. Uh it's been a real pleasure and I can't wait to see what this looks like can't wait to see what this looks like can't wait to see what this looks like next. next. next. >> Thank you for having us. Great question. >> Thank you for having us. Great question. >> Thank you for having us. Great question. >> Thank you everyone. [applause]