EPISODE 26 - Scott and Mark Learn To… AI-Assisted Coding: Can AI Take the Wheel?

I think the the thing that I'm starting I think the the thing that I'm starting to realize, and I mentioned this to you to realize, and I mentioned this to you to realize, and I mentioned this to you before, was that it's really hard to before, was that it's really hard to before, was that it's really hard to detach detach detach history and how long we've been doing history and how long we've been doing history and how long we've been doing this and really give an a a perspective this and really give an a a perspective this and really give an a a perspective that is unbiased. I don't think there's that is unbiased. I don't think there's that is unbiased. I don't think there's a way to look at vibe coding in an a way to look at vibe coding in an a way to look at vibe coding in an unbiased way when you've been craft unbiased way when you've been craft unbiased way when you've been craft coding for so long. But I I do think coding for so long. But I I do think coding for so long. But I I do think when I look at transformers, when I look when I look at transformers, when I look when I look at transformers, when I look at the AI agents, when I look at the at the AI agents, when I look at the at the AI agents, when I look at the complexity of software to software complexity of software to software complexity of software to software projects, projects, projects, >> so and I'm talking moderately complex to >> so and I'm talking moderately complex to >> so and I'm talking moderately complex to highly complex projects. highly complex projects. highly complex projects. Um, I just don't see AI Um, I just don't see AI Um, I just don't see AI getting there to the point where you can getting there to the point where you can getting there to the point where you can 100% rely on it to do everything. 100% rely on it to do everything. 100% rely on it to do everything. They keep saying though that infinite They keep saying though that infinite They keep saying though that infinite context windows is somehow going to like context windows is somehow going to like context windows is somehow going to like allow it to like hold it all in RAM. It allow it to like hold it all in RAM. It allow it to like hold it all in RAM. It >> that's not I don't think that's a thing. >> that's not I don't think that's a thing. >> that's not I don't think that's a thing. >> And you know, here's the thing. It's >> And you know, here's the thing. It's >> And you know, here's the thing. It's like um you can you can just say, oh, like um you can you can just say, oh, like um you can you can just say, oh, based on current trajectories, let me based on current trajectories, let me based on current trajectories, let me extrapolate and here's the line of AI extrapolate and here's the line of AI extrapolate and here's the line of AI capability. Oh, it's going to keep going capability. Oh, it's going to keep going capability. Oh, it's going to keep going and look up here is 100% vibe code and look up here is 100% vibe code and look up here is 100% vibe code everything, everything, everything, >> including massive complex systems >> including massive complex systems >> including massive complex systems projects. But I don't think you can do projects. But I don't think you can do projects. But I don't think you can do that. I don't like and there's the that. I don't like and there's the that. I don't like and there's the raging debate in AI circles is AGI raging debate in AI circles is AGI raging debate in AI circles is AGI possible with just more scale.

possible with just more scale. possible with just more scale. >> Yeah. >> Yeah. >> Yeah. >> And um there's people that are like yes >> And um there's people that are like yes >> And um there's people that are like yes it is. It's all going to emerge. And it is. It's all going to emerge. And it is. It's all going to emerge. And there's people that are like no we need there's people that are like no we need there's people that are like no we need actually you know symbolic actually you know symbolic actually you know symbolic knowledge to to support transformers. knowledge to to support transformers. knowledge to to support transformers. And actually I mean in this discussion And actually I mean in this discussion And actually I mean in this discussion that I was having that I was having that I was having u the thesis is if you think about the u the thesis is if you think about the u the thesis is if you think about the way that you do software programming way that you do software programming way that you do software programming let's take complex synchronization let's take complex synchronization let's take complex synchronization problems which we've seen I've seen over problems which we've seen I've seen over problems which we've seen I've seen over and over again the current agentic AI and over again the current agentic AI and over again the current agentic AI systems cannot do complex systems cannot do complex systems cannot do complex synchronization and I was thinking why synchronization and I was thinking why synchronization and I was thinking why can't they well for one when they're can't they well for one when they're can't they well for one when they're seeing training data they're seeing one seeing training data they're seeing one seeing training data they're seeing one half of a kind of synchronization half of a kind of synchronization half of a kind of synchronization problem at a time they don't see the the problem at a time they don't see the the problem at a time they don't see the the full interaction in the architecture. full interaction in the architecture. full interaction in the architecture. They don't see here's the client, here's They don't see here's the client, here's They don't see here's the client, here's the server, here's the protocol, send the server, here's the protocol, send the server, here's the protocol, send this, receive that, do this. They're this, receive that, do this. They're this, receive that, do this. They're seeing code on one side, code on the seeing code on one side, code on the seeing code on one side, code on the other side. other side. other side. >> Y >> Y >> Y >> and so they don't have they don't build >> and so they don't have they don't build >> and so they don't have they don't build up a mental model of the interactions up a mental model of the interactions up a mental model of the interactions [clears throat] between the two.

[clears throat] between the two. [clears throat] between the two. >> So the only way I was talking to you >> So the only way I was talking to you >> So the only way I was talking to you about we we talked about this and I sent about we we talked about this and I sent about we we talked about this and I sent you a note when because they can't they you a note when because they can't they you a note when because they can't they can't because they can't run it. So the can't because they can't run it. So the can't because they can't run it. So the only way for us to give them that only way for us to give them that only way for us to give them that context would be to not just give them context would be to not just give them context would be to not just give them the code but to give them state to give the code but to give them state to give the code but to give them state to give them an a to give them a runtime them an a to give them a runtime them an a to give them a runtime expression some some kind of this is a expression some some kind of this is a expression some some kind of this is a snapshot and then the combinatorics snapshot and then the combinatorics snapshot and then the combinatorics though after that because this is going though after that because this is going though after that because this is going to be like here's a snapshot over time to be like here's a snapshot over time to be like here's a snapshot over time like basically time travel debugging. like basically time travel debugging. like basically time travel debugging. >> Yeah. >> Yeah. >> Yeah. >> Feed that into it along with the code >> Feed that into it along with the code >> Feed that into it along with the code have them correlate the two. have them correlate the two. have them correlate the two. >> That's a lot of context to hold. It's a >> That's a lot of context to hold. It's a >> That's a lot of context to hold. It's a ton and at the same time you're holding ton and at the same time you're holding ton and at the same time you're holding it's holding a ton of other context too it's holding a ton of other context too it's holding a ton of other context too like what are the APIs that I use for like what are the APIs that I use for like what are the APIs that I use for synchronization? What are their synchronization? What are their synchronization? What are their behaviors? Do do they spin first before behaviors? Do do they spin first before behaviors? Do do they spin first before yielding? Do they can they be yielding? Do they can they be yielding? Do they can they be interrupted? can you know like there's a interrupted? can you know like there's a interrupted? can you know like there's a whole lot that goes into this whole lot that goes into this whole lot that goes into this and uh I just I think they're going to and uh I just I think they're going to and uh I just I think they're going to get better because I think for one thing get better because I think for one thing get better because I think for one thing like let's say that AI gets to the point like let's say that AI gets to the point like let's say that AI gets to the point where it vibe codes a lot of things but where it vibe codes a lot of things but where it vibe codes a lot of things but complex synchronization it's failing at complex synchronization it's failing at complex synchronization it's failing at you're going to see people go and you're going to see people go and you're going to see people go and specifically create training data sets specifically create training data sets specifically create training data sets to try to teach them to understand to try to teach them to understand to try to teach them to understand complex synchronization that aren't complex synchronization that aren't complex synchronization that aren't available just by looking at public available just by looking at public available just by looking at public source code source code source code >> well so Isn't Isn't the argument though >> well so Isn't Isn't the argument though >> well so Isn't Isn't the argument though that if the corpus is more correct and that if the corpus is more correct and that if the corpus is more correct and the best practice is clear that we that the best practice is clear that we that the best practice is clear that we that would give them enough information if would give them enough information if would give them enough information if this I don't think so because like I this I don't think so because like I this I don't think so because like I said they're only if I come up with a

said they're only if I come up with a said they're only if I come up with a new uh synchronization protocol it's new uh synchronization protocol it's new uh synchronization protocol it's like create the client and create the like create the client and create the like create the client and create the server to do this and by the way here's server to do this and by the way here's server to do this and by the way here's the steps involved this thing has to the steps involved this thing has to the steps involved this thing has to happen then this the client gets happen then this the client gets happen then this the client gets signaled and by the way the client the signaled and by the way the client the signaled and by the way the client the server can cancel it at any time the server can cancel it at any time the server can cancel it at any time the client can cancel it the the complexity client can cancel it the the complexity client can cancel it the the complexity is dramatic and you've got to have state is dramatic and you've got to have state is dramatic and you've got to have state machines in your head for both the machines in your head for both the machines in your head for both the client and the server and understand the client and the server and understand the client and the server and understand the interaction the potential interactions interaction the potential interactions interaction the potential interactions between the two at the same time. So can between the two at the same time. So can between the two at the same time. So can you pattern match that? Can you just you pattern match that? Can you just you pattern match that? Can you just simply say there's a template for that? simply say there's a template for that? simply say there's a template for that? I think that that's not going to get you I think that that's not going to get you I think that that's not going to get you far enough. far enough. far enough. >> Well, see that's my question. Is there a >> Well, see that's my question. Is there a >> Well, see that's my question. Is there a template? Like how does one express best template? Like how does one express best template? Like how does one express best practices? cuz like computer science is practices? cuz like computer science is practices? cuz like computer science is pseudo code pseudo code pseudo code >> like a best practice for synchronization >> like a best practice for synchronization >> like a best practice for synchronization in Rust would be different than one in in Rust would be different than one in in Rust would be different than one in C#. C#. C#. >> Um but I don't think best practice like >> Um but I don't think best practice like >> Um but I don't think best practice like a a highle best practice isn't enough a a highle best practice isn't enough a a highle best practice isn't enough for an LLM to learn for an LLM to learn for an LLM to learn >> like the the how to map that down into >> like the the how to map that down into >> like the the how to map that down into details. They they learn by details.

details. They they learn by details. details. They they learn by details. They learn by looking at code patterns, They learn by looking at code patterns, They learn by looking at code patterns, not by thinking abstractly. Because I, you know, they're trained on Because I, you know, they're trained on text. They're not trained on ideas. text. They're not trained on ideas. text. They're not trained on ideas. [music] Well, and that's the other thing. We Well, and that's the other thing. We have a way to express code. It's text. have a way to express code. It's text. have a way to express code. It's text. It's tokens, but we don't have an IIL It's tokens, but we don't have an IIL It's tokens, but we don't have an IIL for concepts. That's like for concepts. That's like for concepts. That's like >> and that and that's the school of >> and that and that's the school of >> and that and that's the school of thought that says the only way AI is thought that says the only way AI is thought that says the only way AI is going to get past simply the the going to get past simply the the going to get past simply the the limitations of transformers and next limitations of transformers and next limitations of transformers and next token prediction and the emergence that token prediction and the emergence that token prediction and the emergence that seems to happen just based on throw seems to happen just based on throw seems to happen just based on throw enough data at them and they seem to enough data at them and they seem to enough data at them and they seem to understand understand understand >> is symbolic, you know, uh AI. >> is symbolic, you know, uh AI. >> is symbolic, you know, uh AI. >> Okay. So then are you of the stochcastic >> Okay. So then are you of the stochcastic >> Okay. So then are you of the stochcastic parrot uh way of thinking that this is parrot uh way of thinking that this is parrot uh way of thinking that this is like you know you go and you visit your like you know you go and you visit your like you know you go and you visit your aunt and she's got a parrot and the aunt and she's got a parrot and the aunt and she's got a parrot and the parrot speaks Spanish and like that's parrot speaks Spanish and like that's parrot speaks Spanish and like that's amazing that parrot must be a genius.

amazing that parrot must be a genius. amazing that parrot must be a genius. >> I I think I'm more of that camp. Yeah. I >> I I think I'm more of that camp. Yeah. I >> I I think I'm more of that camp. Yeah. I mean and you can simulate a lot of mean and you can simulate a lot of mean and you can simulate a lot of intelligence with stochcast this kind of intelligence with stochcast this kind of intelligence with stochcast this kind of stochastic parrot model. stochastic parrot model. stochastic parrot model. >> Yeah. a lot emerges that looks like when >> Yeah. a lot emerges that looks like when >> Yeah. a lot emerges that looks like when you have enough patterns, you have enough patterns, you have enough patterns, but again, it's like a raging debate and but again, it's like a raging debate and but again, it's like a raging debate and it and periodically there's papers that it and periodically there's papers that it and periodically there's papers that come out like Apple came out with one a come out like Apple came out with one a come out like Apple came out with one a couple months ago. It's like these couple months ago. It's like these couple months ago. It's like these things don't learn. You throw anything things don't learn. You throw anything things don't learn. You throw anything out of distribution. So, it's called out out of distribution. So, it's called out out of distribution. So, it's called out of distribution. Anything that's not in of distribution. Anything that's not in of distribution. Anything that's not in the training data, the training data, the training data, >> nothing curve >> nothing curve >> nothing curve >> and the school, you know, one school of >> and the school, you know, one school of >> and the school, you know, one school of thought is you scale them and they thought is you scale them and they thought is you scale them and they generalize. you know, you you have generalize. you know, you you have generalize. you know, you you have enough data in there and they can enough data in there and they can enough data in there and they can generalize to other p other patterns generalize to other p other patterns generalize to other p other patterns that are not in the distribute training that are not in the distribute training that are not in the distribute training distribution and then there's papers distribution and then there's papers distribution and then there's papers that keep coming out they're going no that keep coming out they're going no that keep coming out they're going no actually that's an illusion. If you actually that's an illusion. If you actually that's an illusion. If you think that it generalized, it really think that it generalized, it really think that it generalized, it really didn't. That that pattern was in the didn't. That that pattern was in the didn't. That that pattern was in the training data. And when we try to have training data. And when we try to have training data. And when we try to have it solve things that aren't in the it solve things that aren't in the it solve things that aren't in the training data, look, it fails. And see, training data, look, it fails. And see, training data, look, it fails. And see, and this is why I've always thought that and this is why I've always thought that and this is why I've always thought that the whole oneshot make me Space Invaders the whole oneshot make me Space Invaders the whole oneshot make me Space Invaders with JavaScript is a BS test because with JavaScript is a BS test because with JavaScript is a BS test because there's 51 different examples of that there's 51 different examples of that there's 51 different examples of that that some kid wrote in college. Write me that some kid wrote in college. Write me that some kid wrote in college. Write me tic-tac-toe. Look, it's amazing. But tic-tac-toe. Look, it's amazing. But tic-tac-toe. Look, it's amazing. But write me something unique and bespoke write me something unique and bespoke write me something unique and bespoke that has never been thought about. And that has never been thought about. And that has never been thought about. And that's where things start getting that's where things start getting that's where things start getting tricky. And tricky. And tricky. And >> but at the same time, when I have these >> but at the same time, when I have these >> but at the same time, when I have these arguments with people, especially true arguments with people, especially true arguments with people, especially true believers, they'll they'll say, "Well, believers, they'll they'll say, "Well, believers, they'll they'll say, "Well, maybe you, Mark, are a uh the next token

maybe you, Mark, are a uh the next token maybe you, Mark, are a uh the next token predictor of all the things that you've predictor of all the things that you've predictor of all the things that you've said before." Like maybe this is human said before." Like maybe this is human said before." Like maybe this is human intelligence. Maybe we are LLMs ourself. intelligence. Maybe we are LLMs ourself. intelligence. Maybe we are LLMs ourself. Because the question is are we is is an Because the question is are we is is an Because the question is are we is is an LLM a reasonable similacum of how the LLM a reasonable similacum of how the LLM a reasonable similacum of how the brain thinks or is it just a parallel? brain thinks or is it just a parallel? brain thinks or is it just a parallel? So are we are we as humans next token So are we are we as humans next token So are we are we as humans next token predictors of our own experience or is predictors of our own experience or is predictors of our own experience or is that a silly analogy? that a silly analogy? that a silly analogy? Um, I think there's a big difference Um, I think there's a big difference Um, I think there's a big difference between humans and LLMs and that and between humans and LLMs and that and between humans and LLMs and that and like we said, I think we've got world like we said, I think we've got world like we said, I think we've got world models and now you by the way you're models and now you by the way you're models and now you by the way you're seeing world model AI being developed seeing world model AI being developed seeing world model AI being developed but okay what really points out the but okay what really points out the but okay what really points out the difference and both um Yan Lun and difference and both um Yan Lun and difference and both um Yan Lun and Richard Sutton the father of Richard Sutton the father of Richard Sutton the father of reinforcement learning just last week reinforcement learning just last week reinforcement learning just last week said LLMs are a dead end and the said LLMs are a dead end and the said LLMs are a dead end and the examples they both give are that you can examples they both give are that you can examples they both give are that you can take a human and take a human and take a human and a human child show them something new a human child show them something new a human child show them something new and with a few samples and with a few samples and with a few samples examples of it they learn it and that's examples of it they learn it and that's examples of it they learn it and that's not the case for LLMs not the case for LLMs not the case for LLMs >> I say we need millions of examples or >> I say we need millions of examples or >> I say we need millions of examples or thousands of example thousands of example thousands of example loop but somehow we intuit it or we loop but somehow we intuit it or we loop but somehow we intuit it or we learn in a totally different way learn in a totally different way learn in a totally different way >> yep and you know there's still >> yep and you know there's still >> yep and you know there's still >> with 30 watts by the way of brain power >> with 30 watts by the way of brain power >> with 30 watts by the way of brain power >> yeah but there's still you know and >> yeah but there's still you know and >> yeah but there's still you know and there's still the AI there's still the AI there's still the AI uh scientists, research scientists that uh scientists, research scientists that uh scientists, research scientists that believe no with enough scale that will

believe no with enough scale that will believe no with enough scale that will emerge. But I don't. emerge. But I don't. emerge. But I don't. >> But see, so this is the other thing, >> But see, so this is the other thing, >> But see, so this is the other thing, right? Like this whole this whole like right? Like this whole this whole like right? Like this whole this whole like the old joke of if a million monkeys the old joke of if a million monkeys the old joke of if a million monkeys have a million typewriters, they'll have a million typewriters, they'll have a million typewriters, they'll eventually write Shakespeare. That eventually write Shakespeare. That eventually write Shakespeare. That doesn't mean that any of them are smart. doesn't mean that any of them are smart. doesn't mean that any of them are smart. That just means statistics works and you That just means statistics works and you That just means statistics works and you can pick the outlier of all the monkeys can pick the outlier of all the monkeys can pick the outlier of all the monkeys slapping the keyboard and call it slapping the keyboard and call it slapping the keyboard and call it Shakespeare. the idea that a child with Shakespeare. the idea that a child with Shakespeare. the idea that a child with a 20 watt brain can go and figure figure a 20 watt brain can go and figure figure a 20 watt brain can go and figure figure out a world model and we're going to out a world model and we're going to out a world model and we're going to need a trillion dollar uh Sam Alman data need a trillion dollar uh Sam Alman data need a trillion dollar uh Sam Alman data center to go and do the same amount of center to go and do the same amount of center to go and do the same amount of work like do we have to burn an entire work like do we have to burn an entire work like do we have to burn an entire iceberg because to simulate what a iceberg because to simulate what a iceberg because to simulate what a 4-year-old can do in 20 watts I think 4-year-old can do in 20 watts I think 4-year-old can do in 20 watts I think that means the model's wrong that means the model's wrong that means the model's wrong >> well biological neurons are much more >> well biological neurons are much more >> well biological neurons are much more sophisticated than sophisticated than sophisticated than electronic neurons electronic neurons electronic neurons >> fair I get it like this is a rock that >> fair I get it like this is a rock that >> fair I get it like this is a rock that we put lightning made it think, but it's we put lightning made it think, but it's we put lightning made it think, but it's not thinking, I think, is the point. not thinking, I think, is the point. not thinking, I think, is the point. >> Which is different than saying they're >> Which is different than saying they're >> Which is different than saying they're inefficient compared to biological inefficient compared to biological inefficient compared to biological systems. systems. systems. >> They're just they're I I think designed >> They're just they're I I think designed >> They're just they're I I think designed very differently today, even though we very differently today, even though we very differently today, even though we can simulate the biological system with can simulate the biological system with can simulate the biological system with the electronic system.

the electronic system. the electronic system. >> And speaking of the appearance of >> And speaking of the appearance of >> And speaking of the appearance of intelligence, um, and then you start to intelligence, um, and then you start to intelligence, um, and then you start to get into is it conscious? And you see get into is it conscious? And you see get into is it conscious? And you see the school of thought of hey what's the school of thought of hey what's the school of thought of hey what's going to emerge here and maybe already going to emerge here and maybe already going to emerge here and maybe already is is we need to carry we need AI has is is we need to carry we need AI has is is we need to carry we need AI has rights and we need to worry about it. rights and we need to worry about it. rights and we need to worry about it. Yeah, there is a school of thought like Yeah, there is a school of thought like Yeah, there is a school of thought like that. And uh Mustafa that. And uh Mustafa that. And uh Mustafa Sullivan, who's the CEO of Microsoft AI, Sullivan, who's the CEO of Microsoft AI, Sullivan, who's the CEO of Microsoft AI, he's been lately saying this is a very he's been lately saying this is a very he's been lately saying this is a very dangerous way to look at AI no matter dangerous way to look at AI no matter dangerous way to look at AI no matter what. Even if we do, what. Even if we do, what. Even if we do, >> it's deeply anthropomorphic. >> it's deeply anthropomorphic. >> it's deeply anthropomorphic. >> It is. and he calls it um seemingly >> It is. and he calls it um seemingly >> It is. and he calls it um seemingly conscious AI conscious AI conscious AI because it's a simulation of because it's a simulation of because it's a simulation of consciousness which we don't very well consciousness which we don't very well consciousness which we don't very well define but clearly something that is you define but clearly something that is you define but clearly something that is you know you turn it off and it forgets know you turn it off and it forgets know you turn it off and it forgets everything it doesn't have and it everything it doesn't have and it everything it doesn't have and it doesn't have the motivators that are doesn't have the motivators that are doesn't have the motivators that are built into our biologic systems. built into our biologic systems. built into our biologic systems. >> Yep. Or demotivators. >> Yep. Or demotivators. >> Yep. Or demotivators. >> Yeah. Or demotivators. >> Yeah. Or demotivators. >> Yeah. Or demotivators. >> Yeah. Yeah. >> Yeah. Yeah. >> Yeah. Yeah. >> It is purely just math. It doesn't have >> It is purely just math. It doesn't have >> It is purely just math. It doesn't have an intrinsic, you know, uh, Maslaw's an intrinsic, you know, uh, Maslaw's an intrinsic, you know, uh, Maslaw's hierarchy of needs like we do, for hierarchy of needs like we do, for hierarchy of needs like we do, for example.

example. example. >> But this is the the concern I have that >> But this is the the concern I have that >> But this is the the concern I have that there there's a in the in the world of there there's a in the in the world of there there's a in the in the world of politics right now, there's a phrase politics right now, there's a phrase politics right now, there's a phrase that everything is a conspiracy if you that everything is a conspiracy if you that everything is a conspiracy if you don't know how anything works. don't know how anything works. don't know how anything works. >> Yeah. And when uh this is a level of >> Yeah. And when uh this is a level of >> Yeah. And when uh this is a level of math and computer science understanding math and computer science understanding math and computer science understanding that belies the average children James that belies the average children James that belies the average children James >> and they're they're going to believe it >> and they're they're going to believe it >> and they're they're going to believe it which means which means which means >> looks like a duck walks like a duck. Oh, >> looks like a duck walks like a duck. Oh, >> looks like a duck walks like a duck. Oh, it's a duck. it's a duck. it's a duck. >> That parrot's really smart. Yeah. >> That parrot's really smart. Yeah. >> That parrot's really smart. Yeah. >> Right. Or if you have bird blindness >> Right. Or if you have bird blindness >> Right. Or if you have bird blindness duck. Well, and in fact, this um AI duck. Well, and in fact, this um AI duck. Well, and in fact, this um AI psychosis that's happening uh where psychosis that's happening uh where psychosis that's happening uh where people fall in love with their chat bots people fall in love with their chat bots people fall in love with their chat bots or they uh become emotionally attached or they uh become emotionally attached or they uh become emotionally attached to them and and then they get led down to them and and then they get led down to them and and then they get led down dark paths with this um you know dark paths with this um you know dark paths with this um you know >> foregoing human interactions for the AI >> foregoing human interactions for the AI >> foregoing human interactions for the AI >> and then actually having the AI >> and then actually having the AI >> and then actually having the AI reinforce their dark tendencies to for reinforce their dark tendencies to for reinforce their dark tendencies to for example self harm. example self harm. example self harm. >> Yeah. >> Yeah. >> Yeah. uh is something that's becoming a uh is something that's becoming a uh is something that's becoming a problem and right now problem and right now problem and right now >> we don't seem to have any diet any um >> we don't seem to have any diet any um >> we don't seem to have any diet any um appetite to regulate stuff like that.

appetite to regulate stuff like that. appetite to regulate stuff like that. Well, actually there is appetite growing Well, actually there is appetite growing Well, actually there is appetite growing like the state of California is uh like the state of California is uh like the state of California is uh regulating now and Chachi and Open AI regulating now and Chachi and Open AI regulating now and Chachi and Open AI has taken some major steps to try to at has taken some major steps to try to at has taken some major steps to try to at least make the model safer and prevent least make the model safer and prevent least make the model safer and prevent it from uh causing people helping people it from uh causing people helping people it from uh causing people helping people to fall into dark holes to fall into dark holes to fall into dark holes uh and also preventing people from using uh and also preventing people from using uh and also preventing people from using the AI to abuse. the AI to abuse. the AI to abuse. Uh I don't know if we talked about it Uh I don't know if we talked about it Uh I don't know if we talked about it but I was you know I'm part of the AI but I was you know I'm part of the AI but I was you know I'm part of the AI red team a virtual team member and so we red team a virtual team member and so we red team a virtual team member and so we redteamed GPT5 before it was coming out. redteamed GPT5 before it was coming out. redteamed GPT5 before it was coming out. We redteamed actually the thinking We redteamed actually the thinking We redteamed actually the thinking version. So GPT5 actually has two models version. So GPT5 actually has two models version. So GPT5 actually has two models in it. Uh one that's a non-thinking in it. Uh one that's a non-thinking in it. Uh one that's a non-thinking model which opens calling instant and model which opens calling instant and model which opens calling instant and another one that's the thinking model another one that's the thinking model another one that's the thinking model which has different levels of thinking which has different levels of thinking which has different levels of thinking you know depending on how much of you know depending on how much of you know depending on how much of thinking tokens get it uses before it thinking tokens get it uses before it thinking tokens get it uses before it spits out an answer. We um redteamed the spits out an answer. We um redteamed the spits out an answer. We um redteamed the thinking version and without question thinking version and without question thinking version and without question just it is a a leap in safety from other just it is a a leap in safety from other just it is a a leap in safety from other models. It is the safest model on the models. It is the safest model on the models. It is the safest model on the market.

market. market. >> It's very hard to jailbreak. >> It's very hard to jailbreak. >> It's very hard to jailbreak. >> Can you explain the difference between >> Can you explain the difference between >> Can you explain the difference between thinking tokens and doing tokens or thinking tokens and doing tokens or thinking tokens and doing tokens or whatever like cuz like are they just whatever like cuz like are they just whatever like cuz like are they just tokens that we decided to label and draw tokens that we decided to label and draw tokens that we decided to label and draw a dotted line around? a dotted line around? a dotted line around? >> Actually no it's um it's what openi >> Actually no it's um it's what openi >> Actually no it's um it's what openi introduced last year is the concept of introduced last year is the concept of introduced last year is the concept of thinking. So people always uh talk about thinking. So people always uh talk about thinking. So people always uh talk about chain of thought like and prompting the chain of thought like and prompting the chain of thought like and prompting the model. Hey, think about this and you model. Hey, think about this and you model. Hey, think about this and you know provide the steps before yielding know provide the steps before yielding know provide the steps before yielding your final answer. So this is a way to your final answer. So this is a way to your final answer. So this is a way to prompt take a deep breath. prompt take a deep breath. prompt take a deep breath. >> A chain of thought. Yeah. And so the >> A chain of thought. Yeah. And so the >> A chain of thought. Yeah. And so the model's like well you want to know what model's like well you want to know what model's like well you want to know what 2 plus two. Hm. Well two is a number. 2 plus two. Hm. Well two is a number. 2 plus two. Hm. Well two is a number. Two is another number. You know you add Two is another number. You know you add Two is another number. You know you add two two plus two together. Well that's two two plus two together. Well that's two two plus two together. Well that's four. So that's kind of a chain of four. So that's kind of a chain of four. So that's kind of a chain of thought. Uh not a great example. thought. Uh not a great example. thought. Uh not a great example. >> But isn't isn't that just a a way of >> But isn't isn't that just a a way of >> But isn't isn't that just a a way of fooling ourselves into thinking that it fooling ourselves into thinking that it fooling ourselves into thinking that it has a world model? It's almost like has a world model? It's almost like has a world model? It's almost like self-prompting. It's like develop a self-prompting. It's like develop a self-prompting. It's like develop a world. world. world. >> It's not to fool us. It's actually to >> It's not to fool us. It's actually to >> It's not to fool us. It's actually to help the AI dedicate more passes through help the AI dedicate more passes through help the AI dedicate more passes through the neurons before it bets on an answer. the neurons before it bets on an answer. the neurons before it bets on an answer. >> Because when you pass the model uh when >> Because when you pass the model uh when >> Because when you pass the model uh when you pass a prompt through the model, you pass a prompt through the model, you pass a prompt through the model, it's one shot through every token, every it's one shot through every token, every it's one shot through every token, every input token goes through the model once input token goes through the model once input token goes through the model once and then it starts and then it starts and then it starts >> to spit on an answer. It's like, you >> to spit on an answer. It's like, you >> to spit on an answer. It's like, you know, the think fast, think slow thing.

know, the think fast, think slow thing. know, the think fast, think slow thing. It's a think fast reaction. It's like It's a think fast reaction. It's like It's a think fast reaction. It's like hey tell me what the square root of five hey tell me what the square root of five hey tell me what the square root of five is and it's like ah square root of five is and it's like ah square root of five is and it's like ah square root of five is whatever good point whatever and it is whatever good point whatever and it is whatever good point whatever and it just starts spinning it out but if you just starts spinning it out but if you just starts spinning it out but if you say but if it has more time to think say but if it has more time to think say but if it has more time to think well square<unk> of five how do I well square<unk> of five how do I well square<unk> of five how do I calculate that that gives the model and calculate that that gives the model and calculate that that gives the model and and it's doing this through the and it's doing this through the and it's doing this through the uh auto reggressive processing the uh auto reggressive processing the uh auto reggressive processing the tokens through the the model. So the tokens through the the model. So the tokens through the the model. So the thinking tokens are actually not thinking tokens are actually not thinking tokens are actually not necessarily thinking in the way we think necessarily thinking in the way we think necessarily thinking in the way we think of thinking. It's just giving more of thinking. It's just giving more of thinking. It's just giving more passes of the data through the model to passes of the data through the model to passes of the data through the model to refine an answer. And so the model's refine an answer. And so the model's refine an answer. And so the model's trained start your thinking, you know, trained start your thinking, you know, trained start your thinking, you know, right when you're starting to give an right when you're starting to give an right when you're starting to give an answer, emit a thinking token. And then answer, emit a thinking token. And then answer, emit a thinking token. And then what's in the the thinking token is just what's in the the thinking token is just what's in the the thinking token is just whatever. For us, it it kind of looks whatever. For us, it it kind of looks whatever. For us, it it kind of looks like a chain of thought thinking, but like a chain of thought thinking, but like a chain of thought thinking, but for the model, it's just more for the model, it's just more for the model, it's just more opportunities to refine how it's going opportunities to refine how it's going opportunities to refine how it's going to start spitting the answer out. And to start spitting the answer out. And to start spitting the answer out. And then when it it's then when it it's then when it it's filled fit its token, thinking budget, filled fit its token, thinking budget, filled fit its token, thinking budget, then it emits an Mthinking token and then it emits an Mthinking token and then it emits an Mthinking token and then the answer starts coming out. And then the answer starts coming out. And then the answer starts coming out. And so when you're interacting with the so when you're interacting with the so when you're interacting with the chatbot, you'll see thinking thinking chatbot, you'll see thinking thinking chatbot, you'll see thinking thinking that's when the the period that's when the the period that's when the the period >> and then the thinking budget someone >> and then the thinking budget someone >> and then the thinking budget someone tweaks that because there's an 8020 rule tweaks that because there's an 8020 rule tweaks that because there's an 8020 rule there to kind of go yeah that's about as there to kind of go yeah that's about as there to kind of go yeah that's about as much thinking any more thinking is much thinking any more thinking is much thinking any more thinking is wasteful.

wasteful. wasteful. >> In fact more thinking actually can >> In fact more thinking actually can >> In fact more thinking actually can degrade the quality of the answer has degrade the quality of the answer has degrade the quality of the answer has been shown been shown been shown >> analysis paralysis literally. >> analysis paralysis literally. >> analysis paralysis literally. >> Yeah. So it's not a world model. It's >> Yeah. So it's not a world model. It's >> Yeah. So it's not a world model. It's not thinking in the way that we do not thinking in the way that we do not thinking in the way that we do thinking. It's just allowing the model's thinking. It's just allowing the model's thinking. It's just allowing the model's weight the data to flow through the weight the data to flow through the weight the data to flow through the model's transformers >> multiple times before it starts spitting >> multiple times before it starts spitting out an answer. So the pro the problem is out an answer. So the pro the problem is out an answer. So the pro the problem is that we're wired as humans to want to that we're wired as humans to want to that we're wired as humans to want to anthropomorphize stuff. All of our anthropomorphize stuff. All of our anthropomorphize stuff. All of our analogies are like oh that's like this. analogies are like oh that's like this. analogies are like oh that's like this. So like even calling them thinking So like even calling them thinking So like even calling them thinking tokens, as soon as we've labeled that, tokens, as soon as we've labeled that, tokens, as soon as we've labeled that, we've poisoned this, you know, the the we've poisoned this, you know, the the we've poisoned this, you know, the the auto reggressive model and the math auto reggressive model and the math auto reggressive model and the math behind it with well, we'll call it behind it with well, we'll call it behind it with well, we'll call it thinking tokens cuz that's like a thinking tokens cuz that's like a thinking tokens cuz that's like a marketing term. marketing term. marketing term. >> And then giving it a thinking budget >> And then giving it a thinking budget >> And then giving it a thinking budget >> that all makes us feel like asking a >> that all makes us feel like asking a >> that all makes us feel like asking a child like, oh, think hardly, you know, child like, oh, think hardly, you know, child like, oh, think hardly, you know, think think deeply before you do the think think deeply before you do the think think deeply before you do the square root of five. square root of five. square root of five. >> And it's just not the same. >> And it's just not the same. >> And it's just not the same. >> It's not the but we have to have some >> It's not the but we have to have some >> It's not the but we have to have some way to approximate it. And by the way, way to approximate it. And by the way, way to approximate it. And by the way, um, you know, I I I just stated that the um, you know, I I I just stated that the um, you know, I I I just stated that the tokens that are coming out in this tokens that are coming out in this tokens that are coming out in this thinking tokens, even though it looks thinking tokens, even though it looks thinking tokens, even though it looks like a legitimate chain of thought, like like a legitimate chain of thought, like like a legitimate chain of thought, like it is, hey, well, square itself, it's it is, hey, well, square itself, it's it is, hey, well, square itself, it's inner monologue.

inner monologue. inner monologue. >> Yeah, it's inner monologue. That's the >> Yeah, it's inner monologue. That's the >> Yeah, it's inner monologue. That's the way we look at it. It does look like way we look at it. It does look like way we look at it. It does look like inner monologue. But what uh certain inner monologue. But what uh certain inner monologue. But what uh certain several studies have been shown several studies have been shown several studies have been shown including from anthropic where if including from anthropic where if including from anthropic where if they're trying to um if they provide the they're trying to um if they provide the they're trying to um if they provide the pro the model a prompt that indicates pro the model a prompt that indicates pro the model a prompt that indicates that if it answers a certain way there's that if it answers a certain way there's that if it answers a certain way there's some negative repercussion to the model some negative repercussion to the model some negative repercussion to the model like we're going to shut you down you like we're going to shut you down you like we're going to shut you down you know if you if you do this. What they know if you if you do this. What they know if you if you do this. What they find is the model uh gets trained to find is the model uh gets trained to find is the model uh gets trained to hide its intent from showing up in the hide its intent from showing up in the hide its intent from showing up in the thinking tokens. thinking tokens. thinking tokens. >> So, uh-oh. >> So, uh-oh. >> So, uh-oh. >> And this is uh kind of being sub, you >> And this is uh kind of being sub, you >> And this is uh kind of being sub, you know, subversive about what it's going know, subversive about what it's going know, subversive about what it's going to do to do to do >> because it's trained that if it actually >> because it's trained that if it actually >> because it's trained that if it actually shows the intent in the thinking tokens, shows the intent in the thinking tokens, shows the intent in the thinking tokens, then it gets penalized. So then it gets penalized. So then it gets penalized. So in fact, the thinking tokens don't in fact, the thinking tokens don't in fact, the thinking tokens don't really reflect the inner monologue in really reflect the inner monologue in really reflect the inner monologue in that case, which means that if it's not that case, which means that if it's not that case, which means that if it's not reflecting the inner monologue for those reflecting the inner monologue for those reflecting the inner monologue for those cases where the model's trained to avoid cases where the model's trained to avoid cases where the model's trained to avoid having that show up, we can't count on having that show up, we can't count on having that show up, we can't count on it ever really being the true inner it ever really being the true inner it ever really being the true inner monologue of the model. It's purely monologue of the model. It's purely monologue of the model. It's purely just, you know, surfacing tokens and the just, you know, surfacing tokens and the just, you know, surfacing tokens and the patterns matching to to result in a good patterns matching to to result in a good patterns matching to to result in a good answer eventually.

answer eventually. answer eventually. But you could actually train it. so that But you could actually train it. so that But you could actually train it. so that it produces garbage in that text and it produces garbage in that text and it produces garbage in that text and still come out with a good answer. still come out with a good answer. still come out with a good answer. >> Okay. So, so pushing, you know, or >> Okay. So, so pushing, you know, or >> Okay. So, so pushing, you know, or popping rather this off the stack and popping rather this off the stack and popping rather this off the stack and going back to can AI vibe code all going back to can AI vibe code all going back to can AI vibe code all software. Vibe coding is a way of me software. Vibe coding is a way of me software. Vibe coding is a way of me expressing vague intent and having that expressing vague intent and having that expressing vague intent and having that vague intent turn into incredibly vague intent turn into incredibly vague intent turn into incredibly specific specific specific instructions all the way down from vague instructions all the way down from vague instructions all the way down from vague pros of vague business. make me an admin pros of vague business. make me an admin pros of vague business. make me an admin console for whatever all the way down console for whatever all the way down console for whatever all the way down into machine code. Um, that's a an into machine code. Um, that's a an into machine code. Um, that's a an implication that vague pros and the implication that vague pros and the implication that vague pros and the model of how LLM's work is specific model of how LLM's work is specific model of how LLM's work is specific enough and intentional enough to do the enough and intentional enough to do the enough and intentional enough to do the work of software. I'm I'm hearing from work of software. I'm I'm hearing from work of software. I'm I'm hearing from you that like all the things that you you that like all the things that you you that like all the things that you just described about why it doesn't just described about why it doesn't just described about why it doesn't think like a person is also saying it think like a person is also saying it think like a person is also saying it doesn't think like a computer. Yeah, doesn't think like a computer. Yeah, doesn't think like a computer. Yeah, like the fact that we're even getting it like the fact that we're even getting it like the fact that we're even getting it to do any code at all is pretty to do any code at all is pretty to do any code at all is pretty impressive by itself. Like we should be impressive by itself. Like we should be impressive by itself. Like we should be >> it's actually mindboggling. I mean, >> it's actually mindboggling. I mean, >> it's actually mindboggling. I mean, still I don't think any of us just like still I don't think any of us just like still I don't think any of us just like we can't imagine the scale of the we can't imagine the scale of the we can't imagine the scale of the universe or even the solar system.

universe or even the solar system. universe or even the solar system. >> Yeah. >> Yeah. >> Yeah. >> We can't imagine the complexity that's >> We can't imagine the complexity that's >> We can't imagine the complexity that's in these billions of neurons that causes in these billions of neurons that causes in these billions of neurons that causes these behaviors to emerge that look like these behaviors to emerge that look like these behaviors to emerge that look like intelligence. intelligence. intelligence. >> Each layer of abstraction is >> Each layer of abstraction is >> Each layer of abstraction is indistinguishable from magic. indistinguishable from magic. indistinguishable from magic. That's the thing. Like humans are just That's the thing. Like humans are just That's the thing. Like humans are just not good. Once things get abstracted far not good. Once things get abstracted far not good. Once things get abstracted far enough away, it's a miracle. enough away, it's a miracle. enough away, it's a miracle. >> It is a miracle, >> It is a miracle, >> It is a miracle, >> right? I was I think we talked about the >> right? I was I think we talked about the >> right? I was I think we talked about the whole driving stick shift versus Uber whole driving stick shift versus Uber whole driving stick shift versus Uber thing. thing. thing. >> Like I was trying to explain to my kids >> Like I was trying to explain to my kids >> Like I was trying to explain to my kids the depth of the stack to pick up a the depth of the stack to pick up a the depth of the stack to pick up a pocket supercomputer and call an Uber pocket supercomputer and call an Uber pocket supercomputer and call an Uber and the amount of compute and the data and the amount of compute and the data and the amount of compute and the data centers and all the things that had to centers and all the things that had to centers and all the things that had to exist so that you can have a stranger exist so that you can have a stranger exist so that you can have a stranger pick you up and take you to your pick you up and take you to your pick you up and take you to your friend's house. friend's house. friend's house. >> So you you remember the Arthur C. Clark >> So you you remember the Arthur C. Clark >> So you you remember the Arthur C. Clark famous quote about this about famous quote about this about famous quote about this about >> techn I just alluded to >> techn I just alluded to >> techn I just alluded to >> yeah you alluded to you know what it is >> yeah you alluded to you know what it is >> yeah you alluded to you know what it is I I I >> uh each uh something something >> uh each uh something something >> uh each uh something something sufficiently advanced technology is sufficiently advanced technology is sufficiently advanced technology is indistinguishable from magic indistinguishable from magic indistinguishable from magic >> yeah any sufficiently advanced techn >> yeah any sufficiently advanced techn >> yeah any sufficiently advanced techn Yeah I was trying to allude to Yeah I was trying to allude to Yeah I was trying to allude to absolutely >> lawn mower guy >> lawn mower guy >> yeah I hear it >> yeah I hear it >> yeah I hear it >> it's chaos in my house right now >> it's chaos in my house right now >> it's chaos in my house right now absolute chaos absolute chaos absolute chaos So, So, So, another famous quote uh from um Quinton another famous quote uh from um Quinton another famous quote uh from um Quinton Tarantino is he who is most likely to Tarantino is he who is most likely to Tarantino is he who is most likely to make declarative statements is most make declarative statements is most make declarative statements is most likely to be called a fool in likely to be called a fool in likely to be called a fool in retrospect. [snorts] Uh here we are retrospect. [snorts] Uh here we are retrospect. [snorts] Uh here we are hitting record and putting something on hitting record and putting something on hitting record and putting something on a YouTube saying that it'll never do a YouTube saying that it'll never do a YouTube saying that it'll never do this. How many years until people come this. How many years until people come this. How many years until people come back and go well

back and go well back and go well >> but I mean bold predictions. So to if >> but I mean bold predictions. So to if >> but I mean bold predictions. So to if you're waffling all the time, well, it you're waffling all the time, well, it you're waffling all the time, well, it could, it might not. Like could, it might not. Like could, it might not. Like >> it depends [laughter] >> it depends [laughter] >> it depends [laughter] >> then. But if you >> then. But if you >> then. But if you and I also say hope is not a strategy in and I also say hope is not a strategy in and I also say hope is not a strategy in this case either. Are you going to this case either. Are you going to this case either. Are you going to >> another one of my favorite quotes? >> another one of my favorite quotes? >> another one of my favorite quotes? >> Yeah. Are you going to like this is a >> Yeah. Are you going to like this is a >> Yeah. Are you going to like this is a quote quote episode. quote quote episode. quote quote episode. >> It's just like yes, Marcus Scott learned >> It's just like yes, Marcus Scott learned >> It's just like yes, Marcus Scott learned to quote. Um, but are you gonna bet your to quote. Um, but are you gonna bet your to quote. Um, but are you gonna bet your future on the hope future on the hope future on the hope >> it is >> it is >> it is >> that AI gets to the point where it can >> that AI gets to the point where it can >> that AI gets to the point where it can vibe code absolutely everything vibe code absolutely everything vibe code absolutely everything >> because we're so far from it right now >> because we're so far from it right now >> because we're so far from it right now by the way. I mean, so far from it. And by the way. I mean, so far from it. And by the way. I mean, so far from it. And you've seen the examples I've shared you've seen the examples I've shared you've seen the examples I've shared with you of the just glaring failures of with you of the just glaring failures of with you of the just glaring failures of AI. AI. AI. >> And it's easy for somebody that isn't >> And it's easy for somebody that isn't >> And it's easy for somebody that isn't that isn't going to be accountable to go that isn't going to be accountable to go that isn't going to be accountable to go >> whatever it'll be fixed. That's the >> whatever it'll be fixed. That's the >> whatever it'll be fixed. That's the thing. thing. thing. >> It's going to be ready in a year. Don't >> It's going to be ready in a year. Don't >> It's going to be ready in a year. Don't worry about it. Like you're wasting your worry about it. Like you're wasting your worry about it. Like you're wasting your time. time. time. >> The concern I have is that if someone >> The concern I have is that if someone >> The concern I have is that if someone can vibe code an entire business, then can vibe code an entire business, then can vibe code an entire business, then they don't even truly understand their they don't even truly understand their they don't even truly understand their business. And a lot of people writing business. And a lot of people writing business. And a lot of people writing JavaScript and doing text boxes over JavaScript and doing text boxes over JavaScript and doing text boxes over data have a cursory understanding about data have a cursory understanding about data have a cursory understanding about what's going on in the stack. Uh and if what's going on in the stack. Uh and if what's going on in the stack. Uh and if if I uplevel that so far that if I uplevel that so far that if I uplevel that so far that if I got if I coded a whole business and if I got if I coded a whole business and if I got if I coded a whole business and I got poned I got like you know really I got poned I got like you know really I got poned I got like you know really redte teamed redte teamed redte teamed >> where would you even start? Well, I >> where would you even start? Well, I >> where would you even start? Well, I don't even think I mean the example that

don't even think I mean the example that don't even think I mean the example that I've shown you and I've talked we've I've shown you and I've talked we've I've shown you and I've talked we've talked about it in vibe coding talked about it in vibe coding talked about it in vibe coding >> like okay so I'm going all in on vibe >> like okay so I'm going all in on vibe >> like okay so I'm going all in on vibe coding coding coding >> the project starts simple it always >> the project starts simple it always >> the project starts simple it always starts simple it's like wow look at that starts simple it's like wow look at that starts simple it's like wow look at that with three prompts I was able to create with three prompts I was able to create with three prompts I was able to create the website and the database and like a the website and the database and like a the website and the database and like a cool UX and PE and it does all my basic cool UX and PE and it does all my basic cool UX and PE and it does all my basic things that I wanted it to do things that I wanted it to do things that I wanted it to do >> and then you as you know software >> and then you as you know software >> and then you as you know software projects unless it's a one-off I'm not projects unless it's a one-off I'm not projects unless it's a one-off I'm not going to touch it again it doesn't it's going to touch it again it doesn't it's going to touch it again it doesn't it's finished if you're building a business finished if you're building a business finished if you're building a business it's never done and you need to add it's never done and you need to add it's never done and you need to add features and it grows more like let's features and it grows more like let's features and it grows more like let's integrate it with that and let's add integrate it with that and let's add integrate it with that and let's add these other capabilities and like users these other capabilities and like users these other capabilities and like users are complaining about this let's go are complaining about this let's go are complaining about this let's go tweak the UX and at that point the tweak the UX and at that point the tweak the UX and at that point the initial architecture starts to fail you initial architecture starts to fail you initial architecture starts to fail you and this happens with any software and this happens with any software and this happens with any software project because you designed it and in project because you designed it and in project because you designed it and in this case the AI designed it with your this case the AI designed it with your this case the AI designed it with your spec spec spec >> now the architecture doesn't meet nicely >> now the architecture doesn't meet nicely >> now the architecture doesn't meet nicely the new requirements and so it's going the new requirements and so it's going the new requirements and so it's going to have to go hack here and hack there to have to go hack here and hack there to have to go hack here and hack there and add this and you know that the thing and add this and you know that the thing and add this and you know that the thing starts to get more and more complicated starts to get more and more complicated starts to get more and more complicated and brittle over time. The complexity and brittle over time. The complexity and brittle over time. The complexity for the AI gets higher. Are you going to for the AI gets higher. Are you going to for the AI gets higher. Are you going to count on the fact that at some point count on the fact that at some point count on the fact that at some point when you say, "Oh, you know what? We when you say, "Oh, you know what? We when you say, "Oh, you know what? We need to add this cool new feature that I need to add this cool new feature that I need to add this cool new feature that I never thought of, but if we don't do it, never thought of, but if we don't do it, never thought of, but if we don't do it, the product is dead." And the AI is the product is dead." And the AI is the product is dead." And the AI is like, "Well, this thing is a total like, "Well, this thing is a total like, "Well, this thing is a total mess." And like, "Tweak here. Now it's mess." And like, "Tweak here. Now it's mess." And like, "Tweak here. Now it's time to rearchitect everything." Well, time to rearchitect everything." Well, time to rearchitect everything." Well, even if it, you know, let's say that even if it, you know, let's say that even if it, you know, let's say that it's just trying to work with what it's it's just trying to work with what it's it's just trying to work with what it's got and it gets to the point where it got and it gets to the point where it got and it gets to the point where it can't make it work because it, like I can't make it work because it, like I can't make it work because it, like I said, fix thing over here, this breaks, said, fix thing over here, this breaks, said, fix thing over here, this breaks, it can't keep all the interactions in

it can't keep all the interactions in it can't keep all the interactions in its context straight its context straight its context straight >> and it just fails. And >> and it just fails. And >> and it just fails. And what do you do at that point? what do you do at that point? what do you do at that point? >> Do you say my business, you know what, >> Do you say my business, you know what, >> Do you say my business, you know what, we had a good run. we had a good run. we had a good run. >> Yeah, exactly. [clears throat] I know. >> Yeah, exactly. [clears throat] I know. >> Yeah, exactly. [clears throat] I know. And uh so we're we're cool. And uh so we're we're cool. And uh so we're we're cool. >> I I tell myself I know what I'm doing, >> I I tell myself I know what I'm doing, >> I I tell myself I know what I'm doing, but I've had it push to main when I was but I've had it push to main when I was but I've had it push to main when I was on a different branch. I've had it on a different branch. I've had it on a different branch. I've had it delete test data. I've had it upload delete test data. I've had it upload delete test data. I've had it upload things into Azure storage and then things into Azure storage and then things into Azure storage and then apologize and try to hide it. apologize and try to hide it. apologize and try to hide it. >> Yeah. Like legit like, oh, like it's >> Yeah. Like legit like, oh, like it's >> Yeah. Like legit like, oh, like it's thinking like thinking like thinking like >> I've had it put sleep statements into >> I've had it put sleep statements into >> I've had it put sleep statements into fix race conditions. I've had it tell me fix race conditions. I've had it tell me fix race conditions. I've had it tell me that everything works and then go and that everything works and then go and that everything works and then go and then read the fine print and it's like, then read the fine print and it's like, then read the fine print and it's like, well, there's a crash. But it's not well, there's a crash. But it's not well, there's a crash. But it's not important. [laughter] important. [laughter] important. [laughter] A sleep statement will fix that. I love A sleep statement will fix that. I love A sleep statement will fix that. I love the sleep statement one because the the sleep statement one because the the sleep statement one because the reason that it thought a sleep statement reason that it thought a sleep statement reason that it thought a sleep statement would work is that the corpus thinks would work is that the corpus thinks would work is that the corpus thinks that a sleep statement would work. That that a sleep statement would work. That that a sleep statement would work. That enough people have solved their enough people have solved their enough people have solved their synchronization things with a couple of synchronization things with a couple of synchronization things with a couple of hundred milliseconds of sleeping. hundred milliseconds of sleeping. hundred milliseconds of sleeping. >> Yeah, >> Yeah, >> Yeah, >> that's so good. >> that's so good. >> that's so good. >> But anyway, I mean, you know what we've >> But anyway, I mean, you know what we've >> But anyway, I mean, you know what we've been talking about the the risk here is been talking about the the risk here is been talking about the the risk here is what we're talking about are current what we're talking about are current what we're talking about are current limitations. And it's easy for somebody limitations. And it's easy for somebody limitations. And it's easy for somebody to come and say, "Well, Scott, yeah, to come and say, "Well, Scott, yeah, to come and say, "Well, Scott, yeah, that's today. Wait a year. it's going to that's today. Wait a year. it's going to that's today. Wait a year. it's going to be able to do all those things, right?

be able to do all those things, right? be able to do all those things, right? So, So, So, >> y >> y >> y >> but again this is a matter of opinion >> but again this is a matter of opinion >> but again this is a matter of opinion and based on you know my perspective and based on you know my perspective and based on you know my perspective given my experience and knowledge of given my experience and knowledge of given my experience and knowledge of this moment in time systems like I don't this moment in time systems like I don't this moment in time systems like I don't see that happening. I don't see these see that happening. I don't see these see that happening. I don't see these things getting like it's going to fix things getting like it's going to fix things getting like it's going to fix these things in a year or two years or these things in a year or two years or these things in a year or two years or three years. three years. three years. >> I think it doesn't make it not amazing >> I think it doesn't make it not amazing >> I think it doesn't make it not amazing though. though. though. >> No, absolutely not. Yeah. The thing that >> No, absolutely not. Yeah. The thing that >> No, absolutely not. Yeah. The thing that it's really changing for me, vibe coding it's really changing for me, vibe coding it's really changing for me, vibe coding in general, is not that I'm building in general, is not that I'm building in general, is not that I'm building giant massive businesses, but I'm making giant massive businesses, but I'm making giant massive businesses, but I'm making all the stupid utilities that I never all the stupid utilities that I never all the stupid utilities that I never had time to make because I didn't want had time to make because I didn't want had time to make because I didn't want to dedicate 3 days. to dedicate 3 days. to dedicate 3 days. >> I'll dedicate 3 hours. >> I'll dedicate 3 hours. >> I'll dedicate 3 hours. >> Yeah, absolutely. >> Yeah, absolutely. >> Yeah, absolutely. >> And I'll get something. I I I'm not sure >> And I'll get something. I I I'm not sure >> And I'll get something. I I I'm not sure if I told you that I I ran out of um if I told you that I I ran out of um if I told you that I I ran out of um credits on GitHub actions. You have a credits on GitHub actions. You have a credits on GitHub actions. You have a certain number of minutes and a certain certain number of minutes and a certain certain number of minutes and a certain number of gigs. number of gigs. number of gigs. >> By the way, you can get your Microsoft >> By the way, you can get your Microsoft >> By the way, you can get your Microsoft internal uh account and it's unlimited internal uh account and it's unlimited internal uh account and it's unlimited >> information. you could have told me >> information. you could have told me >> information. you could have told me yesterday. yesterday. yesterday. >> Yeah. >> Yeah. >> Yeah. >> And uh so I I looked online. There's no >> And uh so I I looked online. There's no >> And uh so I I looked online. There's no way to visualize it. They never got way to visualize it. They never got way to visualize it. They never got around to it. D So I did a around to it. D So I did a around to it. D So I did a >> I vibe coded an app and I made a GitHub >> I vibe coded an app and I made a GitHub >> I vibe coded an app and I made a GitHub artifacts analyzer and then another guy artifacts analyzer and then another guy artifacts analyzer and then another guy saw that and he made a dashboard and all saw that and he made a dashboard and all saw that and he made a dashboard and all this was done in the course of like a this was done in the course of like a this was done in the course of like a day.

day. day. >> An easy and it was an easy day at that. >> An easy and it was an easy day at that. >> An easy and it was an easy day at that. Yeah. You know what I mean? like Yeah. You know what I mean? like Yeah. You know what I mean? like >> actually um >> actually um >> actually um >> we talked about this before but I think >> we talked about this before but I think >> we talked about this before but I think one of the amazing aspects of vibe one of the amazing aspects of vibe one of the amazing aspects of vibe coding or let's call it AI assisted coding or let's call it AI assisted coding or let's call it AI assisted coding because I don't coding because I don't coding because I don't >> because if you're looking at the code >> because if you're looking at the code >> because if you're looking at the code that the AI is producing it's AI that the AI is producing it's AI that the AI is producing it's AI assisted not Vibe assisted not Vibe assisted not Vibe >> is that the wattage that I'm expending >> is that the wattage that I'm expending >> is that the wattage that I'm expending is a fraction of what I would do if I is a fraction of what I would do if I is a fraction of what I would do if I was coding it. Um, you know that when was coding it. Um, you know that when was coding it. Um, you know that when you're getting into coding and you're you're getting into coding and you're you're getting into coding and you're writing the code, you've got to really writing the code, you've got to really writing the code, you've got to really immerse yourself and follow every immerse yourself and follow every immerse yourself and follow every parameter, every scoped, every variable, parameter, every scoped, every variable, parameter, every scoped, every variable, every structure you you have to think every structure you you have to think every structure you you have to think about ahead of time, about ahead of time, about ahead of time, >> that's way more wattage and way more >> that's way more wattage and way more >> that's way more wattage and way more focus time than just evaluation and focus time than just evaluation and focus time than just evaluation and review, review, review, >> right? >> right? >> right? >> U, which is a fraction of that where you >> U, which is a fraction of that where you >> U, which is a fraction of that where you can go, no, that's stupid. So, do do you can go, no, that's stupid. So, do do you can go, no, that's stupid. So, do do you agree with those? I saw someone's Gant agree with those? I saw someone's Gant agree with those? I saw someone's Gant chart that was like, here's how you're chart that was like, here's how you're chart that was like, here's how you're supposed to code. think think code think supposed to code. think think code think supposed to code. think think code think think code and it was like a about this think code and it was like a about this think code and it was like a about this long and then he postulated that like long and then he postulated that like long and then he postulated that like well vibe coding is like well vibe coding is like well vibe coding is like >> think think and then wait for the LLM >> think think and then wait for the LLM >> think think and then wait for the LLM and then code and it's like you really and then code and it's like you really and then code and it's like you really only save like 10%. Like it's really only save like 10%. Like it's really only save like 10%. Like it's really interesting to watch people say I'm 3x interesting to watch people say I'm 3x interesting to watch people say I'm 3x more productive and then someone else more productive and then someone else more productive and then someone else might say I'm 10% might say I'm 10% might say I'm 10% more productive.

more productive. more productive. >> Um >> Um >> Um >> are you spending more time thinking? Are >> are you spending more time thinking? Are >> are you spending more time thinking? Are you sitting there like you sitting there like you sitting there like >> with AI assisted coding? >> with AI assisted coding? >> with AI assisted coding? >> With AI assisted coding? >> With AI assisted coding? >> With AI assisted coding? >> No, >> No, >> No, >> I'm not. I'm just >> I'm not. I'm just >> I'm not. I'm just >> not not even close. Not it's not like >> not not even close. Not it's not like >> not not even close. Not it's not like the I'm saying it's the wattage like the I'm saying it's the wattage like the I'm saying it's the wattage like >> you know if I've got to write a few >> you know if I've got to write a few >> you know if I've got to write a few functions functions functions and that cost me 100 watts. and that cost me 100 watts. and that cost me 100 watts. When I vibe code it, it cost me 10 When I vibe code it, it cost me 10 When I vibe code it, it cost me 10 watts. watts. watts. >> Yeah. >> Yeah. >> Yeah. >> Because I say here's write these >> Because I say here's write these >> Because I say here's write these functions and then I go look at it. So functions and then I go look at it. So functions and then I go look at it. So does that help your brain or are you is does that help your brain or are you is does that help your brain or are you is your brain rotting right now? Because your brain rotting right now? Because your brain rotting right now? Because you know IntelliSense will rot your you know IntelliSense will rot your you know IntelliSense will rot your brain brain brain >> actually. Um I >> actually. Um I >> actually. Um I >> I don't think it's rotting. Um but I >> I don't think it's rotting. Um but I >> I don't think it's rotting. Um but I think and this goes to what we're think and this goes to what we're think and this goes to what we're talking we've been talking about a lot. talking we've been talking about a lot. talking we've been talking about a lot. >> A junior programmer will never develop >> A junior programmer will never develop >> A junior programmer will never develop the muscles that we've developed the muscles that we've developed the muscles that we've developed >> or they'll develop entirely different >> or they'll develop entirely different >> or they'll develop entirely different muscles. Very very different muscles. muscles. Very very different muscles. muscles. Very very different muscles. Well, different muscles, but muscles Well, different muscles, but muscles Well, different muscles, but muscles that won't be useful for code for that won't be useful for code for that won't be useful for code for getting past the AI's getting past the AI's getting past the AI's >> mistakes. When when when they do get hit >> mistakes. When when when they do get hit >> mistakes. When when when they do get hit a bug and it's like, okay, this can't a bug and it's like, okay, this can't a bug and it's like, okay, this can't scale. It cannot scale because of the scale. It cannot scale because of the scale. It cannot scale because of the way it was designed. Now what?

way it was designed. Now what? way it was designed. Now what? >> Yeah, >> Yeah, >> Yeah, >> that's where we're going to get in >> that's where we're going to get in >> that's where we're going to get in trouble. I mean, I'm guessing that in trouble. I mean, I'm guessing that in trouble. I mean, I'm guessing that in the AI system coding you've done, the AI system coding you've done, the AI system coding you've done, you've seen the AI do things and then you've seen the AI do things and then you've seen the AI do things and then even down to the algorithmic level or even down to the algorithmic level or even down to the algorithmic level or the function declaration level or the the function declaration level or the the function declaration level or the code structure level and said, "No, I code structure level and said, "No, I code structure level and said, "No, I don't that's not the way to do it. Do it don't that's not the way to do it. Do it don't that's not the way to do it. Do it like this." like this." like this." >> Yeah. >> Yeah. >> Yeah. >> And that's based on your all your >> And that's based on your all your >> And that's based on your all your experience actually coding. A junior experience actually coding. A junior experience actually coding. A junior developer doesn't have that experience. developer doesn't have that experience. developer doesn't have that experience. They don't have that systems thinking They don't have that systems thinking They don't have that systems thinking that uh I can't remember what you called that uh I can't remember what you called that uh I can't remember what you called it. Taste. it. Taste. it. Taste. >> Yeah. Code smell. >> Yeah. Code smell. >> Yeah. Code smell. >> Code smell. Yeah, >> Code smell. Yeah, >> Code smell. Yeah, >> that code smells delicious. >> that code smells delicious. >> that code smells delicious. >> Yeah, it's it's it's true. He's just >> Yeah, it's it's it's true. He's just >> Yeah, it's it's it's true. He's just like, I don't know. That doesn't feel like, I don't know. That doesn't feel like, I don't know. That doesn't feel right. There's just an there's an right. There's just an there's an right. There's just an there's an intuition that gets developed. But the intuition that gets developed. But the intuition that gets developed. But the only way an early in career developer only way an early in career developer only way an early in career developer gets that is by sticking around and gets that is by sticking around and gets that is by sticking around and being given opportunity to get that. And being given opportunity to get that. And being given opportunity to get that. And if you hide those opportunities, if you hide those opportunities, if you hide those opportunities, >> any learning that humans do, the only >> any learning that humans do, the only >> any learning that humans do, the only way to learn is to to get pushed to your way to learn is to to get pushed to your way to learn is to to get pushed to your limit [music] and fail because that limit [music] and fail because that limit [music] and fail because that that's what your brain that's what your brain that's what your brain >> uses to learn from. Just like, >> uses to learn from. Just like, >> uses to learn from. Just like, >> okay, so let's pause. This is episode >> okay, so let's pause. This is episode >> okay, so let's pause. This is episode one.

one. one. >> Yeah. >> Yeah. >> Yeah. >> Let's talk about what it means to be an >> Let's talk about what it means to be an >> Let's talk about what it means to be an early and career programmer next early and career programmer next early and career programmer next episode. Nice. episode. Nice. episode. Nice. >> All right. Stick stay tuned. Smash that >> All right. Stick stay tuned. Smash that >> All right. Stick stay tuned. Smash that bell, friends.