EPISODE 15 - Scott & Mark Learn To… Vibe coding and its limits (Part 2)

So one of the things about the vibe So one of the things about the vibe coding is we didn't talk about you know coding is we didn't talk about you know coding is we didn't talk about you know day one and vibe coding implications and day one and vibe coding implications and day one and vibe coding implications and complexity and SC you know which I we complexity and SC you know which I we complexity and SC you know which I we touched on. Well so we could we could do touched on. Well so we could we could do touched on. Well so we could we could do a vibe coding part two which is just us a vibe coding part two which is just us a vibe coding part two which is just us talking about brownfield green field talking about brownfield green field talking about brownfield green field what's your goal what's the difference what's your goal what's the difference what's your goal what's the difference between you're doing vibe coding as a between you're doing vibe coding as a between you're doing vibe coding as a hobby versus vibe coding for the hobby versus vibe coding for the hobby versus vibe coding for the enterprise versus vi coding for a enterprise versus vi coding for a enterprise versus vi coding for a startup. First five episodes about how much time First five episodes about how much time I spent not paying attention to you. I spent not paying attention to you. I spent not paying attention to you. That's true. Turnabouts fair play. So in That's true. Turnabouts fair play. So in That's true. Turnabouts fair play. So in our last episode, we vibecoded without our last episode, we vibecoded without our last episode, we vibecoded without realizing it, right? And this idea that realizing it, right? And this idea that realizing it, right? And this idea that one can write code with AI and not one can write code with AI and not one can write code with AI and not really worry about the results or the really worry about the results or the really worry about the results or the correctness, but rather worry about it correctness, but rather worry about it correctness, but rather worry about it worked. It it we wanted to make a worked. It it we wanted to make a worked. It it we wanted to make a fractal shader for Windows Terminal with fractal shader for Windows Terminal with fractal shader for Windows Terminal with minimal if not any if not zero uh minimal if not any if not zero uh minimal if not any if not zero uh experience with any of those things. And experience with any of those things. And experience with any of those things. And we achieved it in less than 15 minutes.

we achieved it in less than 15 minutes. we achieved it in less than 15 minutes. Is that healthy? Is that healthy? Is that healthy? That's fun. That's fun. That's fun. [Laughter] [Laughter] [Laughter] Um, how long do you think it would have Um, how long do you think it would have Um, how long do you think it would have taken you to do taken you to do taken you to do that in real world? I I would have taken that in real world? I I would have taken that in real world? I I would have taken me four to six hours and I would have me four to six hours and I would have me four to six hours and I would have been ended up eight hours because I been ended up eight hours because I been ended up eight hours because I would Yeah. Yeah. I think you're about would Yeah. Yeah. I think you're about would Yeah. Yeah. I think you're about right. You're probably right. Um, right. You're probably right. Um, right. You're probably right. Um, it would have taken about that time it would have taken about that time it would have taken about that time because I would need to first I would I because I would need to first I would I because I would need to first I would I would chop that problem. So the problem would chop that problem. So the problem would chop that problem. So the problem was make a fractal shader using the was make a fractal shader using the was make a fractal shader using the HLSL HS uh HLSL language and then figure HLSL HS uh HLSL language and then figure HLSL HS uh HLSL language and then figure out how to do it in Windows Terminal. So out how to do it in Windows Terminal. So out how to do it in Windows Terminal. So um these HLSL shaders are high level um these HLSL shaders are high level um these HLSL shaders are high level shader language, right, that you use to shader language, right, that you use to shader language, right, that you use to do programmable shaders in DirectX. It's do programmable shaders in DirectX. It's do programmable shaders in DirectX. It's a proprietary language, but it's very a proprietary language, but it's very a proprietary language, but it's very well known. And there's also GLSL. Um well known. And there's also GLSL. Um well known. And there's also GLSL. Um gamer people, people who write games and gamer people, people who write games and gamer people, people who write games and stuff like that make shaders all day. stuff like that make shaders all day. stuff like that make shaders all day. They could just type that stuff from They could just type that stuff from They could just type that stuff from scratch. But um we wanted to do a scratch. But um we wanted to do a scratch. But um we wanted to do a fractal. So I would go and find some fractal. So I would go and find some fractal. So I would go and find some fractal generating math, translate it fractal generating math, translate it fractal generating math, translate it into HLSL or find an existing chunk.

into HLSL or find an existing chunk. into HLSL or find an existing chunk. Then there was the Windows Then there was the Windows Then there was the Windows terminal scaffolding that needed to go terminal scaffolding that needed to go terminal scaffolding that needed to go around it. Windows terminal specific around it. Windows terminal specific around it. Windows terminal specific pound defines and stuff like that. Um pound defines and stuff like that. Um pound defines and stuff like that. Um and Windows terminal doesn't give you a and Windows terminal doesn't give you a and Windows terminal doesn't give you a lot of um debugging options. It just lot of um debugging options. It just lot of um debugging options. It just goes nope that failed to compile and it goes nope that failed to compile and it goes nope that failed to compile and it gives you a line number. So then I would gives you a line number. So then I would gives you a line number. So then I would have probably built a test harness or have probably built a test harness or have probably built a test harness or found some DirectX test harness like it found some DirectX test harness like it found some DirectX test harness like it would have been tedious. Yeah. would have been tedious. Yeah. would have been tedious. Yeah. Um, were you surprised how fast we are Um, were you surprised how fast we are Um, were you surprised how fast we are able to get that working? No. In fact, able to get that working? No. In fact, able to get that working? No. In fact, that's why I suggested just give the that's why I suggested just give the that's why I suggested just give the file and tell it to do this. Um, I I file and tell it to do this. Um, I I file and tell it to do this. Um, I I thought there was an 80% chance it would thought there was an 80% chance it would thought there was an 80% chance it would work the first time. Did you think it work the first time. Did you think it work the first time. Did you think it needed the previous file? We worked from needed the previous file? We worked from needed the previous file? We worked from one that had that had no cont. I do. I think it may maybe not. But I I do. I think it may maybe not. But I thought if the more information you can thought if the more information you can thought if the more information you can get it give it to get it on track the get it give it to get it on track the get it give it to get it on track the more reliable the more likely it's going more reliable the more likely it's going more reliable the more likely it's going to succeed I think. So we could have to succeed I think. So we could have to succeed I think. So we could have said just go create one from scratch. I said just go create one from scratch. I said just go create one from scratch. I think that my I would have estimated the think that my I would have estimated the think that my I would have estimated the chance of success at lower not zero but chance of success at lower not zero but chance of success at lower not zero but not 80 to 90 which is what we got. Okay.

not 80 to 90 which is what we got. Okay. not 80 to 90 which is what we got. Okay. And I actually expected it to get the And I actually expected it to get the And I actually expected it to get the fractal, you know, design wrong or or I fractal, you know, design wrong or or I fractal, you know, design wrong or or I kind of expected maybe it's just going kind of expected maybe it's just going kind of expected maybe it's just going to show a static fractal, not a moving to show a static fractal, not a moving to show a static fractal, not a moving shifting one or that it would use this shifting one or that it would use this shifting one or that it would use this fixed set of fixed colors instead of the fixed set of fixed colors instead of the fixed set of fixed colors instead of the dynamic colors that it was using. So it dynamic colors that it was using. So it dynamic colors that it was using. So it actually, you know, achieved everything actually, you know, achieved everything actually, you know, achieved everything that I would have hoped for in one shot. that I would have hoped for in one shot. that I would have hoped for in one shot. And did you and to be clear I think you And did you and to be clear I think you And did you and to be clear I think you know as our four to six hours of work know as our four to six hours of work know as our four to six hours of work would have been like okay so here's I've would have been like okay so here's I've would have been like okay so here's I've got a basic fractal now let me make it got a basic fractal now let me make it got a basic fractal now let me make it zoom in and out and pan and how do you zoom in and out and pan and how do you zoom in and out and pan and how do you do that? Oh, now how do you make it do that? Oh, now how do you make it do that? Oh, now how do you make it change colors? change colors? change colors? And you know, I think I imagine that's And you know, I think I imagine that's And you know, I think I imagine that's the process that I would have um now we the process that I would have um now we the process that I would have um now we we we edited just a little bit because I we we edited just a little bit because I we we edited just a little bit because I picked the wrong model before. Uh you picked the wrong model before. Uh you picked the wrong model before. Uh you said pick Claude 37 and I picked Claude said pick Claude 37 and I picked Claude said pick Claude 37 and I picked Claude 37 thinking. Yeah. And it did not work 37 thinking. Yeah. And it did not work 37 thinking. Yeah. And it did not work on the first try. Why do you think that on the first try. Why do you think that on the first try. Why do you think that was?

was? was? Um I think Um I think Um I think the the thinking models are you know the the thinking models are you know the the thinking models are you know actually it's I'm not sure because actually it's I'm not sure because actually it's I'm not sure because um most of the model makers tell you um most of the model makers tell you um most of the model makers tell you that the thinking models are better at that the thinking models are better at that the thinking models are better at coding. Mhm. So theoretically it should coding. Mhm. So theoretically it should coding. Mhm. So theoretically it should have done a better job and maybe we just have done a better job and maybe we just have done a better job and maybe we just got unlucky on that one and lucky on the got unlucky on that one and lucky on the got unlucky on that one and lucky on the non-thinking prompt because that's non-thinking prompt because that's non-thinking prompt because that's another aspect of this. It's completely another aspect of this. It's completely another aspect of this. It's completely non-deterministic. And if who's to say non-deterministic. And if who's to say non-deterministic. And if who's to say that we didn't happen on the that we didn't happen on the that we didn't happen on the non-thinking prompt where it achieved non-thinking prompt where it achieved non-thinking prompt where it achieved everything we wanted 100% the first everything we wanted 100% the first everything we wanted 100% the first time, that that was a 10% likelihood time, that that was a 10% likelihood time, that that was a 10% likelihood given that same prompt. And if we did it given that same prompt. And if we did it given that same prompt. And if we did it nine 10 more times or nine more times, nine 10 more times or nine more times, nine 10 more times or nine more times, it fail some way at every one of those it fail some way at every one of those it fail some way at every one of those or not do everything that it did in that or not do everything that it did in that or not do everything that it did in that one. You can't know. And that's actually one. You can't know. And that's actually one. You can't know. And that's actually one of the you know aspects of this is one of the you know aspects of this is one of the you know aspects of this is um and what some people say when you um and what some people say when you um and what some people say when you vibe code is ask the model multiple vibe code is ask the model multiple vibe code is ask the model multiple times to do it and then you know get a times to do it and then you know get a times to do it and then you know get a comparison. Pick the one that vibes the comparison. Pick the one that vibes the comparison. Pick the one that vibes the most and then uh token limits is also a most and then uh token limits is also a most and then uh token limits is also a consideration right for what we might consideration right for what we might consideration right for what we might have hit a and we might have hit a have hit a and we might have hit a have hit a and we might have hit a GitHub or cloud model token limit and GitHub or cloud model token limit and GitHub or cloud model token limit and actually this is you know leads us into actually this is you know leads us into actually this is you know leads us into what is vibe coding good for and what what is vibe coding good for and what what is vibe coding good for and what it's not. M kind of it's not. M kind of it's not. M kind of discussion because vibe coding when discussion because vibe coding when discussion because vibe coding when you're giving it a green field like you you're giving it a green field like you you're giving it a green field like you said, hey, let's just ask it without said, hey, let's just ask it without said, hey, let's just ask it without giving it the file. But a lot of um

giving it the file. But a lot of um giving it the file. But a lot of um coding for real projects isn't start coding for real projects isn't start coding for real projects isn't start from scratch. It's take this existing from scratch. It's take this existing from scratch. It's take this existing thing and update it or or be inspired by thing and update it or or be inspired by thing and update it or or be inspired by it like we did. Give it the file. But it like we did. Give it the file. But it like we did. Give it the file. But what if the project was more complicated what if the project was more complicated what if the project was more complicated and we were working with a some shader and we were working with a some shader and we were working with a some shader that actually had um and we wanted to that actually had um and we wanted to that actually had um and we wanted to tweak it and it had 10 files each one of tweak it and it had 10 files each one of tweak it and it had 10 files each one of you know a thousand lines of code you know a thousand lines of code you know a thousand lines of code because it was really complicated. Mhm. because it was really complicated. Mhm. because it was really complicated. Mhm. And we wanted to tweak parts of it and And we wanted to tweak parts of it and And we wanted to tweak parts of it and so giving it the whole thing and saying so giving it the whole thing and saying so giving it the whole thing and saying go tweak these parts of it. go tweak these parts of it. go tweak these parts of it. A you probably won't be able to fit them A you probably won't be able to fit them A you probably won't be able to fit them all in the context. all in the context. all in the context. So what about this with this? We we see So what about this with this? We we see So what about this with this? We we see Meta particularly it really loves their Meta particularly it really loves their Meta particularly it really loves their big contexts. Uh and they're they're big contexts. Uh and they're they're big contexts. Uh and they're they're saying yeah a million contexts go for saying yeah a million contexts go for saying yeah a million contexts go for you know we can go this far. We can go you know we can go this far. We can go you know we can go this far. We can go this big for context. Is that is that a this big for context. Is that is that a this big for context. Is that is that a solution? Is just like unlimited context solution? Is just like unlimited context solution? Is just like unlimited context windows a solution? You just feed your windows a solution? You just feed your windows a solution? You just feed your entire uh you know brownfield enterprise entire uh you know brownfield enterprise entire uh you know brownfield enterprise app into a giant context window. Uh I app into a giant context window. Uh I app into a giant context window. Uh I don't think that's the answer either don't think that's the answer either don't think that's the answer either because these models have limitations because these models have limitations because these models have limitations with large context even and I we talked with large context even and I we talked with large context even and I we talked about this in our responsible AI talk at about this in our responsible AI talk at about this in our responsible AI talk at Ignite. Um this needle in a hay stack Ignite. Um this needle in a hay stack Ignite. Um this needle in a hay stack which is can the model really reason which is can the model really reason which is can the model really reason over these very large contexts. Mhm. And over these very large contexts. Mhm. And over these very large contexts. Mhm. And by reason over in the context of code it by reason over in the context of code it by reason over in the context of code it would be can I go and look and would be can I go and look and would be can I go and look and understand these lines are the ones that understand these lines are the ones that understand these lines are the ones that are relevant to what I'm being asked to are relevant to what I'm being asked to are relevant to what I'm being asked to do versus these lines over here or and

do versus these lines over here or and do versus these lines over here or and these especially over working over a these especially over working over a these especially over working over a large project these functions are large project these functions are large project these functions are related to each other and indirect ways related to each other and indirect ways related to each other and indirect ways but to modify this top level function I but to modify this top level function I but to modify this top level function I need to go four levels deep down this need to go four levels deep down this need to go four levels deep down this path and modify this thing and return a path and modify this thing and return a path and modify this thing and return a parameter all the way up. That kind of parameter all the way up. That kind of parameter all the way up. That kind of stuff requires really understanding the stuff requires really understanding the stuff requires really understanding the full context. when we were talking about full context. when we were talking about full context. when we were talking about reasoning uh and things you need to be reasoning uh and things you need to be reasoning uh and things you need to be aware of when you're using these models aware of when you're using these models aware of when you're using these models for reasoning, we talked about this for reasoning, we talked about this for reasoning, we talked about this Eureka benchmark that Microsoft has come Eureka benchmark that Microsoft has come Eureka benchmark that Microsoft has come up with which are I stopped doing email up with which are I stopped doing email up with which are I stopped doing email which um stress uh the capabilities of which um stress uh the capabilities of which um stress uh the capabilities of frontier models. the benchmarks are frontier models. the benchmarks are frontier models. the benchmarks are generally designed to not be fully generally designed to not be fully generally designed to not be fully achievable or where the complexity can achievable or where the complexity can achievable or where the complexity can be increased to test where the limits of be increased to test where the limits of be increased to test where the limits of the models are. And one of them is a the models are. And one of them is a the models are. And one of them is a needle in a haststack type test where needle in a haststack type test where needle in a haststack type test where you give the model a big chunk of text you give the model a big chunk of text you give the model a big chunk of text and there's two pieces of information and there's two pieces of information and there's two pieces of information that are related to each other and that are related to each other and that are related to each other and they're far apart from each other. they're far apart from each other. they're far apart from each other. That's the trick. You can place them That's the trick. You can place them That's the trick. You can place them close together like Mary was sitting in close together like Mary was sitting in close together like Mary was sitting in her living room and her living room had her living room and her living room had her living room and her living room had a marble floor. Mary's living room had a a marble floor. Mary's living room had a a marble floor. Mary's living room had a marble floor and then and then so those marble floor and then and then so those marble floor and then and then so those are the two pieces of information and if are the two pieces of information and if are the two pieces of information and if you put them in a large context of other you put them in a large context of other you put them in a large context of other text and in the middle of it you put text and in the middle of it you put text and in the middle of it you put those two together and say what was the those two together and say what was the those two together and say what was the floor Mary standing on made of floor Mary standing on made of floor Mary standing on made of models even you know uh even the models even you know uh even the models even you know uh even the simplest ones that can handle the simplest ones that can handle the simplest ones that can handle the context will say uh marble floor Mary's context will say uh marble floor Mary's context will say uh marble floor Mary's was standing on a marble floor because was standing on a marble floor because was standing on a marble floor because they're close to each other but separate they're close to each other but separate they're close to each other but separate them and put one at the beginning and them and put one at the beginning and them and put one at the beginning and one at the end of a very large context

one at the end of a very large context one at the end of a very large context that was my question and it's got to that was my question and it's got to that was my question and it's got to connect connect connect and then say what was the you know floor and then say what was the you know floor and then say what was the you know floor that Mary was standing on and even these that Mary was standing on and even these that Mary was standing on and even these frontier models will often times fail at frontier models will often times fail at frontier models will often times fail at context window sizes that are much much context window sizes that are much much context window sizes that are much much smaller than the million tokens or smaller than the million tokens or smaller than the million tokens or 128,000 tokens in fact just at 32k 128,000 tokens in fact just at 32k 128,000 tokens in fact just at 32k tokens or even 8k tokens they start they tokens or even 8k tokens they start they tokens or even 8k tokens they start they can fail at that test and so if they can can fail at that test and so if they can can fail at that test and so if they can fail at that test in language that fail at that test in language that fail at that test in language that they'll fail at that test in coding as they'll fail at that test in coding as they'll fail at that test in coding as well right Right. Right. well right Right. Right. well right Right. Right. when did I remember that it was there when did I remember that it was there when did I remember that it was there was a lot about about the the distance was a lot about about the the distance was a lot about about the the distance between the tokens but then there's also between the tokens but then there's also between the tokens but then there's also is there not a recency bias towards is there not a recency bias towards is there not a recency bias towards things that happened earlier in the things that happened earlier in the things that happened earlier in the later rather in the context window and later rather in the context window and later rather in the context window and well it's actually there was a paper well it's actually there was a paper well it's actually there was a paper that came out I think last year called that came out I think last year called that came out I think last year called lost in the middle and it was lost in the middle and it was lost in the middle and it was specifically talking about these long specifically talking about these long specifically talking about these long contexts the needle in the hack test and contexts the needle in the hack test and contexts the needle in the hack test and what they found is that information in what they found is that information in what they found is that information in the middle of the context models have a the middle of the context models have a the middle of the context models have a hard time you know tend to ignore they hard time you know tend to ignore they hard time you know tend to ignore they don't they don't don't they don't don't they don't explicitly look look at it like they do explicitly look look at it like they do explicitly look look at it like they do information at the beginning of the information at the beginning of the information at the beginning of the context or the end of the context. Yeah.

context or the end of the context. Yeah. context or the end of the context. Yeah. Interesting. Okay. Um, one of the things Interesting. Okay. Um, one of the things Interesting. Okay. Um, one of the things I really liked about the way that we I really liked about the way that we I really liked about the way that we were doing it also, and I don't I think were doing it also, and I don't I think were doing it also, and I don't I think it's a feature of GitHub Copilot, was it's a feature of GitHub Copilot, was it's a feature of GitHub Copilot, was that it was starting to call out pieces that it was starting to call out pieces that it was starting to call out pieces of text that it found that it attributed of text that it found that it attributed of text that it found that it attributed to other humans and it said this was on to other humans and it said this was on to other humans and it said this was on Stack Overflow in this question and that Stack Overflow in this question and that Stack Overflow in this question and that was on this chunk of stuff over at um uh was on this chunk of stuff over at um uh was on this chunk of stuff over at um uh at GitHub and then showed me the at GitHub and then showed me the at GitHub and then showed me the license. Um, and we were fortunate in license. Um, and we were fortunate in license. Um, and we were fortunate in our vibe coding that we found licenses our vibe coding that we found licenses our vibe coding that we found licenses that were compatible with our goal. Um, that were compatible with our goal. Um, that were compatible with our goal. Um, early criticisms of co-pilot had early criticisms of co-pilot had early criticisms of co-pilot had indicated that it's out there just kind indicated that it's out there just kind indicated that it's out there just kind of in the world randing randomly of in the world randing randomly of in the world randing randomly grabbing stuff and anything it could see grabbing stuff and anything it could see grabbing stuff and anything it could see is appropriately well actually we is appropriately well actually we is appropriately well actually we debunked that right away like it's only debunked that right away like it's only debunked that right away like it's only public repos with appropriate licenses. public repos with appropriate licenses. public repos with appropriate licenses. Yeah, it's only public repos but the Yeah, it's only public repos but the Yeah, it's only public repos but the license still matters. Yeah. And it's uh license still matters. Yeah. And it's uh license still matters. Yeah. And it's uh licenses that are that permit copying licenses that are that permit copying licenses that are that permit copying like this. Exactly. So, we found code like this. Exactly. So, we found code like this. Exactly. So, we found code that allowed us achieve our goal from a that allowed us achieve our goal from a that allowed us achieve our goal from a couple of different folks and we were couple of different folks and we were couple of different folks and we were able to ask Claude to generate a license able to ask Claude to generate a license able to ask Claude to generate a license uh me a readme file that attributed uh me a readme file that attributed uh me a readme file that attributed those people said hey good job and uh those people said hey good job and uh those people said hey good job and uh and then linked to the place where it and then linked to the place where it and then linked to the place where it found the code and that made me feel found the code and that made me feel found the code and that made me feel better about yeah but you know I mean better about yeah but you know I mean better about yeah but you know I mean just from a hey people are getting just from a hey people are getting just from a hey people are getting credit where it's due and maybe you can credit where it's due and maybe you can credit where it's due and maybe you can also go look and see what you know also go look and see what you know also go look and see what you know what's the context of this code if what's the context of this code if what's the context of this code if you're digging deeper I want to mention you're digging deeper I want to mention you're digging deeper I want to mention too in the pro in addition to debunking too in the pro in addition to debunking too in the pro in addition to debunking the myth that Microsoft was just going the myth that Microsoft was just going the myth that Microsoft was just going and or GitHub was going and just and or GitHub was going and just and or GitHub was going and just grabbing any old code. Yeah. Uh grabbing any old code. Yeah. Uh grabbing any old code. Yeah. Uh regardless of license or whether it's

regardless of license or whether it's regardless of license or whether it's public or private, we also Microsoft public or private, we also Microsoft public or private, we also Microsoft will indemnify anybody and if they get will indemnify anybody and if they get will indemnify anybody and if they get involved in a copyright dispute over involved in a copyright dispute over involved in a copyright dispute over that content that was generated by that content that was generated by that content that was generated by GitHub Copilot. Yeah. Yeah. Yeah. Yeah. GitHub Copilot. Yeah. Yeah. Yeah. Yeah. GitHub Copilot. Yeah. Yeah. Yeah. Yeah. I've had a couple of times GitHub I've had a couple of times GitHub I've had a couple of times GitHub copilot start writing start telling me copilot start writing start telling me copilot start writing start telling me that like it's writing and then it goes that like it's writing and then it goes that like it's writing and then it goes uh nope this is too much of this uh uh nope this is too much of this uh uh nope this is too much of this uh looks like like it doesn't want to copy looks like like it doesn't want to copy looks like like it doesn't want to copy paste. Yeah, right. It'll synthesize. So paste. Yeah, right. It'll synthesize. So paste. Yeah, right. It'll synthesize. So let me ask you this. We said how could let me ask you this. We said how could let me ask you this. We said how could we have done this before? or we would we have done this before? or we would we have done this before? or we would have probably I would have looked up the have probably I would have looked up the have probably I would have looked up the the math and looked up how to learn the the math and looked up how to learn the the math and looked up how to learn the HLSL. Somewhere in the middle between HLSL. Somewhere in the middle between HLSL. Somewhere in the middle between learning it completely and I don't know learning it completely and I don't know learning it completely and I don't know if I want to burn a Saturday learning if I want to burn a Saturday learning if I want to burn a Saturday learning HLSL and vibe coding which achieved in HLSL and vibe coding which achieved in HLSL and vibe coding which achieved in 15 minutes is a middle ground somewhere 15 minutes is a middle ground somewhere 15 minutes is a middle ground somewhere which is Franken coding which is which is Franken coding which is which is Franken coding which is googling around opening up 42 tabs. We googling around opening up 42 tabs. We googling around opening up 42 tabs. We might have been able to do it in two might have been able to do it in two might have been able to do it in two hours three hours instead of four to six hours three hours instead of four to six hours three hours instead of four to six or less than 15 minutes. Paste it all or less than 15 minutes. Paste it all or less than 15 minutes. Paste it all together. we still wouldn't understand together. we still wouldn't understand together. we still wouldn't understand fully how it worked and then when we fully how it worked and then when we fully how it worked and then when we were done we would close the browser were done we would close the browser were done we would close the browser triumphantly throwing the 42 tabs into triumphantly throwing the 42 tabs into triumphantly throwing the 42 tabs into the abyss and also would have achieved the abyss and also would have achieved the abyss and also would have achieved the same goal. How often do you do that the same goal. How often do you do that the same goal. How often do you do that kind of coding?

kind of coding? kind of coding? A lot. And actually that's you know A lot. And actually that's you know A lot. And actually that's you know that's Stack Overflow coding, right? that's Stack Overflow coding, right? that's Stack Overflow coding, right? Yeah. Yeah. Because that's how many Yeah. Yeah. Because that's how many Yeah. Yeah. Because that's how many stories are there of people just going stories are there of people just going stories are there of people just going and even at companies like Microsoft and even at companies like Microsoft and even at companies like Microsoft it's just like oh I found this on Stack it's just like oh I found this on Stack it's just like oh I found this on Stack Overflow. Slap. Exactly. just directly Overflow. Slap. Exactly. just directly Overflow. Slap. Exactly. just directly directly into production. Yeah. What is directly into production. Yeah. What is directly into production. Yeah. What is that? So that's not vibe coding. What is that? So that's not vibe coding. What is that? So that's not vibe coding. What is that? I called it Franken coding. that? I called it Franken coding. that? I called it Franken coding. There's got to be a better That's not There's got to be a better That's not There's got to be a better That's not You don't like that? No. On you, man. You don't like that? No. On you, man. You don't like that? No. On you, man. All right. All right. I'll come up with All right. All right. I'll come up with All right. All right. I'll come up with Actually, I'll have AI come up with a Actually, I'll have AI come up with a Actually, I'll have AI come up with a better name. Are you going to be the one better name. Are you going to be the one better name. Are you going to be the one that invents that? Yeah. I mean, maybe that invents that? Yeah. I mean, maybe that invents that? Yeah. I mean, maybe it's just called Stack Overflow Coding. it's just called Stack Overflow Coding. it's just called Stack Overflow Coding. Everybody knows what that means. Yeah, Everybody knows what that means. Yeah, Everybody knows what that means. Yeah, that's that's Stack Overflow coding is that's that's Stack Overflow coding is that's that's Stack Overflow coding is probably it. Yeah. Yeah. It's not as probably it. Yeah. Yeah. It's not as probably it. Yeah. Yeah. It's not as cool as vibe coding, but yes. Okay. So cool as vibe coding, but yes. Okay. So cool as vibe coding, but yes. Okay. So then vibe coding has a place where when then vibe coding has a place where when then vibe coding has a place where when you want to get things done, when you you want to get things done, when you you want to get things done, when you want to prototype quickly, I don't think want to prototype quickly, I don't think want to prototype quickly, I don't think I would want the FAA vibe coding. I I would want the FAA vibe coding. I I would want the FAA vibe coding. I don't think I want enterprises vibe don't think I want enterprises vibe don't think I want enterprises vibe coding. No. In fact, enterprises coding. No. In fact, enterprises coding. No. In fact, enterprises probably don't want their coders vibe probably don't want their coders vibe probably don't want their coders vibe coding either. As much as the coding either. As much as the coding either. As much as the productivity benefits of it, one of the productivity benefits of it, one of the productivity benefits of it, one of the like we and like I said, we talked about like we and like I said, we talked about like we and like I said, we talked about some of these things um in our last one some of these things um in our last one some of these things um in our last one of our podcast a couple of episodes ago.

of our podcast a couple of episodes ago. of our podcast a couple of episodes ago. Yep. Systems thinking one. Yeah. But Yep. Systems thinking one. Yeah. But Yep. Systems thinking one. Yeah. But let's just start with some of the basics let's just start with some of the basics let's just start with some of the basics here. What we did was vibe code a v1. here. What we did was vibe code a v1. here. What we did was vibe code a v1. That's a good point and it was a simple That's a good point and it was a simple That's a good point and it was a simple relatively simple project. A so you know relatively simple project. A so you know relatively simple project. A so you know getting past the hey the complexity getting past the hey the complexity getting past the hey the complexity limits that of current limits that of current limits that of current AI we created a v1 and for real projects AI we created a v1 and for real projects AI we created a v1 and for real projects like enterprise projects it's not just a like enterprise projects it's not just a like enterprise projects it's not just a v1 it's a it's a v1 that needs to be v1 it's a it's a v1 that needs to be v1 it's a it's a v1 that needs to be iterated into v1.1 v 1.2 2.0 know as iterated into v1.1 v 1.2 2.0 know as iterated into v1.1 v 1.2 2.0 know as requirements change and and maybe um the requirements change and and maybe um the requirements change and and maybe um the specification gets clearer, the specification gets clearer, the specification gets clearer, the requirements get clearer as you start to requirements get clearer as you start to requirements get clearer as you start to use the V1 or the V actually in use the V1 or the V actually in use the V1 or the V actually in enterprise this would have been a V dot enterprise this would have been a V dot enterprise this would have been a V dot whatever two or whatever. Mhm. And then whatever two or whatever. Mhm. And then whatever two or whatever. Mhm. And then at that point you go to the AI and say at that point you go to the AI and say at that point you go to the AI and say well actually the fractals need to be well actually the fractals need to be well actually the fractals need to be this and you need to mod it move it like this and you need to mod it move it like this and you need to mod it move it like this rate and this speed and this is how this rate and this speed and this is how this rate and this speed and this is how they need to zoom in and out and when they need to zoom in and out and when they need to zoom in and out and when they and the user says something stop they and the user says something stop they and the user says something stop the stop it freeze it you know things the stop it freeze it you know things the stop it freeze it you know things like that I'm just making up like that I'm just making up like that I'm just making up enhancements here but the models then enhancements here but the models then enhancements here but the models then have been shown to struggle with have been shown to struggle with have been shown to struggle with brownfield with taking existing codebase brownfield with taking existing codebase brownfield with taking existing codebase and knowing how to modify them. In my and knowing how to modify them. In my and knowing how to modify them. In my own personal experience, again ignoring own personal experience, again ignoring own personal experience, again ignoring the scale problems we talked about, just the scale problems we talked about, just the scale problems we talked about, just it goes or goes in the context is I've it goes or goes in the context is I've it goes or goes in the context is I've had it and I use vibe, you know, as much had it and I use vibe, you know, as much had it and I use vibe, you know, as much AI assisted coding, not purely vibe AI assisted coding, not purely vibe AI assisted coding, not purely vibe coding, but AI assisted coding as coding, but AI assisted coding as coding, but AI assisted coding as possible. And if you give it a big chunk possible. And if you give it a big chunk possible. And if you give it a big chunk of code and say, I need this, I need to

of code and say, I need this, I need to of code and say, I need this, I need to change the code so that it does these change the code so that it does these change the code so that it does these things which mean modifications or things which mean modifications or things which mean modifications or enhancements. Many times it just loses enhancements. Many times it just loses enhancements. Many times it just loses track of state and you know part of it track of state and you know part of it track of state and you know part of it is updated, part of it's not. it forgets is updated, part of it's not. it forgets is updated, part of it's not. it forgets to update a function. Um, or it actually to update a function. Um, or it actually to update a function. Um, or it actually refactors something and leaves out refactors something and leaves out refactors something and leaves out functionality that's unrelated but still functionality that's unrelated but still functionality that's unrelated but still important. And so I end up with a important. And so I end up with a important. And so I end up with a function that's missing things that were function that's missing things that were function that's missing things that were actually required. Yeah. And you end up actually required. Yeah. And you end up actually required. Yeah. And you end up in this loop with the model like wait a in this loop with the model like wait a in this loop with the model like wait a minute it's broken now. Oh, I know what minute it's broken now. Oh, I know what minute it's broken now. Oh, I know what happened. You know, I I think everybody happened. You know, I I think everybody happened. You know, I I think everybody that does AI coding is like the oh I that does AI coding is like the oh I that does AI coding is like the oh I see. I'm sorry. Yeah. And then here here see. I'm sorry. Yeah. And then here here see. I'm sorry. Yeah. And then here here is the fix. And you're like, "Oh, okay. is the fix. And you're like, "Oh, okay. is the fix. And you're like, "Oh, okay. Awesome. It fixed it." And you drop that Awesome. It fixed it." And you drop that Awesome. It fixed it." And you drop that in, run it, and it's not fixed. Yeah. in, run it, and it's not fixed. Yeah. in, run it, and it's not fixed. Yeah. It's literally not like it says that it It's literally not like it says that it It's literally not like it says that it fixed the thing that you pointed out, fixed the thing that you pointed out, fixed the thing that you pointed out, and it literally did not. And then and it literally did not. And then and it literally did not. And then you're like, "Wait a minute. You didn't you're like, "Wait a minute. You didn't you're like, "Wait a minute. You didn't fix it." And it's like, "Oh, I see what fix it." And it's like, "Oh, I see what fix it." And it's like, "Oh, I see what happened." Yeah, you're right. It's so happened." Yeah, you're right. It's so happened." Yeah, you're right. It's so funny. I wish I understood what it was funny. I wish I understood what it was funny. I wish I understood what it was about the model that made it go, "Oh, about the model that made it go, "Oh, about the model that made it go, "Oh, right."

right." right." And then, you're right. I should have And then, you're right. I should have And then, you're right. I should have seen that. I'm so silly. Yeah. Anyway, seen that. I'm so silly. Yeah. Anyway, seen that. I'm so silly. Yeah. Anyway, that's then then you end up in this that's then then you end up in this that's then then you end up in this frustrating loop. But then any like the frustrating loop. But then any like the frustrating loop. But then any like the point here that I'm trying to make is point here that I'm trying to make is point here that I'm trying to make is that you get down this vibe cutting that you get down this vibe cutting that you get down this vibe cutting route route like we end up in a route route like we end up in a route route like we end up in a situation where actually the model just situation where actually the model just situation where actually the model just can't figure out how to do it. As much can't figure out how to do it. As much can't figure out how to do it. As much as you prompt it, as much as you give as you prompt it, as much as you give as you prompt it, as much as you give it, hey, didn't look at it. No, it's not it, hey, didn't look at it. No, it's not it, hey, didn't look at it. No, it's not fixed. Here's what's going on. And it's fixed. Here's what's going on. And it's fixed. Here's what's going on. And it's like, oh, I get it. And it tries and it like, oh, I get it. And it tries and it like, oh, I get it. And it tries and it thinks it's fixing it and it's not. thinks it's fixing it and it's not. thinks it's fixing it and it's not. Yeah. Is at that point, what do you do? Yeah. Is at that point, what do you do? Yeah. Is at that point, what do you do? like this isn't a hobby thing at that like this isn't a hobby thing at that like this isn't a hobby thing at that you know where in this scenario it's not you know where in this scenario it's not you know where in this scenario it's not like oh you know what I'll do what's like oh you know what I'll do what's like oh you know what I'll do what's funny is that you've used the thing to funny is that you've used the thing to funny is that you've used the thing to generate green field stuff and now it generate green field stuff and now it generate green field stuff and now it instantly becomes brownfield and you've instantly becomes brownfield and you've instantly becomes brownfield and you've inherited a crappy code base by a inherited a crappy code base by a inherited a crappy code base by a mediocre programmer that's right exactly mediocre programmer that's right exactly mediocre programmer that's right exactly you yeah and now you're responsible for you yeah and now you're responsible for you yeah and now you're responsible for maintaining upgrading it and nobody maintaining upgrading it and nobody maintaining upgrading it and nobody gives a crap that you you used AI and gives a crap that you you used AI and gives a crap that you you used AI and now don't understand it like your job is now don't understand it like your job is now don't understand it like your job is to go and deliver feature X and like you to go and deliver feature X and like you to go and deliver feature X and like you better figure out how to do it. And at better figure out how to do it. And at better figure out how to do it. And at some point that might require you to some point that might require you to some point that might require you to actually go spend that four to six actually go spend that four to six actually go spend that four to six hours. Yeah. To learn whatever hours. Yeah. To learn whatever hours. Yeah. To learn whatever hls, whatever, hls. There was a uh there hls, whatever, hls. There was a uh there hls, whatever, hls. There was a uh there was a statistic that I thought was not a was a statistic that I thought was not a was a statistic that I thought was not a good statistic that Google came out with good statistic that Google came out with good statistic that Google came out with recently that was something like 25% of recently that was something like 25% of recently that was something like 25% of our code is AI generated. And I'm like, our code is AI generated. And I'm like, our code is AI generated. And I'm like, okay, well, first like lines of code is okay, well, first like lines of code is okay, well, first like lines of code is a lie. Like lines of code is just a a lie. Like lines of code is just a a lie. Like lines of code is just a number. But also, does that mean number. But also, does that mean number. But also, does that mean committed and into production code or

committed and into production code or committed and into production code or does that just mean code that's up for does that just mean code that's up for does that just mean code that's up for review? Like there's so much to be said review? Like there's so much to be said review? Like there's so much to be said in that. Well, there's also like take a in that. Well, there's also like take a in that. Well, there's also like take a look at auto like the auto, you know, look at auto like the auto, you know, look at auto like the auto, you know, co-pilot autocompletes. Yeah. Do you co-pilot autocompletes. Yeah. Do you co-pilot autocompletes. Yeah. Do you count those as code I wrote because I count those as code I wrote because I count those as code I wrote because I didn't Yeah, but but that's probably didn't Yeah, but but that's probably didn't Yeah, but but that's probably that's one thing that you can do is say, that's one thing that you can do is say, that's one thing that you can do is say, oh, the number of characters that were oh, the number of characters that were oh, the number of characters that were actually spit out. They're counting how actually spit out. They're counting how actually spit out. They're counting how much code is getting generated versus much code is getting generated versus much code is getting generated versus how much is being accepted. Yeah. Or how how much is being accepted. Yeah. Or how how much is being accepted. Yeah. Or how much I mean it can even be how much is much I mean it can even be how much is much I mean it can even be how much is accepted but the autocompletes are like accepted but the autocompletes are like accepted but the autocompletes are like you know oh it completed this for loop you know oh it completed this for loop you know oh it completed this for loop that's obvious. Right. Right. Right. I that's obvious. Right. Right. Right. I that's obvious. Right. Right. Right. I mean that that's assisted code that's mean that that's assisted code that's mean that that's assisted code that's not purely like hey it wrote the whole not purely like hey it wrote the whole not purely like hey it wrote the whole thing. Right. Right. Right. When you say thing. Right. Right. Right. When you say thing. Right. Right. Right. When you say it's generated by AI you know people it's generated by AI you know people it's generated by AI you know people imagine oh I just told it create this imagine oh I just told it create this imagine oh I just told it create this cool program. Yeah. And with all these cool program. Yeah. And with all these cool program. Yeah. And with all these bells and whistles and the AI generated bells and whistles and the AI generated bells and whistles and the AI generated 25% of it. No, it always gets back. No, 25% of it. No, it always gets back. No, 25% of it. No, it always gets back. No, it it did it incrementally character by it it did it incrementally character by it it did it incrementally character by character. I I I I do wonder how far are character. I I I I do wonder how far are character. I I I I do wonder how far are we how far away we are to uh that scene we how far away we are to uh that scene we how far away we are to uh that scene that I love so much in Star Trek for that I love so much in Star Trek for that I love so much in Star Trek for when Scotty was like something else when Scotty was like something else when Scotty was like something else computer, you know what I mean? Like computer, you know what I mean? Like computer, you know what I mean? Like Yeah. Well, this again, we talked about Yeah. Well, this again, we talked about Yeah. Well, this again, we talked about this one. I don't think we're anywhere this one. I don't think we're anywhere this one. I don't think we're anywhere close to that because of the context close to that because of the context close to that because of the context limits. Yeah. And by the way, when it limits. Yeah. And by the way, when it limits. Yeah. And by the way, when it comes to comes to comes to um real enterprise code, you know this um real enterprise code, you know this um real enterprise code, you know this vibe coding thing for that vibe coding thing for that vibe coding thing for that HSLs practical thing.

HSLs practical thing. HSLs practical thing. Ranch H. You'll never get high level H Ranch H. You'll never get high level H Ranch H. You'll never get high level H highle shader language. High level highle shader language. High level highle shader language. High level shader language. Yeah. Um say it five shader language. Yeah. Um say it five shader language. Yeah. Um say it five times fast. Exactly. Uh for that we times fast. Exactly. Uh for that we times fast. Exactly. Uh for that we didn't care if it actually and we didn't care if it actually and we didn't care if it actually and we wouldn't have cared. It didn't, but we wouldn't have cared. It didn't, but we wouldn't have cared. It didn't, but we wouldn't have cared if it imported some wouldn't have cared if it imported some wouldn't have cared if it imported some packages and said, "I'm going to import packages and said, "I'm going to import packages and said, "I'm going to import these packages to help me do the these packages to help me do the these packages to help me do the fractals." Yeah. We'd have been like, fractals." Yeah. We'd have been like, fractals." Yeah. We'd have been like, "Cool, import "Cool, import "Cool, import them." But if you're in an enterprise, them." But if you're in an enterprise, them." But if you're in an enterprise, you do care about what packages it's you do care about what packages it's you do care about what packages it's importing. Yep. Because those packages importing. Yep. Because those packages importing. Yep. Because those packages could have vulnerabilities in them. could have vulnerabilities in them. could have vulnerabilities in them. 100%. Or they could be or they could be 100%. Or they could be or they could be 100%. Or they could be or they could be a package that is like it's the a package that is like it's the a package that is like it's the monstrous, you know, I do fractals and I monstrous, you know, I do fractals and I monstrous, you know, I do fractals and I do 50 billion other things package. Yep. do 50 billion other things package. Yep. do 50 billion other things package. Yep. And you really just don't want to carry And you really just don't want to carry And you really just don't want to carry all that baggage with you. you want to all that baggage with you. you want to all that baggage with you. you want to just carry the, you know, there's just carry the, you know, there's just carry the, you know, there's another package that just does fractals another package that just does fractals another package that just does fractals and does them. Awesome. Yeah. But so the and does them. Awesome. Yeah. But so the and does them. Awesome. Yeah. But so the AI is doing things like making decisions AI is doing things like making decisions AI is doing things like making decisions like that that you're now accountable like that that you're now accountable like that that you're now accountable for. for. for. Um and then the other one um and I've Um and then the other one um and I've Um and then the other one um and I've experienced this one with even with chat experienced this one with even with chat experienced this one with even with chat GPT and OpenAI which relates to cuto GPT and OpenAI which relates to cuto GPT and OpenAI which relates to cuto offs and how familiar the LA model is offs and how familiar the LA model is offs and how familiar the LA model is with packages and APIs versions with packages and APIs versions with packages and APIs versions is you say hey go write um use this API is you say hey go write um use this API is you say hey go write um use this API you know the OpenAI chat API to call a you know the OpenAI chat API to call a you know the OpenAI chat API to call a model and it ends up using a version of model and it ends up using a version of model and it ends up using a version of the OpenAI API that's last years and has the OpenAI API that's last years and has the OpenAI API that's last years and has been deprecated. M been deprecated. M been deprecated. M and um it's because the model A either

and um it's because the model A either and um it's because the model A either doesn't know about the new API because doesn't know about the new API because doesn't know about the new API because it's prior it was trained prior to the it's prior it was trained prior to the it's prior it was trained prior to the new API showing up or it's been trained new API showing up or it's been trained new API showing up or it's been trained there's so many examples of the V1 API there's so many examples of the V1 API there's so many examples of the V1 API that it's like oh I'm going to use V1 that it's like oh I'm going to use V1 that it's like oh I'm going to use V1 because I know V1 really well and it because I know V1 really well and it because I know V1 really well and it should be using V2 because that's the should be using V2 because that's the should be using V2 because that's the new API that has security features and new API that has security features and new API that has security features and you know but it doesn't know it that you know but it doesn't know it that you know but it doesn't know it that well either or doesn't know it at all well either or doesn't know it at all well either or doesn't know it at all and so that's context so You talk about and so that's context so You talk about and so that's context so You talk about context windows, but we don't talk about context windows, but we don't talk about context windows, but we don't talk about the context of working for an enterprise the context of working for an enterprise the context of working for an enterprise for 20 years and knowing like where all for 20 years and knowing like where all for 20 years and knowing like where all the bodies are buried and why this API the bodies are buried and why this API the bodies are buried and why this API did this. Like it can only see the text did this. Like it can only see the text did this. Like it can only see the text that you feed it and not the uh prior that you feed it and not the uh prior that you feed it and not the uh prior art and the you know the campfire art and the you know the campfire art and the you know the campfire stories that are told in large stories that are told in large stories that are told in large enterprises. So anyway, I I I uh believe enterprises. So anyway, I I I uh believe enterprises. So anyway, I I I uh believe given the architectures of these models, given the architectures of these models, given the architectures of these models, given the context windows and the given the context windows and the given the context windows and the limitations on context windows which are limitations on context windows which are limitations on context windows which are inherent in transform models, unless inherent in transform models, unless inherent in transform models, unless something dramatically changes and even something dramatically changes and even something dramatically changes and even then you've got to worry about how do I then you've got to worry about how do I then you've got to worry about how do I make sure the model knows what are the make sure the model knows what are the make sure the model knows what are the current versions of the APIs, the current versions of the APIs, the current versions of the APIs, the correct versions of the APIs, which correct versions of the APIs, which correct versions of the APIs, which packages I'm allowed to import versus packages I'm allowed to import versus packages I'm allowed to import versus not, you know, my company's approved not, you know, my company's approved not, you know, my company's approved versus right, you know, I'm inher I'm versus right, you know, I'm inher I'm versus right, you know, I'm inher I'm using a no open source thing that now using a no open source thing that now using a no open source thing that now I'm accountable for. All of those things I'm accountable for. All of those things I'm accountable for. All of those things to me mean that AI is not going to to me mean that AI is not going to to me mean that AI is not going to completely take over and you need people completely take over and you need people completely take over and you need people that know what they're doing to own and that know what they're doing to own and that know what they're doing to own and maintain the code bases. Um so expert maintain the code bases. Um so expert maintain the code bases. Um so expert programming is maybe there's less of it.

programming is maybe there's less of it. programming is maybe there's less of it. Um because you know the the line of Um because you know the the line of Um because you know the the line of business web frontend app is business web frontend app is business web frontend app is you know can can be more guard rail but you know can can be more guard rail but you know can can be more guard rail but for more complex projects you're going for more complex projects you're going for more complex projects you're going to need big picture. You got to see the to need big picture. You got to see the to need big picture. You got to see the big picture. But I think also agents are big picture. But I think also agents are big picture. But I think also agents are going to make things easier. If I had, going to make things easier. If I had, going to make things easier. If I had, for example, an agent where the success for example, an agent where the success for example, an agent where the success metric of that fractal thing could be an metric of that fractal thing could be an metric of that fractal thing could be an agent looking at it and I'm and I could agent looking at it and I'm and I could agent looking at it and I'm and I could let it iterate and I could go to lunch. let it iterate and I could go to lunch. let it iterate and I could go to lunch. Yeah, I mean that'll help too. That'll Yeah, I mean that'll help too. That'll Yeah, I mean that'll help too. That'll help a lot. There's one other thing too help a lot. There's one other thing too help a lot. There's one other thing too I didn't mention is spec specification. I didn't mention is spec specification. I didn't mention is spec specification. Like you and I have some idea in our Like you and I have some idea in our Like you and I have some idea in our head what we think we want. Like we head what we think we want. Like we head what we think we want. Like we didn't write we didn't Yeah. We just didn't write we didn't Yeah. We just didn't write we didn't Yeah. We just said, "Hey, let's see what it has." Oh, said, "Hey, let's see what it has." Oh, said, "Hey, let's see what it has." Oh, that looks good. So that's our we've that looks good. So that's our we've that looks good. So that's our we've That was our specification. Yeah. which That was our specification. Yeah. which That was our specification. Yeah. which was mostly vibes. Yeah. Um and you know was mostly vibes. Yeah. Um and you know was mostly vibes. Yeah. Um and you know that when an enterprise comes and says that when an enterprise comes and says that when an enterprise comes and says we want a app that does X Y and Z that we want a app that does X Y and Z that we want a app that does X Y and Z that you need to like fully spec the thing um you need to like fully spec the thing um you need to like fully spec the thing um or at least spec it like 80% and then or at least spec it like 80% and then or at least spec it like 80% and then you know 20% is going to emerge as you you know 20% is going to emerge as you you know 20% is going to emerge as you start to write it. Um, and so with AI, start to write it. Um, and so with AI, start to write it. Um, and so with AI, you need to that that it doesn't you need to that that it doesn't you need to that that it doesn't alleviate you of having to give it the alleviate you of having to give it the alleviate you of having to give it the spec. Um, which is work by itself. And spec. Um, which is work by itself. And spec. Um, which is work by itself. And it doesn't alleviate you from it doesn't alleviate you from it doesn't alleviate you from now having that 20% of the spec that you now having that 20% of the spec that you now having that 20% of the spec that you didn't articulate being brownfield didn't articulate being brownfield didn't articulate being brownfield requirements on top of what the AI requirements on top of what the AI requirements on top of what the AI generates as V1 or whatever. So any like generates as V1 or whatever. So any like generates as V1 or whatever. So any like there's a whole bunch of complexity I there's a whole bunch of complexity I there's a whole bunch of complexity I think people just wave their hands over.

think people just wave their hands over. think people just wave their hands over. Oh no, AI will just fix it. I just don't Oh no, AI will just fix it. I just don't Oh no, AI will just fix it. I just don't see and I need to be, you know, I need see and I need to be, you know, I need see and I need to be, you know, I need to understand what is the path there and to understand what is the path there and to understand what is the path there and I just don't see one to actually, you I just don't see one to actually, you I just don't see one to actually, you know, hey, make the enterprise app that know, hey, make the enterprise app that know, hey, make the enterprise app that does fubar and then out comes this does fubar and then out comes this does fubar and then out comes this complex app with the right packages and complex app with the right packages and complex app with the right packages and the right API versions and good the right API versions and good the right API versions and good performance and good security and and it performance and good security and and it performance and good security and and it can be evolved and maintained without can be evolved and maintained without can be evolved and maintained without humans to understand that need to humans to understand that need to humans to understand that need to understand it. It's just not going to understand it. It's just not going to understand it. It's just not going to happen. Yeah, I'm finding that the happen. Yeah, I'm finding that the happen. Yeah, I'm finding that the altitude changes so rapidly. I'm getting altitude changes so rapidly. I'm getting altitude changes so rapidly. I'm getting like the bends. It's just like going like the bends. It's just like going like the bends. It's just like going from like low level stuff to high level from like low level stuff to high level from like low level stuff to high level stuff and low level stuff to high level stuff and low level stuff to high level stuff and low level stuff to high level stuff. That's that's very very stuff. That's that's very very stuff. That's that's very very challenging. challenging. challenging. Cool. All right. Well, that said, I Cool. All right. Well, that said, I Cool. All right. Well, that said, I mean, how much of the code how what mean, how much of the code how what mean, how much of the code how what would you say is the percentage of code would you say is the percentage of code would you say is the percentage of code that you have AI generate for you? that you have AI generate for you? that you have AI generate for you? 25 30%. Yeah. Um the part that I'm 25 30%. Yeah. Um the part that I'm 25 30%. Yeah. Um the part that I'm getting better at is the refactoring. getting better at is the refactoring. getting better at is the refactoring. Like I was thinking about this HLSL. I Like I was thinking about this HLSL. I Like I was thinking about this HLSL. I wouldn't just say generically refactor wouldn't just say generically refactor wouldn't just say generically refactor this and make it better. I would say this and make it better. I would say this and make it better. I would say pull out these four variables because I pull out these four variables because I pull out these four variables because I want to make those user changeable, you want to make those user changeable, you want to make those user changeable, you know. So I wouldn't give it a four-step know. So I wouldn't give it a four-step know. So I wouldn't give it a four-step process. I would give it the four steps process. I would give it the four steps process. I would give it the four steps because I know with four steps I want it because I know with four steps I want it because I know with four steps I want it like I know I wanted to get from point A like I know I wanted to get from point A like I know I wanted to get from point A to point D through points B, C. So you to point D through points B, C. So you to point D through points B, C. So you get why would you do that? Could be that get why would you do that? Could be that get why would you do that? Could be that I'm not willing to let go of the I'm not willing to let go of the I'm not willing to let go of the steering wheel yet. Yeah, it could be steering wheel yet. Yeah, it could be steering wheel yet. Yeah, it could be that I don't trust it because I've asked

that I don't trust it because I've asked that I don't trust it because I've asked it to do stuff that was unclear before. it to do stuff that was unclear before. it to do stuff that was unclear before. Um, both myself and David Fowler give Um, both myself and David Fowler give Um, both myself and David Fowler give ridiculously long prompts. Like Fowler ridiculously long prompts. Like Fowler ridiculously long prompts. Like Fowler is well known to give like five is well known to give like five is well known to give like five paragraph fonts to get a one paragraph paragraph fonts to get a one paragraph paragraph fonts to get a one paragraph answer. And I'm like, "Dude, you could answer. And I'm like, "Dude, you could answer. And I'm like, "Dude, you could just type it." And he's like, just type it." And he's like, just type it." And he's like, "No, that that works for him." And I the "No, that that works for him." And I the "No, that that works for him." And I the same way. My my coding prompts are very same way. My my coding prompts are very same way. My my coding prompts are very verbose. Huh. My coding prompts are verbose. Huh. My coding prompts are verbose. Huh. My coding prompts are short. Well, most of my coding is not short. Well, most of my coding is not short. Well, most of my coding is not enterprise coding. That's That's um enterprise coding. That's That's um enterprise coding. That's That's um Yeah, I do a lot I do a lot of web apps. Yeah, I do a lot I do a lot of web apps. Yeah, I do a lot I do a lot of web apps. I think you're poking around in I think you're poking around in I think you're poking around in different kinds of apps. I'm doing different kinds of apps. I'm doing different kinds of apps. I'm doing Python AI. Yeah. Towards you if I can Python AI. Yeah. Towards you if I can Python AI. Yeah. Towards you if I can trans Well, like that chatbot that you trans Well, like that chatbot that you trans Well, like that chatbot that you all did recently with the temperature all did recently with the temperature all did recently with the temperature thing for me. Yeah, that was largely thing for me. Yeah, that was largely thing for me. Yeah, that was largely written by AI, right? Lab coding. Yeah. written by AI, right? Lab coding. Yeah. written by AI, right? Lab coding. Yeah. Actually, we're going to show that one I Actually, we're going to show that one I Actually, we're going to show that one I think next episode or two. Yeah, let's think next episode or two. Yeah, let's think next episode or two. Yeah, let's do a whole episode on that one because do a whole episode on that one because do a whole episode on that one because that's my favorite demo. And the fact that's my favorite demo. And the fact that's my favorite demo. And the fact that they removed temperature from the that they removed temperature from the that they removed temperature from the log props for OpenAI's completion stuff log props for OpenAI's completion stuff log props for OpenAI's completion stuff kind of ruined my my my favorite go-to kind of ruined my my my favorite go-to kind of ruined my my my favorite go-to demo for teaching AI and now I can do it demo for teaching AI and now I can do it demo for teaching AI and now I can do it with yours. Yeah. Cool. All right. Well, with yours. Yeah. Cool. All right. Well, with yours. Yeah. Cool. All right. Well, thanks. Thanks a lot. This is vibe thanks. Thanks a lot. This is vibe thanks. Thanks a lot. This is vibe coding uh in in pra in practice uh in coding uh in in pra in practice uh in coding uh in in pra in practice uh in production.

Production vibe coding. Production vibe. Production vibe coding. Production vibe. Implications of vibe coding. Implications of vibe coding. Implications of vibe coding. Implications of Yeah, there you go. Implications of Yeah, there you go. Implications of Yeah, there you go. We'll figure it out. We'll figure it out. We'll figure it out. Cool. Uh, once again, if you are one of Cool. Uh, once again, if you are one of Cool. Uh, once again, if you are one of the three people who made it this far the three people who made it this far the three people who made it this far into the depths of the podcast, you can into the depths of the podcast, you can into the depths of the podcast, you can help us by sharing the show and help us by sharing the show and help us by sharing the show and reviewing it. It cannot be overstated reviewing it. It cannot be overstated reviewing it. It cannot be overstated how much we appreciate your comments, how much we appreciate your comments, how much we appreciate your comments, particularly folks that comment on particularly folks that comment on particularly folks that comment on YouTube. You're very kind and it keeps YouTube. You're very kind and it keeps YouTube. You're very kind and it keeps us going. So, please do like and us going. So, please do like and us going. So, please do like and subscribe. Smash that bell uh so that subscribe. Smash that bell uh so that subscribe. Smash that bell uh so that Mark Renovich can become the Mr. Beast Mark Renovich can become the Mr. Beast Mark Renovich can become the Mr. Beast of Vibe Coding.