EPISODE 27 - Scott and Mark Learn To… The AI Productivity Trap: Senior Boost, Junior Drag

I had a young lady on my uh I had a young lady on my uh on my podcast a couple days ago on my on my podcast a couple days ago on my on my podcast a couple days ago on my other podcast other podcast other podcast who is uh a year and a half into being who is uh a year and a half into being who is uh a year and a half into being uh an engineer. She went to Oregon uh an engineer. She went to Oregon uh an engineer. She went to Oregon State, she got her computer science State, she got her computer science State, she got her computer science degree, she's done a half dozen degree, she's done a half dozen degree, she's done a half dozen internships at Meta and Apple and internships at Meta and Apple and internships at Meta and Apple and LinkedIn, and now she's working LinkedIn, and now she's working LinkedIn, and now she's working full-time as an engineer, and it seems full-time as an engineer, and it seems full-time as an engineer, and it seems like the only reason that she's doing so like the only reason that she's doing so like the only reason that she's doing so great is that she's just got a really great is that she's just got a really great is that she's just got a really great team. great team. great team. She's got a bunch of people that want She's got a bunch of people that want She's got a bunch of people that want her to succeed. And I feel like that her to succeed. And I feel like that her to succeed. And I feel like that happened to me as well. Like I'm okay happened to me as well. Like I'm okay happened to me as well. Like I'm okay now because the first 5 years of my now because the first 5 years of my now because the first 5 years of my career I had cool people. Like me. career I had cool people. Like me. career I had cool people. Like me. Much like you. Much like you. Much like you. >> [laughter] >> [laughter] >> [laughter] >> Exactly like Do you know what I mean? >> Exactly like Do you know what I mean? >> Exactly like Do you know what I mean? Like I want everyone to have that level Like I want everyone to have that level Like I want everyone to have that level of support. of support. of support. >> [music] >> So >> So do you think that early-in-career people do you think that early-in-career people do you think that early-in-career people have that supportive network and that have that supportive network and that have that supportive network and that supportive moment? Like Like I met this supportive moment? Like Like I met this supportive moment? Like Like I met this young lady, it's 2025, she's thriving young lady, it's 2025, she's thriving young lady, it's 2025, she's thriving because her team is awesome.

because her team is awesome. because her team is awesome. Um I feel like when we give people Um I feel like when we give people Um I feel like when we give people titles like intern or apprentice titles like intern or apprentice titles like intern or apprentice and then say, you know, get in there and and then say, you know, get in there and and then say, you know, get in there and grind, pull yourself up by your own grind, pull yourself up by your own grind, pull yourself up by your own bootstraps, it's not exactly a bootstraps, it's not exactly a bootstraps, it's not exactly a supportive environment. We need more supportive environment. We need more supportive environment. We need more structure for them to understand how to structure for them to understand how to structure for them to understand how to become a a senior. Well become a a senior. Well become a a senior. Well um um um you know, we've talked about this a lot. you know, we've talked about this a lot. you know, we've talked about this a lot. The software organizational The software organizational The software organizational structure Mhm. that has structure Mhm. that has structure Mhm. that has emerged as a pyramid like many in many emerged as a pyramid like many in many emerged as a pyramid like many in many professions. professions. professions. And you hire junior people in And you hire junior people in And you hire junior people in early-in-career people early-in-career people early-in-career people to augment your product your capacity to to augment your product your capacity to to augment your product your capacity to take on the small tasks that the senior take on the small tasks that the senior take on the small tasks that the senior people would rather not do, you know, people would rather not do, you know, people would rather not do, you know, because they've focused their attention because they've focused their attention because they've focused their attention on on on more complex tasks. more complex tasks. more complex tasks. And the junior people get experience, And the junior people get experience, And the junior people get experience, and then they get given more and more and then they get given more and more and then they get given more and more complex tasks, and so they start to complex tasks, and so they start to complex tasks, and so they start to grow, and eventually they move up into grow, and eventually they move up into grow, and eventually they move up into the next levels of the hierarchy. the next levels of the hierarchy. the next levels of the hierarchy. And some of them move up, and some of And some of them move up, and some of And some of them move up, and some of them don't. them don't. them don't. And eventually you get this pyramid And eventually you get this pyramid And eventually you get this pyramid structure, you know, the senior people structure, you know, the senior people structure, you know, the senior people that have been coding for 20 years, and that have been coding for 20 years, and that have been coding for 20 years, and now they're in management roles. They now they're in management roles. They now they're in management roles. They understand coding, they can spot BS, understand coding, they can spot BS, understand coding, they can spot BS, they understand constraints and how to they understand constraints and how to they understand constraints and how to organize software projects. And organize software projects. And organize software projects. And with vibe coding with vibe coding with vibe coding we run into this risk of the seniority we run into this risk of the seniority we run into this risk of the seniority bias. In fact, there was a paper from bias. In fact, there was a paper from bias. In fact, there was a paper from uh MIT a few weeks ago that came out uh MIT a few weeks ago that came out uh MIT a few weeks ago that came out that showed that there's seniority bias that showed that there's seniority bias that showed that there's seniority bias already emerging in AI-first companies.

already emerging in AI-first companies. already emerging in AI-first companies. And what the seniority bias they see And what the seniority bias they see And what the seniority bias they see is that they take a look at a bunch of is that they take a look at a bunch of is that they take a look at a bunch of different sectors different sectors different sectors and they they identify the companies and they they identify the companies and they they identify the companies that have leaned into AI and look at that have leaned into AI and look at that have leaned into AI and look at their hiring their hiring their hiring trends over the last year or so trends over the last year or so trends over the last year or so [clears throat] [clears throat] [clears throat] and ones that aren't leaning into AI and and ones that aren't leaning into AI and and ones that aren't leaning into AI and look at their hiring, and they see a look at their hiring, and they see a look at their hiring, and they see a shift away from hiring shift away from hiring shift away from hiring junior early-in-career people on the AI junior early-in-career people on the AI junior early-in-career people on the AI companies. Is it a conscious shift? companies. Is it a conscious shift? companies. Is it a conscious shift? They're deciding or it's an it's a it's They're deciding or it's an it's a it's They're deciding or it's an it's a it's happening? I mean, I don't know if it's happening? I mean, I don't know if it's happening? I mean, I don't know if it's conscious or not, it's but it's conscious or not, it's but it's conscious or not, it's but it's happening. And then you can explain the happening. And then you can explain the happening. And then you can explain the reason it's happening is because if you reason it's happening is because if you reason it's happening is because if you and I we talk about our vibe coding, and I we talk about our vibe coding, and I we talk about our vibe coding, we're great at vibe coding and very we're great at vibe coding and very we're great at vibe coding and very productive because we spot the nonsense productive because we spot the nonsense productive because we spot the nonsense of, you know, Yeah, the BS detector is of, you know, Yeah, the BS detector is of, you know, Yeah, the BS detector is fully I'm stopping immediately at that I fully I'm stopping immediately at that I fully I'm stopping immediately at that I don't have to be a BS detector, but the don't have to be a BS detector, but the don't have to be a BS detector, but the oh crap, the AI's stuck, I know how to oh crap, the AI's stuck, I know how to oh crap, the AI's stuck, I know how to unstuck it. Yep. Like, oh, you ran into unstuck it. Yep. Like, oh, you ran into unstuck it. Yep. Like, oh, you ran into this this this race condition and you're sitting here race condition and you're sitting here race condition and you're sitting here flailing about. Well, how about if you flailing about. Well, how about if you flailing about. Well, how about if you just add some debug traces? How about if just add some debug traces? How about if just add some debug traces? How about if you add a sleep so you can force the you add a sleep so you can force the you add a sleep so you can force the race condition, and then you can see race condition, and then you can see race condition, and then you can see better understand it. How about if better understand it. How about if better understand it. How about if when the thing is crashing, how about if when the thing is crashing, how about if when the thing is crashing, how about if you look at this the dump of the crash?

you look at this the dump of the crash? you look at this the dump of the crash? How about when it hangs, you get a a How about when it hangs, you get a a How about when it hangs, you get a a crash dump and look at it? crash dump and look at it? crash dump and look at it? >> only because we've seen it before, >> only because we've seen it before, >> only because we've seen it before, right? Like sometimes being an old right? Like sometimes being an old right? Like sometimes being an old programmer, the privilege of being an programmer, the privilege of being an programmer, the privilege of being an old programmer is you're old, you have old programmer is you're old, you have old programmer is you're old, you have seen it before. Exactly. So you need an seen it before. Exactly. So you need an seen it before. Exactly. So you need an opportunity to see that. How about you opportunity to see that. How about you opportunity to see that. How about you call it later in career? Sorry, I call it later in career? Sorry, I call it later in career? Sorry, I apologize. Later in apologize. Later in apologize. Later in But But I I keep coming back to like why But But I I keep coming back to like why But But I I keep coming back to like why are we doing okay? It's cuz we're still are we doing okay? It's cuz we're still are we doing okay? It's cuz we're still here. So the trick is to keep as many here. So the trick is to keep as many here. So the trick is to keep as many juniors here. juniors here. juniors here. Not having them drop out, not having Not having them drop out, not having Not having them drop out, not having them lose their jobs, not having them them lose their jobs, not having them them lose their jobs, not having them leave, not having them not get the leave, not having them not get the leave, not having them not get the skills. The worst thing I hate seeing is skills. The worst thing I hate seeing is skills. The worst thing I hate seeing is someone who's been in the biz 5 or 10 someone who's been in the biz 5 or 10 someone who's been in the biz 5 or 10 years, and you can tell that they're years, and you can tell that they're years, and you can tell that they're missing like a whole series of missing like a whole series of missing like a whole series of fundamental skills. And it's like, oh fundamental skills. And it's like, oh fundamental skills. And it's like, oh man, you didn't have the opportunity to man, you didn't have the opportunity to man, you didn't have the opportunity to learn running a production system. You learn running a production system. You learn running a production system. You never carried the pager. And never carried the pager. And never carried the pager. And going back to what the trends will be going back to what the trends will be going back to what the trends will be with AI Mhm. augmenting senior people with AI Mhm. augmenting senior people with AI Mhm. augmenting senior people Mhm. experienced people Mhm. experienced people Mhm. experienced people seasoned seasoned seasoned >> experienced people with experience >> experienced people with experience >> experienced people with experience because they know how to use the AI because they know how to use the AI because they know how to use the AI effectively, whereas early-in-career effectively, whereas early-in-career effectively, whereas early-in-career doesn't know what they don't know what doesn't know what they don't know what doesn't know what they don't know what they don't know.

they don't know. they don't know. So they're not as productive. In fact, So they're not as productive. In fact, So they're not as productive. In fact, they can slow down, and it's easy to to they can slow down, and it's easy to to they can slow down, and it's easy to to to see how they can get slowed down. One to see how they can get slowed down. One to see how they can get slowed down. One of those examples of like the race of those examples of like the race of those examples of like the race condition that we talked about in part condition that we talked about in part condition that we talked about in part one, where it put in a sleep Mhm. and one, where it put in a sleep Mhm. and one, where it put in a sleep Mhm. and now the preliminary test pass, but now now the preliminary test pass, but now now the preliminary test pass, but now it's in production it's in production it's in production and the thing fails and the thing fails and the thing fails because of the race condition. Mhm. And because of the race condition. Mhm. And because of the race condition. Mhm. And they have no They're like, the race they have no They're like, the race they have no They're like, the race There's you know, it's fixed, but it's There's you know, it's fixed, but it's There's you know, it's fixed, but it's failing, but how do I fix it? And the failing, but how do I fix it? And the failing, but how do I fix it? And the AI's like, well, let me sleep longer. AI's like, well, let me sleep longer. AI's like, well, let me sleep longer. Let me add extend the sleep. And so that Let me add extend the sleep. And so that Let me add extend the sleep. And so that masks it, and but the they're masks it, and but the they're masks it, and but the they're effectively stuck at that point in effectively stuck at that point in effectively stuck at that point in really fixing it. really fixing it. really fixing it. Whereas a senior person's like, well, Whereas a senior person's like, well, Whereas a senior person's like, well, that's a stupid thing to do in the first that's a stupid thing to do in the first that's a stupid thing to do in the first place, never let it go into production. place, never let it go into production. place, never let it go into production. Um Um Um so the incentive is just to hire people so the incentive is just to hire people so the incentive is just to hire people that are that are that are experienced because they're the ones experienced because they're the ones experienced because they're the ones that are going to get what I call the AI that are going to get what I call the AI that are going to get what I call the AI boost. boost. boost. And the junior the early-in-career And the junior the early-in-career And the junior the early-in-career people are going to get an AI people are going to get an AI people are going to get an AI slowdown, actually. slowdown, actually. slowdown, actually. Uh because Uh because Uh because they're going to hit these problems and they're going to hit these problems and they're going to hit these problems and have have have >> Which is similar to the Google slowdown >> Which is similar to the Google slowdown >> Which is similar to the Google slowdown that you get when you're early-in-career that you get when you're early-in-career that you get when you're early-in-career and you spend 3 hours Googling and you and you spend 3 hours Googling and you and you spend 3 hours Googling and you run down a forums thing, and it shows run down a forums thing, and it shows run down a forums thing, and it shows that you fundamentally don't know what that you fundamentally don't know what that you fundamentally don't know what you're doing, and you're grasping for you're doing, and you're grasping for you're doing, and you're grasping for straws, and now you're just grasping for straws, and now you're just grasping for straws, and now you're just grasping for AI uh augmented straws. We need to come AI uh augmented straws. We need to come AI uh augmented straws. We need to come up with a better name. So AI boost, up with a better name. So AI boost, up with a better name. So AI boost, what's the counter to that? AI delay? I what's the counter to that? AI delay? I what's the counter to that? AI delay? I don't know. Mhm. Tell us in the chat don't know. Mhm. Tell us in the chat don't know. Mhm. Tell us in the chat what you [clears throat] think cuz Mark what you [clears throat] think cuz Mark what you [clears throat] think cuz Mark and I are working on a paper about the

and I are working on a paper about the and I are working on a paper about the software engineering profession, and we software engineering profession, and we software engineering profession, and we all know that we're at a all know that we're at a all know that we're at a uh an influx moment. We're at a moment a uh an influx moment. We're at a moment a uh an influx moment. We're at a moment a pivotal shift in what's going on, and pivotal shift in what's going on, and pivotal shift in what's going on, and we've got I think an inflection an we've got I think an inflection an we've got I think an inflection an inflection Not an influx is We have an inflection Not an influx is We have an inflection Not an influx is We have an influx. We are We are facing an influx. influx. We are We are facing an influx. influx. We are We are facing an influx. Influx of what? Of nonsense of Influx of what? Of nonsense of Influx of what? Of nonsense of AI-generated slop. Slop, yeah. So yes, AI-generated slop. Slop, yeah. So yes, AI-generated slop. Slop, yeah. So yes, par- pardon me, yes, not an influx of uh par- pardon me, yes, not an influx of uh par- pardon me, yes, not an influx of uh of an influx of AI slop, but yes, we're of an influx of AI slop, but yes, we're of an influx of AI slop, but yes, we're at an inflection point from a hiring at an inflection point from a hiring at an inflection point from a hiring perspective. We need more people. We perspective. We need more people. We perspective. We need more people. We need people need people need people >> to have a talent pipeline. A Yeah, >> to have a talent pipeline. A Yeah, >> to have a talent pipeline. A Yeah, exactly. We need We need more seniors, exactly. We need We need more seniors, exactly. We need We need more seniors, and the only way you get seniors we keep and the only way you get seniors we keep and the only way you get seniors we keep we keep talking about how we're going to we keep talking about how we're going to we keep talking about how we're going to discover them like there's some cache of discover them like there's some cache of discover them like there's some cache of senior people that are just out there. senior people that are just out there. senior people that are just out there. Um there are Um there are Um there are of course there's the the layoffs right of course there's the the layoffs right of course there's the the layoffs right now are are right are a challenge. Um now are are right are a challenge. Um now are are right are a challenge. Um but even senior people are struggling but even senior people are struggling but even senior people are struggling because they're looking at AI and um and because they're looking at AI and um and because they're looking at AI and um and thinking that they're being replaced by thinking that they're being replaced by thinking that they're being replaced by it.

it. it. Um we don't know what our jobs are going Um we don't know what our jobs are going Um we don't know what our jobs are going to look like in 3 to 5 years, and I to look like in 3 to 5 years, and I to look like in 3 to 5 years, and I think we're all struggling with that. So think we're all struggling with that. So think we're all struggling with that. So what we we've got what we we've got what we we've got uh you we have a we have a paper here, uh you we have a we have a paper here, uh you we have a we have a paper here, and in some of the sections of the and in some of the sections of the and in some of the sections of the paper, one of them is implications of paper, one of them is implications of paper, one of them is implications of coding agents on productivity. Mhm. coding agents on productivity. Mhm. coding agents on productivity. Mhm. >> And we we filled the paper with >> And we we filled the paper with >> And we we filled the paper with references to a lot of the people that references to a lot of the people that references to a lot of the people that are thinking about this this space. are thinking about this this space. are thinking about this this space. We both have referred to AI coding We both have referred to AI coding We both have referred to AI coding agents as enthusiastic interns. agents as enthusiastic interns. agents as enthusiastic interns. And even though we don't like to And even though we don't like to And even though we don't like to anthropomorphize anthropomorphize anthropomorphize >> that's a common term common way people >> that's a common term common way people >> that's a common term common way people are viewing it. are viewing it. are viewing it. >> Yeah, is that a good way to think about >> Yeah, is that a good way to think about >> Yeah, is that a good way to think about it? I absolutely. It's but it's interns it? I absolutely. It's but it's interns it? I absolutely. It's but it's interns that don't learn, too, because you tell that don't learn, too, because you tell that don't learn, too, because you tell the you tell the AI one day, hey, that's the you tell the AI one day, hey, that's the you tell the AI one day, hey, that's not the way to do race condition not the way to do race condition not the way to do race condition 2 days later 2 days later 2 days later the AI is going to do it again because the AI is going to do it again because the AI is going to do it again because it doesn't remember that unless you put it doesn't remember that unless you put it doesn't remember that unless you put it in your coding instructions, but you it in your coding instructions, but you it in your coding instructions, but you can't put the world of tips and tricks can't put the world of tips and tricks can't put the world of tips and tricks and everything in your prompt file for and everything in your prompt file for and everything in your prompt file for the AI. So effectively they just don't the AI. So effectively they just don't the AI. So effectively they just don't learn learn learn from the mistakes.

from the mistakes. from the mistakes. Uh or from your guidance. And so they're Uh or from your guidance. And so they're Uh or from your guidance. And so they're interns that never learn. interns that never learn. interns that never learn. >> So then every developer will become a >> So then every developer will become a >> So then every developer will become a engineering manager engineering manager engineering manager of a team of interns that that don't of a team of interns that that don't of a team of interns that that don't learn. Yeah. But we need to bring in learn. Yeah. But we need to bring in learn. Yeah. But we need to bring in interns and juniors and apprentices that interns and juniors and apprentices that interns and juniors and apprentices that will learn and give them a different will learn and give them a different will learn and give them a different space in which to learn. They're going space in which to learn. They're going space in which to learn. They're going to learn differently than we learned. We to learn differently than we learned. We to learn differently than we learned. We learned by reading books and suffering learned by reading books and suffering learned by reading books and suffering and writing stack overflows and things and writing stack overflows and things and writing stack overflows and things like that. Uh like that. Uh like that. Uh you said that the value of AI is it you said that the value of AI is it you said that the value of AI is it locked unlocked by experience. We need locked unlocked by experience. We need locked unlocked by experience. We need to give them experience. We need to give to give them experience. We need to give to give them experience. We need to give them experience, and we need to give them experience, and we need to give them experience, and we need to give them the room to do it. So them the room to do it. So them the room to do it. So in this world, and in fact, I had in this world, and in fact, I had in this world, and in fact, I had somebody on my team early-in-career somebody on my team early-in-career somebody on my team early-in-career I had a check-in with them, and they I had a check-in with them, and they I had a check-in with them, and they were were were after we did the how are things going after we did the how are things going after we did the how are things going she said [clears throat] she said [clears throat] she said [clears throat] I got a question for you. I'm I got a question for you. I'm I got a question for you. I'm early-in-career, just out of college a early-in-career, just out of college a early-in-career, just out of college a few years. few years. few years. I know I'm an inexperienced programmer.

I know I'm an inexperienced programmer. I know I'm an inexperienced programmer. I know that you and the leadership, I know that you and the leadership, I know that you and the leadership, Satya, you, my manager, are all saying Satya, you, my manager, are all saying Satya, you, my manager, are all saying use AI. use AI. use AI. But how much should I rely on AI? But how much should I rely on AI? But how much should I rely on AI? Because if I'm using it, I'm not going Because if I'm using it, I'm not going Because if I'm using it, I'm not going to learn. to learn. to learn. Very [clears throat] thoughtful thing Very [clears throat] thoughtful thing Very [clears throat] thoughtful thing for for for >> insightful. >> insightful. >> insightful. And um you know, And um you know, And um you know, reinforced cuz we'd already been talking reinforced cuz we'd already been talking reinforced cuz we'd already been talking about this. about this. about this. If you if they're using AI, they're not If you if they're using AI, they're not If you if they're using AI, they're not going to learn. In fact, they're going going to learn. In fact, they're going going to learn. In fact, they're going to get slowed down when the AI makes to get slowed down when the AI makes to get slowed down when the AI makes mistakes and then they're trying to go mistakes and then they're trying to go mistakes and then they're trying to go figure out how to unstick it and they figure out how to unstick it and they figure out how to unstick it and they don't really know how to. don't really know how to. don't really know how to. So, [snorts] So, [snorts] So, [snorts] my recommendation then is my recommendation then is my recommendation then is we should not view the early in career we should not view the early in career we should not view the early in career people we're going to have to shift to a people we're going to have to shift to a people we're going to have to shift to a model where we're not hiring early in model where we're not hiring early in model where we're not hiring early in career people to augment our capacity. career people to augment our capacity. career people to augment our capacity. Mhm. Cuz the intern AI interns are doing Mhm. Cuz the intern AI interns are doing Mhm. Cuz the intern AI interns are doing that job that they human early in that job that they human early in that job that they human early in careers would normally have done. careers would normally have done. careers would normally have done. And the early in career people And the early in career people And the early in career people need to be given opportunities to learn, need to be given opportunities to learn, need to be given opportunities to learn, but with AI, but they're going to be but with AI, but they're going to be but with AI, but they're going to be slower slower slower than they than they than they you know, were perhaps were in the past.

you know, were perhaps were in the past. you know, were perhaps were in the past. Yeah. Cuz they're learning new things Yeah. Cuz they're learning new things Yeah. Cuz they're learning new things and learning AI. and learning AI. and learning AI. And we just and we have to just accept And we just and we have to just accept And we just and we have to just accept that. That if we hire that. That if we hire that. That if we hire a hand someone, that's going to be the a hand someone, that's going to be the a hand someone, that's going to be the equivalent of today's equivalent of today's equivalent of today's 10x developer. 10x developer. 10x developer. With AI. With AI. With AI. Uh but Uh but Uh but >> I got an email today >> I got an email today >> I got an email today with feedback for the show and someone with feedback for the show and someone with feedback for the show and someone had said that had said that had said that they didn't like that term 10x that we they didn't like that term 10x that we they didn't like that term 10x that we talked about talked about talked about before. before. before. And I and I know it's it's a shorthand And I and I know it's it's a shorthand And I and I know it's it's a shorthand and I want to I don't want to derail too and I want to I don't want to derail too and I want to I don't want to derail too much, but like much, but like much, but like is it is it is it helpful or good or problematic to refer helpful or good or problematic to refer helpful or good or problematic to refer to people in to people in to people in >> say that Look, so how do we talk about >> say that Look, so how do we talk about >> say that Look, so how do we talk about an AI boost? So, let's say that AI is an AI boost? So, let's say that AI is an AI boost? So, let's say that AI is going to boost the productivity of going to boost the productivity of going to boost the productivity of somebody that's been in a software somebody that's been in a software somebody that's been in a software programmer today is software programmer programmer today is software programmer programmer today is software programmer that's been in the profession for 10 that's been in the profession for 10 that's been in the profession for 10 years and you're going to get a 5 to 10x years and you're going to get a 5 to 10x years and you're going to get a 5 to 10x boost from AI. I boost from AI. I boost from AI. I if if if they're sheer shipping product if if if they're sheer shipping product if if if they're sheer shipping product productivity, yeah. I mean, depends on productivity, yeah. I mean, depends on productivity, yeah. I mean, depends on what I'm shipping. The early in career what I'm shipping. The early in career what I'm shipping. The early in career person is going to have a 1.5x slowdown person is going to have a 1.5x slowdown person is going to have a 1.5x slowdown because of AI. Mhm. So, who do you want because of AI. Mhm. So, who do you want because of AI. Mhm. So, who do you want to hire?

to hire? to hire? Do you hire and the by the way, the the Do you hire and the by the way, the the Do you hire and the by the way, the the 10-year person is doing all the jobs 10-year person is doing all the jobs 10-year person is doing all the jobs that the new that the new that the new in career people were doing because AI in career people were doing because AI in career people were doing because AI interns are doing that work, setting up interns are doing that work, setting up interns are doing that work, setting up the build pipelines and the build pipelines and the build pipelines and doing the doing the doing the other kind of state tasks that go along other kind of state tasks that go along other kind of state tasks that go along with the with the with the >> the glue work. >> the glue work. >> the glue work. >> Yeah. Right. And and we are proposing in >> Yeah. Right. And and we are proposing in >> Yeah. Right. And and we are proposing in this paper and we're proposing generally this paper and we're proposing generally this paper and we're proposing generally and publicly that to do that, to focus and publicly that to do that, to focus and publicly that to do that, to focus on your seniors and giving them a boost on your seniors and giving them a boost on your seniors and giving them a boost may be exciting, but it is ultimately may be exciting, but it is ultimately may be exciting, but it is ultimately short-sided because It's short-sided. short-sided because It's short-sided. short-sided because It's short-sided. >> for your organization. It's short-sided >> for your organization. It's short-sided >> for your organization. It's short-sided for the whole industry. It's very for the whole industry. It's very for the whole industry. It's very short-sided. So, the industry is going short-sided. So, the industry is going short-sided. So, the industry is going to change and we need to have a a formal to change and we need to have a a formal to change and we need to have a a formal knowledge transfer knowledge transfer knowledge transfer enhanced by AI between the seniors that enhanced by AI between the seniors that enhanced by AI between the seniors that know what they're doing right now and know what they're doing right now and know what they're doing right now and the early in career. And it might be the early in career. And it might be the early in career. And it might be simplistic to just call that an simplistic to just call that an simplistic to just call that an internship. Like formal we should internship. Like formal we should internship. Like formal we should formalize internships. I think it's more formalize internships. I think it's more formalize internships. I think it's more than that. I think we need to make than that. I think we need to make than that. I think we need to make >> as a a >> as a a >> as a a >> or an apprenticeship, an internship, >> or an apprenticeship, an internship, >> or an apprenticeship, an internship, you know, a journey moving someone to you know, a journey moving someone to you know, a journey moving someone to the journeyman. Uh you and I have gone the journeyman. Uh you and I have gone the journeyman. Uh you and I have gone back and forth about whether I'm I'm back and forth about whether I'm I'm back and forth about whether I'm I'm parsing the words too much and I'm parsing the words too much and I'm parsing the words too much and I'm interested in folks in the chat what interested in folks in the chat what interested in folks in the chat what they think, but I feel like if you they think, but I feel like if you they think, but I feel like if you if you put the onus on the senior to if you put the onus on the senior to if you put the onus on the senior to pull the junior up and give them a title pull the junior up and give them a title pull the junior up and give them a title to indicate that they're now a teacher, to indicate that they're now a teacher, to indicate that they're now a teacher, they're a they're a proctor, they're a they're a they're a proctor, they're a they're a they're a proctor, they're a preceptor preceptor preceptor rather than labeling this person an rather than labeling this person an rather than labeling this person an intern or an apprentice.

intern or an apprentice. intern or an apprentice. Um Um Um and then you would you would actually and then you would you would actually and then you would you would actually um metric and and reward the senior for um metric and and reward the senior for um metric and and reward the senior for the more juniors that they brought up. the more juniors that they brought up. the more juniors that they brought up. Like if if one senior is just chewing Like if if one senior is just chewing Like if if one senior is just chewing them up and spitting them out, you're them up and spitting them out, you're them up and spitting them out, you're like, "Hey, you're probably not like, "Hey, you're probably not like, "Hey, you're probably not like right for this job. You shouldn't like right for this job. You shouldn't like right for this job. You shouldn't be teaching." be teaching." be teaching." So, we need to find the best senior So, we need to find the best senior So, we need to find the best senior teachers. I definitely think that that teachers. I definitely think that that teachers. I definitely think that that focus on the focus on the focus on the teacher teacher teacher Mhm. and making clear and we are just Mhm. and making clear and we are just Mhm. and making clear and we are just internal discussions at Microsoft are internal discussions at Microsoft are internal discussions at Microsoft are yes, in fact, that should be their yes, in fact, that should be their yes, in fact, that should be their primary job responsibility. Yes, they primary job responsibility. Yes, they primary job responsibility. Yes, they should still continue to code and should still continue to code and should still continue to code and participate in the you know, participate in the you know, participate in the you know, contribute to the project, but that's contribute to the project, but that's contribute to the project, but that's not their primary role anymore. Right. not their primary role anymore. Right. not their primary role anymore. Right. >> role is bringing up the junior people, >> role is bringing up the junior people, >> role is bringing up the junior people, sitting side by side with them, teaching sitting side by side with them, teaching sitting side by side with them, teaching them the system. them the system. them the system. Um having Um having Um having uh looking for their weaknesses and uh looking for their weaknesses and uh looking for their weaknesses and helping them grow in those places. helping them grow in those places. helping them grow in those places. Uh so that Uh so that Uh so that like let's take that vibe coded, you like let's take that vibe coded, you like let's take that vibe coded, you know, website that from last episode we know, website that from last episode we know, website that from last episode we talked about that's growing over time. talked about that's growing over time. talked about that's growing over time. Mhm. Somebody needs to understand that Mhm. Somebody needs to understand that Mhm. Somebody needs to understand that site so that when the AI fails at it or site so that when the AI fails at it or site so that when the AI fails at it or the system fails in some way, they can the system fails in some way, they can the system fails in some way, they can figure out what what's going on and what figure out what what's going on and what figure out what what's going on and what to do.

to do. to do. They need to be They need to be They need to be mentoring people mentoring people mentoring people to take over to take over to take over that that that giving the transferring that same giving the transferring that same giving the transferring that same knowledge like you said about how the knowledge like you said about how the knowledge like you said about how the system works. system works. system works. And so that requires sitting side down And so that requires sitting side down And so that requires sitting side down by the giving them bigger and bigger by the giving them bigger and bigger by the giving them bigger and bigger tasks tasks tasks having them use AI, but having AI also having them use AI, but having AI also having them use AI, but having AI also teach them at the same time. And but teach them at the same time. And but teach them at the same time. And but right now, we don't give the the seniors right now, we don't give the the seniors right now, we don't give the the seniors enough space. enough space. enough space. They're shipping. Yeah. The juniors are They're shipping. Yeah. The juniors are They're shipping. Yeah. The juniors are getting chewed up and the the the getting chewed up and the the the getting chewed up and the the the seniors don't have time to sit with seniors don't have time to sit with seniors don't have time to sit with them, but if it was their primary job them, but if it was their primary job them, but if it was their primary job function, if the idea that you're not a function, if the idea that you're not a function, if the idea that you're not a principal engineer, you're a principal principal engineer, you're a principal principal engineer, you're a principal engineering manager, you're a principal engineering manager, you're a principal engineering manager, you're a principal engineering preceptor. engineering preceptor. engineering preceptor. You're the teacher and you're going to You're the teacher and you're going to You're the teacher and you're going to manage by walking around making sure manage by walking around making sure manage by walking around making sure that you it is now not a part-time job. that you it is now not a part-time job. that you it is now not a part-time job. It is a full-time job to make sure that It is a full-time job to make sure that It is a full-time job to make sure that the early in career engineering the early in career engineering the early in career engineering practitioners are guided and grown and practitioners are guided and grown and practitioners are guided and grown and they learn the production function. they learn the production function. they learn the production function. Yeah. And I think so we're starting a Yeah. And I think so we're starting a Yeah. And I think so we're starting a pilot on that at Microsoft. Mhm. pilot on that at Microsoft. Mhm. pilot on that at Microsoft. Mhm. What I talked to I by the way, this What I talked to I by the way, this What I talked to I by the way, this topic is a hot topic in all my customer topic is a hot topic in all my customer topic is a hot topic in all my customer engagements.

engagements. engagements. The topic of AI boosting in software The topic of AI boosting in software The topic of AI boosting in software develop what's it doing to software develop what's it doing to software develop what's it doing to software development comes up and when I share development comes up and when I share development comes up and when I share these these these insights that we've had of the insights that we've had of the insights that we've had of the the 5 to 10x boost for somebody that is the 5 to 10x boost for somebody that is the 5 to 10x boost for somebody that is senior or experienced senior or experienced senior or experienced and 1.5 to 2x slowdown for and 1.5 to 2x slowdown for and 1.5 to 2x slowdown for somebody that's early in career, they somebody that's early in career, they somebody that's early in career, they all say they see it at their companies. all say they see it at their companies. all say they see it at their companies. And so they're very interested in our And so they're very interested in our And so they're very interested in our thinking about this kind of model. And thinking about this kind of model. And thinking about this kind of model. And I'll copy at this we you know, we talked I'll copy at this we you know, we talked I'll copy at this we you know, we talked about last episode about making about last episode about making about last episode about making declarative statements and then declarative statements and then declarative statements and then retrospect being completely wrong. We I retrospect being completely wrong. We I retrospect being completely wrong. We I had I'm open and I think this is part of had I'm open and I think this is part of had I'm open and I think this is part of our our our growth mindset Satya would call it growth mindset Satya would call it growth mindset Satya would call it to being wrong or to to not having the to being wrong or to to not having the to being wrong or to to not having the right formula here for this right formula here for this right formula here for this and learning from you know, as we go, and learning from you know, as we go, and learning from you know, as we go, but yeah. but yeah. but yeah. I have found in the pitching of this I have found in the pitching of this I have found in the pitching of this idea as we've been kind of shopping it idea as we've been kind of shopping it idea as we've been kind of shopping it around uh both internally and externally around uh both internally and externally around uh both internally and externally that it is easy to just say, "Oh, so that it is easy to just say, "Oh, so that it is easy to just say, "Oh, so interns? Like you want like more interns? Like you want like more interns? Like you want like more apprentices?" Cuz everyone's done that apprentices?" Cuz everyone's done that apprentices?" Cuz everyone's done that before. "Oh, it's a boot camp."

before. "Oh, it's a boot camp." before. "Oh, it's a boot camp." It's I it's it's not. It's inverted. It's I it's it's not. It's inverted. It's I it's it's not. It's inverted. It's it's changing the job of the It's it's changing the job of the It's it's changing the job of the senior. And if you think about like this senior. And if you think about like this senior. And if you think about like this case of learning the code base, learning case of learning the code base, learning case of learning the code base, learning the system, it can't be an internship. the system, it can't be an internship. the system, it can't be an internship. That takes a lot of time and experience That takes a lot of time and experience That takes a lot of time and experience and energy and and energy and and energy and to to learn the whole system and how it to to learn the whole system and how it to to learn the whole system and how it works and the reason why things are the works and the reason why things are the works and the reason why things are the way they are and way they are and way they are and and the interact the complex and the interact the complex and the interact the complex interactions and have that world model. interactions and have that world model. interactions and have that world model. There's 10 microservices that we've got There's 10 microservices that we've got There's 10 microservices that we've got to play here and these are the different to play here and these are the different to play here and these are the different roles and here's how they interact and roles and here's how they interact and roles and here's how they interact and if you change something over here, well, if you change something over here, well, if you change something over here, well, that has ripples through the whole that has ripples through the whole that has ripples through the whole thing. thing. thing. That that is what a preceptor has to That that is what a preceptor has to That that is what a preceptor has to download. And there's a scope there's a download. And there's a scope there's a download. And there's a scope there's a there's a time scope as well. People there's a time scope as well. People there's a time scope as well. People want to do these things in 90 days. want to do these things in 90 days. want to do these things in 90 days. Everyone Oh, yeah, well, if they're an Everyone Oh, yeah, well, if they're an Everyone Oh, yeah, well, if they're an intern and they're an an apprentice, intern and they're an an apprentice, intern and they're an an apprentice, they'll do a 90-day program. This is a they'll do a 90-day program. This is a they'll do a 90-day program. This is a year, two, three, five years. Yeah. year, two, three, five years. Yeah. year, two, three, five years. Yeah. Yeah, my my instinct is that this is a Yeah, my my instinct is that this is a Yeah, my my instinct is that this is a three-year thing. three-year thing. three-year thing. >> Three-year. Yeah, I think it's an >> Three-year. Yeah, I think it's an >> Three-year. Yeah, I think it's an investment. I don't think So, okay, so investment. I don't think So, okay, so investment. I don't think So, okay, so then here's a question then. What if then here's a question then. What if then here's a question then. What if you're a small company? Right? What if you're a small company? Right? What if you're a small company? Right? What if you're only 10 people? Like big you're only 10 people? Like big you're only 10 people? Like big companies can absorb stuff like this.

companies can absorb stuff like this. companies can absorb stuff like this. How would a small company run a pipeline How would a small company run a pipeline How would a small company run a pipeline like this? Well, I actually that's an like this? Well, I actually that's an like this? Well, I actually that's an interesting question and I actually interesting question and I actually interesting question and I actually think that small companies don't have to think that small companies don't have to think that small companies don't have to face this kind of problem. face this kind of problem. face this kind of problem. It's when you start to get It's when you start to get It's when you start to get medium-sized when medium-sized when medium-sized when when you have complex systems when you have complex systems when you have complex systems and you've got a longevity where you and you've got a longevity where you and you've got a longevity where you have to have this knowledge download have to have this knowledge download have to have this knowledge download >> Mhm. architecture is complicated, has >> Mhm. architecture is complicated, has >> Mhm. architecture is complicated, has evolved over time. evolved over time. evolved over time. When you're a startup, I wouldn't do When you're a startup, I wouldn't do When you're a startup, I wouldn't do this. Okay. this. Okay. this. Okay. >> point in doing it. I I I'm I'm saying by >> point in doing it. I I I'm I'm saying by >> point in doing it. I I I'm I'm saying by the way, this is all like smeared across the way, this is all like smeared across the way, this is all like smeared across time. So, who should the who should the time. So, who should the who should the time. So, who should the who should the startup startup startup yeah. yeah. yeah. >> medium experience. >> medium experience. >> medium experience. I wouldn't hire a early in career in a I wouldn't hire a early in career in a I wouldn't hire a early in career in a startup. At this you know, I'm I'm I'm startup. At this you know, I'm I'm I'm startup. At this you know, I'm I'm I'm I'm saying by the way, this is all like I'm saying by the way, this is all like I'm saying by the way, this is all like smeared across time. Yeah. Yeah. Today, smeared across time. Yeah. Yeah. Today, smeared across time. Yeah. Yeah. Today, maybe I would. In a year, two years as maybe I would. In a year, two years as maybe I would. In a year, two years as AI agents become even more AI agents become even more AI agents become even more >> Right. When AI is everywhere in >> Right. When AI is everywhere in >> Right. When AI is everywhere in engineering. Then engineering. Then engineering. Then you know, I would I wouldn't because you you know, I would I wouldn't because you you know, I would I wouldn't because you get it's just going to slow you down and get it's just going to slow you down and get it's just going to slow you down and you don't need the talent pipeline as a you don't need the talent pipeline as a you don't need the talent pipeline as a startup. Like that's a problem for the startup. Like that's a problem for the startup. Like that's a problem for the future. It feels like traditionally future. It feels like traditionally future. It feels like traditionally early in career people have been hired early in career people have been hired early in career people have been hired by small companies and then they build by small companies and then they build by small companies and then they build up to the large company. You're almost up to the large company. You're almost up to the large company. You're almost implying that large companies should implying that large companies should implying that large companies should start hiring early in career and start hiring early in career and start hiring early in career and training them. Yeah.

training them. Yeah. training them. Yeah. >> And then they can go off and do >> And then they can go off and do >> And then they can go off and do startups. startups. startups. Which is the inverse is inverse inverse Which is the inverse is inverse inverse Which is the inverse is inverse inverse of what we're doing now. Well, um I of what we're doing now. Well, um I of what we're doing now. Well, um I mean, that's why you need for every mean, that's why you need for every mean, that's why you need for every senior person you've got, you probably senior person you've got, you probably senior person you've got, you probably have to you still have to have you don't have to you still have to have you don't have to you still have to have you don't have the the 100 to 10 to one kind of have the the 100 to 10 to one kind of have the the 100 to 10 to one kind of pyramid anymore. Right. It's a lot more pyramid anymore. Right. It's a lot more pyramid anymore. Right. It's a lot more narrow. narrow. narrow. But you need some expansion at the But you need some expansion at the But you need some expansion at the bottom. bottom. bottom. Hey, I'm going to hire four people for Hey, I'm going to hire four people for Hey, I'm going to hire four people for every senior that I've got Right. to every senior that I've got Right. to every senior that I've got Right. to grow them because I know that grow them because I know that grow them because I know that two are going to go do a startup, one is two are going to go do a startup, one is two are going to go do a startup, one is going to decide that it this is not for going to decide that it this is not for going to decide that it this is not for them. them. them. >> Yeah. >> Yeah. >> Yeah. And we also propose that the way that And we also propose that the way that And we also propose that the way that models coding models talk to the models coding models talk to the models coding models talk to the engineers engineers engineers would be different for juniors versus would be different for juniors versus would be different for juniors versus seniors. Um like I'll tell I I was seniors. Um like I'll tell I I was seniors. Um like I'll tell I I was telling a Claude Sonnet model on GitHub telling a Claude Sonnet model on GitHub telling a Claude Sonnet model on GitHub Copilot yesterday to stop being so Copilot yesterday to stop being so Copilot yesterday to stop being so sycophantic. sycophantic. sycophantic. Every freaking You need to put that in Every freaking You need to put that in Every freaking You need to put that in your Claude instructions that I gave your Claude instructions that I gave your Claude instructions that I gave you. you. you. >> You're absolutely right. Like, bro, But >> You're absolutely right. Like, bro, But >> You're absolutely right. Like, bro, But you know what? If you just told it, you know what? If you just told it, you know what? If you just told it, tomorrow it's going to do it again cuz tomorrow it's going to do it again cuz tomorrow it's going to do it again cuz it won't remember.

it won't remember. it won't remember. >> But but the argument being that could >> But but the argument being that could >> But but the argument being that could you make a model that is explicitly you make a model that is explicitly you make a model that is explicitly supportive of early in career engineers? supportive of early in career engineers? supportive of early in career engineers? Cuz I've done talks on how early in Cuz I've done talks on how early in Cuz I've done talks on how early in career engineers should prompt and career engineers should prompt and career engineers should prompt and should talk to the models so that they should talk to the models so that they should talk to the models so that they might learn and prompt, no pun intended, might learn and prompt, no pun intended, might learn and prompt, no pun intended, reflection, prompt explanation. reflection, prompt explanation. reflection, prompt explanation. And uh And uh And uh Not every model works for every I think Not every model works for every I think Not every model works for every I think absolutely early in career to make to absolutely early in career to make to absolutely early in career to make to learn while they're using AI. And again, learn while they're using AI. And again, learn while they're using AI. And again, in that um that model where, "Hey, we in that um that model where, "Hey, we in that um that model where, "Hey, we want you to learn. That's your primary want you to learn. That's your primary want you to learn. That's your primary job is learning, not necessarily, you job is learning, not necessarily, you job is learning, not necessarily, you know, number of lines of code you're know, number of lines of code you're know, number of lines of code you're putting into production." putting into production." putting into production." You I imagine, and I already we're You I imagine, and I already we're You I imagine, and I already we're talking to with GitHub about this, Mhm. talking to with GitHub about this, Mhm. talking to with GitHub about this, Mhm. of of of the agent is saying, the agent is saying, the agent is saying, when the when the user comes and says, when the when the user comes and says, when the when the user comes and says, "Hey, go write a function that does "Hey, go write a function that does "Hey, go write a function that does this." this." this." The AI can say, "Oh, why don't you The AI can say, "Oh, why don't you The AI can say, "Oh, why don't you sketch that out the outline for that for sketch that out the outline for that for sketch that out the outline for that for me first?" Yep. Or here, I'll go do it. me first?" Yep. Or here, I'll go do it. me first?" Yep. Or here, I'll go do it. Now, explain to me why I did this thing Now, explain to me why I did this thing Now, explain to me why I did this thing and and how it works.

and and how it works. and and how it works. Um so, interactive, and this is like Con Um so, interactive, and this is like Con Um so, interactive, and this is like Con Migo, Migo, Migo, um which is the educational um which is the educational um which is the educational uh chatbot that helps students learn uh chatbot that helps students learn uh chatbot that helps students learn math and science in that same way. math and science in that same way. math and science in that same way. Having the user, instead of giving them Having the user, instead of giving them Having the user, instead of giving them the answer, the answer, the answer, kind of guiding them through the thought kind of guiding them through the thought kind of guiding them through the thought process to get the answer, quizzing them process to get the answer, quizzing them process to get the answer, quizzing them about how deeply they understand the about how deeply they understand the about how deeply they understand the answer, and building up, by the way, a answer, and building up, by the way, a answer, and building up, by the way, a knowledge about how much knowledge about how much knowledge about how much where their weaknesses are. I think you where their weaknesses are. I think you where their weaknesses are. I think you see need the same thing here. So, this see need the same thing here. So, this see need the same thing here. So, this that model that Con Migo model, I think that model that Con Migo model, I think that model that Con Migo model, I think translates really nicely. Mhm. translates really nicely. Mhm. translates really nicely. Mhm. So, as we end, you're we're at an infle- So, as we end, you're we're at an infle- So, as we end, you're we're at an infle- we're at an inflection point. we're at an inflection point. we're at an inflection point. What do what should university students What do what should university students What do what should university students that are studying CS right now, that are studying CS right now, that are studying CS right now, sophomores, juniors, seniors in CS, be sophomores, juniors, seniors in CS, be sophomores, juniors, seniors in CS, be thinking about as we are in that point? thinking about as we are in that point? thinking about as we are in that point? >> I got So, this actually um I have a >> I got So, this actually um I have a >> I got So, this actually um I have a friend that's got uh student in computer friend that's got uh student in computer friend that's got uh student in computer science that's a junior, and he said he science that's a junior, and he said he science that's a junior, and he said he was talking to them was talking to them was talking to them and said asking them, "Do you use AI?" and said asking them, "Do you use AI?" and said asking them, "Do you use AI?" And he says, like, "Yeah, a lot of my And he says, like, "Yeah, a lot of my And he says, like, "Yeah, a lot of my uh stu- fellow students uh stu- fellow students uh stu- fellow students are using AI. I don't use AI are using AI. I don't use AI are using AI. I don't use AI cuz I want to learn cuz I want to learn cuz I want to learn the hard way." I think that both is the the hard way." I think that both is the the hard way." I think that both is the right answer though. I feel like both, right answer though. I feel like both, right answer though. I feel like both, but here's what I'll tell you. Right but here's what I'll tell you. Right but here's what I'll tell you. Right now, I think that universities don't now, I think that universities don't now, I think that universities don't have a good have a good have a good model that a a struc- you know, model that a a struc- you know, model that a a struc- you know, consistent consistent consistent >> be- they're behind. Academia might, but >> be- they're behind. Academia might, but >> be- they're behind. Academia might, but like like like research level academia, like like like research level academia, like like like research level academia, but like but like but like Not for teaching them. Not for teaching Not for teaching them. Not for teaching Not for teaching them. Not for teaching undergrads, right?

undergrads, right? undergrads, right? >> undergrads. And actually, I I think what >> undergrads. And actually, I I think what >> undergrads. And actually, I I think what is coming into view for me is that you is coming into view for me is that you is coming into view for me is that you need classes where AI is considered need classes where AI is considered need classes where AI is considered using AI for certain projects or certain using AI for certain projects or certain using AI for certain projects or certain classes is considered cheating. classes is considered cheating. classes is considered cheating. Mhm. And Mhm. And Mhm. And not to say that you don't have classes not to say that you don't have classes not to say that you don't have classes and projects in some classes and projects in some classes and projects in some classes where where where the student is told to use AI. the student is told to use AI. the student is told to use AI. But you need to have like the main But you need to have like the main But you need to have like the main basis for the education on computer basis for the education on computer basis for the education on computer science and programming science and programming science and programming to be AI-less. to be AI-less. to be AI-less. Because that's the only way the students Because that's the only way the students Because that's the only way the students going to learn. That's the I've been going to learn. That's the I've been going to learn. That's the I've been saying drive stick shift, and I get told saying drive stick shift, and I get told saying drive stick shift, and I get told that I'm being gatekeepy when I say that I'm being gatekeepy when I say that I'm being gatekeepy when I say that. that. that. >> I don't think you are cuz >> I don't think you are cuz >> I don't think you are cuz uh there was a great study a couple 3 uh there was a great study a couple 3 uh there was a great study a couple 3 months ago from MIT where months ago from MIT where months ago from MIT where they took um these these are not they took um these these are not they took um these these are not students, but they took people in students, but they took people in students, but they took people in careers already in the workforce, and careers already in the workforce, and careers already in the workforce, and they divided them into three cohorts, they divided them into three cohorts, they divided them into three cohorts, and had them write SAT uh and had them write SAT uh and had them write SAT uh essays from the SAT. essays from the SAT. essays from the SAT. And they had one cohort just do it And they had one cohort just do it And they had one cohort just do it with their own, you know, with their own, you know, with their own, you know, closed book, just write the essay. They closed book, just write the essay. They closed book, just write the essay. They had another cohort that got to use had another cohort that got to use had another cohort that got to use Google, and they had another cohort that Google, and they had another cohort that Google, and they had another cohort that got to use ChatGPT.

got to use ChatGPT. got to use ChatGPT. And they looked at their And they looked at their And they looked at their uh uh uh EEGs, and they EEGs, and they EEGs, and they quizzed them afterwards, quizzed them afterwards, quizzed them afterwards, right after, and then like a week later, right after, and then like a week later, right after, and then like a week later, and the results were exactly what you and the results were exactly what you and the results were exactly what you would expect. Mhm. The people that wrote would expect. Mhm. The people that wrote would expect. Mhm. The people that wrote it it it could answer questions about what they could answer questions about what they could answer questions about what they wrote wrote wrote even a week later, even a week later, even a week later, and their EEGs showed that they were, and their EEGs showed that they were, and their EEGs showed that they were, you know, burning a lot of wattage. you know, burning a lot of wattage. you know, burning a lot of wattage. Yeah. Yeah. Yeah. The people that were using ChatGPT, an The people that were using ChatGPT, an The people that were using ChatGPT, an hour after they wrote the essay, they hour after they wrote the essay, they hour after they wrote the essay, they couldn't remember what they'd written. couldn't remember what they'd written. couldn't remember what they'd written. That's the thing. It's just not even That's the thing. It's just not even That's the thing. It's just not even there. Yeah. That makes [clears throat] there. Yeah. That makes [clears throat] there. Yeah. That makes [clears throat] me really sad. Like me really sad. Like me really sad. Like to n- to n- to n- I I very much enjoy using AI to I I very much enjoy using AI to I I very much enjoy using AI to brainstorm, to plan, but then I want to brainstorm, to plan, but then I want to brainstorm, to plan, but then I want to do the writing part. To vibe your way do the writing part. To vibe your way do the writing part. To vibe your way through life I has me concerned. through life I has me concerned. through life I has me concerned. >> Well, it's just you know, loss of >> Well, it's just you know, loss of >> Well, it's just you know, loss of critical thinking, and they call this critical thinking, and they call this critical thinking, and they call this critical thinking deficit is what it's critical thinking deficit is what it's critical thinking deficit is what it's creating. Which we already have from creating. Which we already have from creating. Which we already have from social media. Yeah, we already have. So, social media. Yeah, we already have. So, social media. Yeah, we already have. So, and if you're talking about the early in and if you're talking about the early in and if you're talking about the early in career programmers that we've been career programmers that we've been career programmers that we've been talking about we're wanting to hire at a talking about we're wanting to hire at a talking about we're wanting to hire at a company, company, company, you want them to have to know what a you want them to have to know what a you want them to have to know what a race condition is. You don't want them race condition is. You don't want them race condition is. You don't want them to have vibed it in the AI's like, to have vibed it in the AI's like, to have vibed it in the AI's like, "Yeah, a race condition, AI will fix "Yeah, a race condition, AI will fix "Yeah, a race condition, AI will fix that."

that." that." Um because [music] Um because [music] Um because [music] at some point, as we've said, I think at some point, as we've said, I think at some point, as we've said, I think with the limitations of AI in software with the limitations of AI in software with the limitations of AI in software programming, at least for the programming, at least for the programming, at least for the foreseeable future, foreseeable future, foreseeable future, somebody needs to know. Yeah. somebody needs to know. Yeah. somebody needs to know. Yeah. Well, we are interested in what you Well, we are interested in what you Well, we are interested in what you think. Please uh comments are think. Please uh comments are think. Please uh comments are appreciated. We read every single appreciated. We read every single appreciated. We read every single comment, [music] and if you think this comment, [music] and if you think this comment, [music] and if you think this is an interesting conversation, uh is an interesting conversation, uh is an interesting conversation, uh certainly we'll share the paper as it certainly we'll share the paper as it certainly we'll share the paper as it gets makes its way out into the world, gets makes its way out into the world, gets makes its way out into the world, and we'll we'll shopping it around, but and we'll we'll shopping it around, but and we'll we'll shopping it around, but we're interested in your thoughts, and we're interested in your thoughts, and we're interested in your thoughts, and uh hopefully you'll subscribe, and uh hopefully you'll subscribe, and uh hopefully you'll subscribe, and you'll share this podcast with your you'll share this podcast with your you'll share this podcast with your friends and colleagues. friends and colleagues. friends and colleagues. As always, thanks for uh hanging out, As always, thanks for uh hanging out, As always, thanks for uh hanging out, Mark. Yeah, thanks, Scott.