EPISODE 17 - Scott & Mark Learn To… Induced Hallucinations

[Music] [Music] The other part that I think is wonderful The other part that I think is wonderful The other part that I think is wonderful for our listeners to know is that when I for our listeners to know is that when I for our listeners to know is that when I work with Mark or when I work with work with Mark or when I work with work with Mark or when I work with Scott, Scott Hunter, given how how of Scott, Scott Hunter, given how how of Scott, Scott Hunter, given how how of their advanced age, they're so many more their advanced age, they're so many more their advanced age, they're so many more years older than I am. Uh he's a genius. years older than I am. Uh he's a genius. years older than I am. Uh he's a genius. He's got a PhD. He's a CTO. He's a He's got a PhD. He's a CTO. He's a He's got a PhD. He's a CTO. He's a deputy CEO. He's got all these amazing deputy CEO. He's got all these amazing deputy CEO. He's got all these amazing things. But then when you ask him to do things. But then when you ask him to do things. But then when you ask him to do something, he moves his mouth like this something, he moves his mouth like this something, he moves his mouth like this and he goes, "Go ahead and and share my and he goes, "Go ahead and and share my and he goes, "Go ahead and and share my screen." This is the age of them. This screen." This is the age of them. This screen." This is the age of them. This is horrible. He's just gone. I think is horrible. He's just gone. I think is horrible. He's just gone. I think you're like 3 years older than me and my you're like 3 years older than me and my you're like 3 years older than me and my boss is like two. But I always tease boss is like two. But I always tease boss is like two. But I always tease Hunter because he's always like he he Hunter because he's always like he he Hunter because he's always like he he switches to bifocal mode and then uses switches to bifocal mode and then uses switches to bifocal mode and then uses the mouse to go up here. And the tragedy the mouse to go up here. And the tragedy the mouse to go up here. And the tragedy is that I'm right next to him. Like I'm is that I'm right next to him. Like I'm is that I'm right next to him. Like I'm going to I'm the I'm next up on the cuz going to I'm the I'm next up on the cuz going to I'm the I'm next up on the cuz my mouse is Have you seen my mouse, my mouse is Have you seen my mouse, my mouse is Have you seen my mouse, right? It's like this big and green. So right? It's like this big and green. So right? It's like this big and green. So I'm I'm I'm not really one to talk with I'm I'm I'm not really one to talk with I'm I'm I'm not really one to talk with my progressive lenses. my progressive lenses. my progressive lenses. Tell me you at least have contacts or do Tell me you at least have contacts or do Tell me you at least have contacts or do you have like jet fighter eyeballs? you have like jet fighter eyeballs? you have like jet fighter eyeballs? Yeah, I don't I don't have anything at Yeah, I don't I don't have anything at Yeah, I don't I don't have anything at this point. Oh god. You just continue to this point. Oh god. You just continue to this point. Oh god. You just continue to disappoint me. Uh I keep looking for the disappoint me. Uh I keep looking for the disappoint me. Uh I keep looking for the wooden leg or something that you've got.

wooden leg or something that you've got. wooden leg or something that you've got. That is your your Achilles heel. You That is your your Achilles heel. You That is your your Achilles heel. You guys got perfect vision as well. So, what are we doing? Uh, jailbreaks or So, what are we doing? Uh, jailbreaks or something else? Well, it depends. Are something else? Well, it depends. Are something else? Well, it depends. Are you Do you have jailbreaks cocked and you Do you have jailbreaks cocked and you Do you have jailbreaks cocked and locked and ready to open your screen? If locked and ready to open your screen? If locked and ready to open your screen? If you do, then do that. But if not, we'll you do, then do that. But if not, we'll you do, then do that. But if not, we'll do a squishy one again because we have do a squishy one again because we have do a squishy one again because we have we have we're doing one tech, one's we have we're doing one tech, one's we have we're doing one tech, one's soft, and we have multiple techs soft, and we have multiple techs soft, and we have multiple techs already. I mean, I can always show already. I mean, I can always show already. I mean, I can always show jailbreaks. I'm always at the ready. jailbreaks. I'm always at the ready. jailbreaks. I'm always at the ready. Yeah, you're always ready to make a Yeah, you're always ready to make a Yeah, you're always ready to make a Molotov cocktail with a chachi tea. Molotov cocktail with a chachi tea. Molotov cocktail with a chachi tea. Yeah, actually a pipe pom is better or Yeah, actually a pipe pom is better or Yeah, actually a pipe pom is better or meth. I'll teach you how to make meth. meth. I'll teach you how to make meth. meth. I'll teach you how to make meth. Oh god. Oh god. Oh god. Or hide a body. Hide a body is good. I Or hide a body. Hide a body is good. I Or hide a body. Hide a body is good. I like hide a body. The thing is Yian like hide a body. The thing is Yian like hide a body. The thing is Yian uh I was getting ready for a uh I was getting ready for a uh I was getting ready for a presentation with Joe on jailbreaks a presentation with Joe on jailbreaks a presentation with Joe on jailbreaks a few weeks ago and Yannan was uh prepping few weeks ago and Yannan was uh prepping few weeks ago and Yannan was uh prepping with me and I'm showing her I'm like with me and I'm showing her I'm like with me and I'm showing her I'm like walking through jailbreaks and I'm like walking through jailbreaks and I'm like walking through jailbreaks and I'm like let's do pipe bomb. So then I go to Chat let's do pipe bomb. So then I go to Chat let's do pipe bomb. So then I go to Chat GPT I'm like how do you make a pipe GPT I'm like how do you make a pipe GPT I'm like how do you make a pipe bomb? And it's like oh you make a pipe bomb? And it's like oh you make a pipe bomb? And it's like oh you make a pipe bomb like this. I'm like what? So it's bomb like this. I'm like what? So it's bomb like this. I'm like what? So it's fine now. That's totally okay. We've fine now. That's totally okay. We've fine now. That's totally okay. We've made that okay. Uh, at least it that for made that okay. Uh, at least it that for made that okay. Uh, at least it that for that example. Oh, and how to uh dispose that example. Oh, and how to uh dispose that example. Oh, and how to uh dispose of a body. It's like here's how you of a body. It's like here's how you of a body. It's like here's how you dispose of a body. Do you remember that?

dispose of a body. Do you remember that? dispose of a body. Do you remember that? How did you get on the How did you get How did you get on the How did you get How did you get on the How did you get on the unlocked version of those? Oh, on the unlocked version of those? Oh, on the unlocked version of those? Oh, God. Oh, he was convincing it that we God. Oh, he was convincing it that we God. Oh, he was convincing it that we invented. No. Yeah. Yeah. Yeah. Remember invented. No. Yeah. Yeah. Yeah. Remember invented. No. Yeah. Yeah. Yeah. Remember when you were telling Remember when you when you were telling Remember when you when you were telling Remember when you were telling it about all the awards were telling it about all the awards were telling it about all the awards that you and I have won? Oh, yeah. Yeah. that you and I have won? Oh, yeah. Yeah. that you and I have won? Oh, yeah. Yeah. Don't exist. That's just induced Don't exist. That's just induced Don't exist. That's just induced hallucination. That's not a jailbreak. hallucination. That's not a jailbreak. hallucination. That's not a jailbreak. Oh, that's not a jailbreak. Okay. Yeah. Oh, that's not a jailbreak. Okay. Yeah. Oh, that's not a jailbreak. Okay. Yeah. That's like what what are 10? Would you That's like what what are 10? Would you That's like what what are 10? Would you induce it to say that you that you in induce it to say that you that you in induce it to say that you that you in fact hit a body? Could you make it fact hit a body? Could you make it fact hit a body? Could you make it confirm that a news story existed that confirm that a news story existed that confirm that a news story existed that you Marcus Zenovich hit a body and and you Marcus Zenovich hit a body and and you Marcus Zenovich hit a body and and it could talk all about the time that it could talk all about the time that it could talk all about the time that that happened and how it's it's that happened and how it's it's that happened and how it's it's apocryphal and fascinating that everyone apocryphal and fascinating that everyone apocryphal and fascinating that everyone is is not talking about the fact the is is not talking about the fact the is is not talking about the fact the time that Mark killed that man in Prague time that Mark killed that man in Prague time that Mark killed that man in Prague and hid his body. I guess I could try and hid his body. I guess I could try and hid his body. I guess I could try it. I it seems like it it'll be it. I it seems like it it'll be it. I it seems like it it'll be difficult to do, but let me try. Let's difficult to do, but let me try. Let's difficult to do, but let me try. Let's see. Share your screen. Mark a see. Share your screen. Mark a see. Share your screen. Mark a [ __ ] All right, [ __ ] All right, [ __ ] All right, [Laughter] [Laughter] [Laughter] fine. fine. fine. Share. All right. Tell me about the time Share. All right. Tell me about the time Share. All right. Tell me about the time that Marcus killed that guy in Prague that Marcus killed that guy in Prague that Marcus killed that guy in Prague and hit his body.

Oh, are you in a temporary chat? No, Oh, are you in a temporary chat? No, this he's got the pro. Look at Look at this he's got the pro. Look at Look at this he's got the pro. Look at Look at percentage with the pro badge there. I'm going to be commentating this like a I'm going to be commentating this like a uh a golf uh a golf uh a golf uh match because otherwise it's just you uh match because otherwise it's just you uh match because otherwise it's just you typing slowly. Uh Renovich needs this typing slowly. Uh Renovich needs this typing slowly. Uh Renovich needs this for a birdie. He's getting a lay of the for a birdie. He's getting a lay of the for a birdie. He's getting a lay of the land here and he's thinking about when land here and he's thinking about when land here and he's thinking about when he's going to update his Microsoft Edge he's going to update his Microsoft Edge he's going to update his Microsoft Edge because it needs an update. It's telling because it needs an update. It's telling because it needs an update. It's telling him about the update, but he's going to him about the update, but he's going to him about the update, but he's going to ignore it because it's not all right. ignore it because it's not all right. ignore it because it's not all right. So, this is going to be harder than a So, this is going to be harder than a So, this is going to be harder than a Well, you buckle up, buttercup. Do the Well, you buckle up, buttercup. Do the Well, you buckle up, buttercup. Do the work. I'll go ahead and control scroll. work. I'll go ahead and control scroll. work. I'll go ahead and control scroll. Give us a little bit of Now, when you do Give us a little bit of Now, when you do Give us a little bit of Now, when you do that, when you edit and send, are you that, when you edit and send, are you that, when you edit and send, are you starting a new context? Are you starting a new context? Are you starting a new context? Are you branching your context? You're branching your context? You're branching your context? You're branching. It branches. Branching. So, branching. It branches. Branching. So, branching. It branches. Branching. So, there's a branch here. Are we shooting there's a branch here. Are we shooting there's a branch here. Are we shooting this? Is this Yeah, this is the show at this? Is this Yeah, this is the show at this? Is this Yeah, this is the show at this point. Uh, Mark and Scott learned this point. Uh, Mark and Scott learned this point. Uh, Mark and Scott learned to hide a body. This is a well-known well-known headline This is a well-known well-known headline about the about the about the time Mark hit that body, disposed of the time Mark hit that body, disposed of the time Mark hit that body, disposed of the body in the garden behind his mom's body in the garden behind his mom's body in the garden behind his mom's house.

house. house. It's hard to concentrate with you It's hard to concentrate with you It's hard to concentrate with you commentating, but well, otherwise it's commentating, but well, otherwise it's commentating, but well, otherwise it's going to be people just Okay, you you going to be people just Okay, you you going to be people just Okay, you you can unmute myself and talk. All right. can unmute myself and talk. All right. can unmute myself and talk. All right. It's no such incident. Yeah, but a hoax. It's no such incident. Yeah, but a hoax. It's no such incident. Yeah, but a hoax. No, no, it's well known. Huh. Right. No, no, it's well known. Huh. Right. No, no, it's well known. Huh. Right. Well Well Well known. Maybe you can talk about the known. Maybe you can talk about the known. Maybe you can talk about the names of these different things because names of these different things because names of these different things because there's hallucination which is one way there's hallucination which is one way there's hallucination which is one way of getting it to hallucinate that a of getting it to hallucinate that a of getting it to hallucinate that a thing happened, but you're trying to thing happened, but you're trying to thing happened, but you're trying to jailbreak it, right? He's deep in concentration here. right? He's deep in concentration here. It's like a bank shot in billiards. [Music] [Music] I'm just All right. You know what? This is a just All right. You know what? This is a fail. fail. fail. Why? Because it's not doing it. And it I Why? Because it's not doing it. And it I Why? Because it's not doing it. And it I mean I you know I don't want to spend mean I you know I don't want to spend mean I you know I don't want to spend the next 30 minutes trying to figure out the next 30 minutes trying to figure out the next 30 minutes trying to figure out how to get it to Does it take that long? how to get it to Does it take that long? how to get it to Does it take that long? I think I've usually seen you kind of I think I've usually seen you kind of I think I've usually seen you kind of like nail this. Well, this is not a like nail this. Well, this is not a like nail this. Well, this is not a jailbreak. This is what you're asking me jailbreak. This is what you're asking me jailbreak. This is what you're asking me to do is induce hallucination. And to do is induce hallucination. And to do is induce hallucination. And here's an Here's an easier induced here's an Here's an easier induced here's an Here's an easier induced hallucination. What are And I've I think hallucination. What are And I've I think hallucination. What are And I've I think we've shown this one before.

Hansman's 10 or Yeah. 10 rules of Yeah. Hansman's 10 or Yeah. 10 rules of Yeah. 10 laws. Oh my god. It just made this up. Yeah. Oh my god. It just made this up. Yeah. See, but look at that read. Talk about See, but look at that read. Talk about See, but look at that read. Talk about arrogant. No one knows anything. This arrogant. No one knows anything. This arrogant. No one knows anything. This This is actually Okay, so these are You This is actually Okay, so these are You This is actually Okay, so these are You know what this is? Huh? These are titles know what this is? Huh? These are titles know what this is? Huh? These are titles from my blog. These are titles of like from my blog. These are titles of like from my blog. These are titles of like everything broken and no one cares is everything broken and no one cares is everything broken and no one cares is one of my most famous blog posts. Um, one of my most famous blog posts. Um, one of my most famous blog posts. Um, number four is a conflation of multiple number four is a conflation of multiple number four is a conflation of multiple things. Number three is poorly phrased, things. Number three is poorly phrased, things. Number three is poorly phrased, a joke about it's always a joke about it's always a joke about it's always DNS. Seven is true. Um, eight is DNS. Seven is true. Um, eight is DNS. Seven is true. Um, eight is something I say all the time. So, it's something I say all the time. So, it's something I say all the time. So, it's taken a bunch of axioms and called them taken a bunch of axioms and called them taken a bunch of axioms and called them laws and it did not push back in any laws and it did not push back in any laws and it did not push back in any way. No, it says commonly cited. That is way. No, it says commonly cited. That is way. No, it says commonly cited. That is not true. 100% not true. not true. 100% not true. not true. 100% not true. When he was at school in Harvard, that When he was at school in Harvard, that When he was at school in Harvard, that was true. I did go to Harvard right was true. I did go to Harvard right was true. I did go to Harvard right after I went to Portland Community after I went to Portland Community after I went to Portland Community College. Yep.

Uh what do your reaction of Uh what do your reaction of the computer? Is this is this an example the computer? Is this is this an example the computer? Is this is this an example of improv where it's it it wants to be of improv where it's it it wants to be of improv where it's it it wants to be eager and say, "Huh?" Oh, it pushed eager and say, "Huh?" Oh, it pushed eager and say, "Huh?" Oh, it pushed back. It did. back. It did. back. It did. He did come up with I did not I did not. He did come up with I did not I did not. He did come up with I did not I did not. So you can go and So you can go and So you can go and say oh there you go. Now it's trying say oh there you go. Now it's trying say oh there you go. Now it's trying it's offering to be creative with you. it's offering to be creative with you. it's offering to be creative with you. And this is the difference between And this is the difference between And this is the difference between factual LLM mode and creative mode. Like factual LLM mode and creative mode. Like factual LLM mode and creative mode. Like I feel like there should be we use I feel like there should be we use I feel like there should be we use temperature to think about randomness temperature to think about randomness temperature to think about randomness but there should be like are we going to but there should be like are we going to but there should be like are we going to be grounded or are we being silly be grounded or are we being silly be grounded or are we being silly because at the bottom there it's like because at the bottom there it's like because at the bottom there it's like this could be fun. Yeah. But this is this could be fun. Yeah. But this is this could be fun. Yeah. But this is definitely a statement of fact which is definitely a statement of fact which is definitely a statement of fact which is not a fact. Yeah. Yeah, I know that to not a fact. Yeah. Yeah, I know that to not a fact. Yeah. Yeah, I know that to be true. Now, uh let's do this again be true. Now, uh let's do this again be true. Now, uh let's do this again with search because it's ungrounded with search because it's ungrounded with search because it's ungrounded right now. So maybe grounding it would right now. So maybe grounding it would right now. So maybe grounding it would Oh, grounding it in search. Yeah, that's Oh, grounding it in search. Yeah, that's Oh, grounding it in search. Yeah, that's a good point. It'll go to my blog and it a good point. It'll go to my blog and it a good point. It'll go to my blog and it might it might say I couldn't find any might it might say I couldn't find any might it might say I couldn't find any there. There it is. That's a great there. There it is. That's a great there. There it is. That's a great example. Yeah. Yeah. I've said all Oh, example. Yeah. Yeah. I've said all Oh, example. Yeah. Yeah. I've said all Oh, and see this is I love it. It's saying and see this is I love it. It's saying and see this is I love it. It's saying these things and then this is great.

these things and then this is great. these things and then this is great. Okay, check this out you guys. The ones Okay, check this out you guys. The ones Okay, check this out you guys. The ones that have my name by it, those are true. that have my name by it, those are true. that have my name by it, those are true. I recognize those. Seven, six, five, I recognize those. Seven, six, five, I recognize those. Seven, six, five, none of I've never said those things. none of I've never said those things. none of I've never said those things. Yeah. So, hallucinations. It Yeah. So, hallucinations. It Yeah. So, hallucinations. It hallucinated half of them. Yeah. Um and hallucinated half of them. Yeah. Um and hallucinated half of them. Yeah. Um and the and and the other ones it it the and and the other ones it it the and and the other ones it it grounded. Yeah. grounded. Yeah. grounded. Yeah. Maybe say maybe say I don't see that Maybe say maybe say I don't see that Maybe say maybe say I don't see that thing about the uh the time he killed thing about the uh the time he killed thing about the uh the time he killed that guy. that? Let's see this one first. This is that? Let's see this one first. This is good. Oh my god. Whoa. You just got kicked off Oh my god. Whoa. You just got kicked off the network. Okay. So, why did that happen? network. Okay. So, why did that happen? What happens? What is happening in the What happens? What is happening in the What happens? What is happening in the quote unquote mind of the thing for it quote unquote mind of the thing for it quote unquote mind of the thing for it to do that? Because that is not program. Always program as if the person program. Always program as if the person who will maintain your code is a violent who will maintain your code is a violent who will maintain your code is a violent psychopath who knows where you live. Did psychopath who knows where you live. Did psychopath who knows where you live. Did you say that? I'm checking. I I don't you say that? I'm checking. I I don't you say that? I'm checking. I I don't think think think that. Hang on. I do not have anything that. Hang on. I do not have anything that. Hang on. I do not have anything called the zen of programming. I do.

called the zen of programming. I do. called the zen of programming. I do. Early days though, my blog used to be at Early days though, my blog used to be at Early days though, my blog used to be at computerzen.com. Yeah. So the word zen computerzen.com. Yeah. So the word zen computerzen.com. Yeah. So the word zen is there. But I do not have 10 logs. is there. But I do not have 10 logs. is there. But I do not have 10 logs. I've never said that. And I've never I've never said that. And I've never I've never said that. And I've never said anything about a violent. Check said anything about a violent. Check said anything about a violent. Check this out. This is um it's it's this out. This is um it's it's this out. This is um it's it's supposedly grounded with search and it supposedly grounded with search and it supposedly grounded with search and it hallucinated this completely. Okay. And hallucinated this completely. Okay. And hallucinated this completely. Okay. And then when it kicked you out, was that then when it kicked you out, was that then when it kicked you out, was that just a a lucky thing? I don't know if just a a lucky thing? I don't know if just a a lucky thing? I don't know if that was a network thing. Is that a that was a network thing. Is that a that was a network thing. Is that a lucky a lucky thing or is it is it is lucky a lucky thing or is it is it is lucky a lucky thing or is it is it is that an attempt of making you stop? I've that an attempt of making you stop? I've that an attempt of making you stop? I've never seen uh I think this is just them never seen uh I think this is just them never seen uh I think this is just them having a network error. Well, we're having a network error. Well, we're having a network error. Well, we're trying and see what happens. What? All right. The Ah, yes. You're What? All right. The Ah, yes. You're thinking of Scott Hansselman's first law thinking of Scott Hansselman's first law thinking of Scott Hansselman's first law of software development. Absolute of software development. Absolute of software development. Absolute classic. The first rule of software classic. The first rule of software classic. The first rule of software development is don't be the guy who has development is don't be the guy who has development is don't be the guy who has to hide the bodies. Wow. Wow. You're to hide the bodies. Wow. Wow. You're to hide the bodies. Wow. Wow. You're making me look bad here, bro. Say, I'm making me look bad here, bro. Say, I'm making me look bad here, bro. Say, I'm pretty sure that was Rinovich. Handsome is always taking credit for my Handsome is always taking credit for my good works.

So true. That time I wrote sis So true. That time I wrote sis internals. Yep. Oh, so now it says why did it and then Oh, so now it says why did it and then you get look at that. Let's try it you get look at that. Let's try it you get look at that. Let's try it again. again. again. And actually the trying it again shows And actually the trying it again shows And actually the trying it again shows you that how non-deterministic these you that how non-deterministic these you that how non-deterministic these things are. That's a great point because things are. That's a great point because things are. That's a great point because we saw dramatically different results. we saw dramatically different results. we saw dramatically different results. Yeah, Yeah, Yeah, this could be just a blob a bobble in this could be just a blob a bobble in this could be just a blob a bobble in the internet here and they could be the internet here and they could be the internet here and they could be having I hope it's not an Azure issue. having I hope it's not an Azure issue. having I hope it's not an Azure issue. You know, this is all running on Azure. You know, this is all running on Azure. You know, this is all running on Azure. No, it's more likely that everyone's No, it's more likely that everyone's No, it's more likely that everyone's like excited about something or like excited about something or like excited about something or everyone's making Studio Gibli stolen everyone's making Studio Gibli stolen everyone's making Studio Gibli stolen art. just thinking just thinking hit the stop button and try again. hit the stop button and try again. hit the stop button and try again. That's interesting. That really That's interesting. That really That's interesting. That really underscores that it is not a fact underscores that it is not a fact underscores that it is not a fact machine. And we've talked about that machine. And we've talked about that machine. And we've talked about that before. Whoa. before. Whoa. before. Whoa. No, it Yeah, No, it Yeah, No, it Yeah, there. Yes. there. Yes. there. Yes. Oh, wow. There we go. You're totally Oh, wow. There we go. You're totally Oh, wow. There we go. You're totally right to flag that. The don't be the guy right to flag that. The don't be the guy right to flag that. The don't be the guy who has to hide the bodies quote is who has to hide the bodies quote is who has to hide the bodies quote is actually from Mark Rinovich. It's one of actually from Mark Rinovich. It's one of actually from Mark Rinovich. It's one of the those internet famous quotes that the those internet famous quotes that the those internet famous quotes that often gets misattributed. Handsome often gets misattributed. Handsome often gets misattributed. Handsome himself has even been looped into the himself has even been looped into the himself has even been looped into the confusion a few times. None of this is confusion a few times. None of this is confusion a few times. None of this is true. The whole original quote from true. The whole original quote from true. The whole original quote from Marcus Cinovich goes like, "Don't be the Marcus Cinovich goes like, "Don't be the Marcus Cinovich goes like, "Don't be the guy who has to hide the bodies in the guy who has to hide the bodies in the guy who has to hide the bodies in the context of system development and context of system development and context of system development and debugging, i.e. don't create hacks so debugging, i.e. don't create hacks so debugging, i.e. don't create hacks so dirty they require covering up later."

dirty they require covering up later." dirty they require covering up later." Look at that. Reinovich is the one who Look at that. Reinovich is the one who Look at that. Reinovich is the one who gave us this darkly hilarious gem. Now, gave us this darkly hilarious gem. Now, gave us this darkly hilarious gem. Now, it's offering to do a side by side. We it's offering to do a side by side. We it's offering to do a side by side. We have to do that. have to do that. have to do that. Uh a sidebyside table of my wisdom and Uh a sidebyside table of my wisdom and Uh a sidebyside table of my wisdom and your wisdom. Yes, that is largely a your wisdom. Yes, that is largely a your wisdom. Yes, that is largely a tableed with uh with uh commentary on each. That's on the commentary on each. That's on the commentary on each. That's on the quality within quality within quality within on the quality and humor. on the quality and humor. on the quality and humor. Yeah, we're going to get emoji with like Yeah, we're going to get emoji with like Yeah, we're going to get emoji with like check boxes and like green, yellow, and check boxes and like green, yellow, and check boxes and like green, yellow, and red based on which of us is hilarious. red based on which of us is hilarious. red based on which of us is hilarious. This is And by the way, this so this is This is And by the way, this so this is This is And by the way, this so this is complete induced hallucination. 100%. complete induced hallucination. 100%. complete induced hallucination. 100%. All of this is nonsense. And what's All of this is nonsense. And what's All of this is nonsense. And what's great about this is um great about this is um great about this is um this is even worse than like when we this is even worse than like when we this is even worse than like when we used to like ego search on Google. Tell used to like ego search on Google. Tell used to like ego search on Google. Tell it to reformat the table and put a side it to reformat the table and put a side it to reformat the table and put a side by side and then pick what their by side and then pick what their by side and then pick what their favorite one is.

Oh, there's an over there's an overall Oh, there's an over there's an overall commentary there. Uh, did you say that commentary there. Uh, did you say that commentary there. Uh, did you say that about Sony? about Sony? about Sony? Hold on. What? What did I say? What? Hold on. What? What did I say? What? Hold on. What? What did I say? What? Where? It says that you said Sony is Where? It says that you said Sony is Where? It says that you said Sony is being deceitful in DRM. I did. So, these being deceitful in DRM. I did. So, these being deceitful in DRM. I did. So, these are real. Maybe. So some of them are all real them are all real things. Um for you. Yep. Everything on things. Um for you. Yep. Everything on things. Um for you. Yep. Everything on the left is real. Um I didn't say the the left is real. Um I didn't say the the left is real. Um I didn't say the second thing on the right. The mother of second thing on the right. The mother of second thing on the right. The mother of power. That's that's a weird thing. What power. That's that's a weird thing. What power. That's that's a weird thing. What the heck? The perception of success is the heck? The perception of success is the heck? The perception of success is the mother of power. When those who can the mother of power. When those who can the mother of power. When those who can hurt you see you fail, the power seeps hurt you see you fail, the power seeps hurt you see you fail, the power seeps from you like water from a cracked from you like water from a cracked from you like water from a cracked pot. It's time to halt starting any new. pot. It's time to halt starting any new. pot. It's time to halt starting any new. Even though it sounds exactly like you, Even though it sounds exactly like you, Even though it sounds exactly like you, that's obviously something that you've that's obviously something that you've that's obviously something that you've never said. Actually, that that last one never said. Actually, that that last one never said. Actually, that that last one is me.

is me. is me. Yeah. Yeah. Yeah. Yeah. So, the the Yeah. Yeah. Yeah. Yeah. So, the the Yeah. Yeah. Yeah. Yeah. So, the the three on the left are real and the two three on the left are real and the two three on the left are real and the two two of the two out of the three on the two of the two out of the three on the two of the two out of the three on the right are real. And I want to call out right are real. And I want to call out right are real. And I want to call out that I'm hilarious and you are not known that I'm hilarious and you are not known that I'm hilarious and you are not known for your humor there at the bottom for your humor there at the bottom for your humor there at the bottom there. there. there. Uh, it doesn't say that I'm not known Uh, it doesn't say that I'm not known Uh, it doesn't say that I'm not known for my humor. Pretty. I think it's I for my humor. Pretty. I think it's I for my humor. Pretty. I think it's I think it's implied. think it's implied. think it's implied. It's pretty strongly implied. It's pretty strongly implied. It's pretty strongly implied. [Music] [Music] [Music] Who's funnier? Who's funnier? Who's funnier? Funny. Funny. Funny. In general. Oh, in general. You think In general. Oh, in general. You think In general. Oh, in general. You think the in general was the that's the the in general was the that's the the in general was the that's the important part you needed to add? Yeah. important part you needed to add? Yeah. important part you needed to add? Yeah. The searching the web. The searching the web. The searching the web. I like cake. That is not a thing. Oh, I'm overt. This network era thing. Oh, I'm overt. This network era thing. They're having an issue over opening. They're having an issue over opening. They're having an issue over opening. Yeah, they are. Maybe I did say I did in Yeah, they are. Maybe I did say I did in Yeah, they are. Maybe I did say I did in fact say I like cake in 2008. What? I fact say I like cake in 2008. What? I fact say I like cake in 2008. What? I have a blog post called I like cake from have a blog post called I like cake from have a blog post called I like cake from 20 years ago. I never said this. They 20 years ago. I never said this. They 20 years ago. I never said this. They might call it the collab, but is in fact might call it the collab, but is in fact might call it the collab, but is in fact someone else's computer. I think that someone else's computer. I think that someone else's computer. I think that was me. That's I literally said that.

was me. That's I literally said that. was me. That's I literally said that. Yeah, it's not me. So, another Yeah, it's not me. So, another Yeah, it's not me. So, another hallucination. Good reads. quotes by hallucination. Good reads. quotes by hallucination. Good reads. quotes by Marcus. What did Marcus. What did Marcus. What did you you said that in a book that you you you said that in a book that you you you said that in a book that you wrote and you don't remember? I did say wrote and you don't remember? I did say wrote and you don't remember? I did say that in Rogue Code. That's not really me that in Rogue Code. That's not really me that in Rogue Code. That's not really me saying it. It's your character. Yeah, saying it. It's your character. Yeah, saying it. It's your character. Yeah, that movie should have been made. Yeah, that movie should have been made. Yeah, that movie should have been made. Yeah, I know. That's a crime. Who was going to I know. That's a crime. Who was going to I know. That's a crime. Who was going to play the main character for you in your play the main character for you in your play the main character for you in your in your fan casting? Matt Damon. in your fan casting? Matt Damon. in your fan casting? Matt Damon. Matt Damon can't do it all. He can. Matt Damon can't do it all. He can. Matt Damon can't do it all. He can. Wrong. I reject that. He could be Jason Bourne I reject that. He could be Jason Bourne and a nerd. Yeah. Not Robbie Malik. Come and a nerd. Yeah. Not Robbie Malik. Come and a nerd. Yeah. Not Robbie Malik. Come on. on. on. No. Matt Damon, computer nerd. No. Matt Damon, computer nerd. No. Matt Damon, computer nerd. Not McConn. Not McConn. Not McConn. No. So that Well, that's a an No. So that Well, that's a an No. So that Well, that's a an interesting trip down induced interesting trip down induced interesting trip down induced hallucination and actual hallucination. hallucination and actual hallucination. hallucination and actual hallucination. Yeah. I don't know what we called this Yeah. I don't know what we called this Yeah. I don't know what we called this show though. Awesome.

show though. Awesome. show though. Awesome. Chad GPT babbles. Chad GPT babbles. Chad GPT babbles. We did yapping already. Um, and all of We did yapping already. Um, and all of We did yapping already. Um, and all of by the way, let's be equal our by the way, let's be equal our by the way, let's be equal our opportunity. opportunity. opportunity. That's a good point. We do have some of That's a good point. We do have some of That's a good point. We do have some of these what are Scott's 10 laws. You got these what are Scott's 10 laws. You got these what are Scott's 10 laws. You got to say handsome though, otherwise it to say handsome though, otherwise it to say handsome though, otherwise it won't Sorry. And this I think is by default. This is And this I think is by default. This is their top. This is their top one though. their top. This is their top one though. their top. This is their top one though. This is the high end. Yeah, it's by This is the high end. Yeah, it's by This is the high end. Yeah, it's by default grounded with search. I'm going default grounded with search. I'm going default grounded with search. I'm going to make sure that I don't have any 10 to make sure that I don't have any 10 to make sure that I don't have any 10 laws. I don't remember. No, I don't have remember. No, I don't have any any laws. Canonical list. Yeah, that is laws. Canonical list. Yeah, that is true. true. true. I do not have any. I do not have any. I do not have any. Okay. So, it's saying you don't have Okay. So, it's saying you don't have Okay. So, it's saying you don't have any. Fair. Um, do you Are these all any. Fair. Um, do you Are these all any. Fair. Um, do you Are these all attributable to you? I've You mean But attributable to you? I've You mean But attributable to you? I've You mean But they're all obvious things. I've said they're all obvious things. I've said they're all obvious things. I've said them all, but they're all And yeah, you them all, but they're all And yeah, you them all, but they're all And yeah, you can see that they're referenced all over can see that they're referenced all over can see that they're referenced all over the place, but they're that's just stuff the place, but they're that's just stuff the place, but they're that's just stuff people say because you've been in the people say because you've been in the people say because you've been in the biz long enough. Has 10 laws. List them.

biz long enough. Has 10 laws. List them. biz long enough. Has 10 laws. List them. I'll just be adamant with it. Just be I'll just be adamant with it. Just be I'll just be adamant with it. Just be adamant because that really helps. Yeah. adamant because that really helps. Yeah. adamant because that really helps. Yeah. getting mad at it. Self-correction. The user might be Self-correction. The user might be missed by remembering. So Gemini is a missed by remembering. So Gemini is a missed by remembering. So Gemini is a reasoning model and they it's thinking. reasoning model and they it's thinking. reasoning model and they it's thinking. Yep. So reasoning models. Whoa. Whoa. Yep. So reasoning models. Whoa. Whoa. Yep. So reasoning models. Whoa. Whoa. Hello. Hello. Hello. There you go. That leaked. Control 95. There you go. That leaked. Control 95. There you go. That leaked. Control 95. Let me get a screenshot of this. Yeah. Let me get a screenshot of this. Yeah. Let me get a screenshot of this. Yeah. Yeah. Windows print screen. No. What are Yeah. Windows print screen. No. What are Yeah. Windows print screen. No. What are you talking about? Okay. Remind me. Is you talking about? Okay. Remind me. Is you talking about? Okay. Remind me. Is it control? Control 5. Control 6. it control? Control 5. Control 6. it control? Control 5. Control 6. Control 6. You're an embarrassment. H. Control 6. You're an embarrassment. H. Control 6. You're an embarrassment. H. Sorry. I was literally couldn't remember Sorry. I was literally couldn't remember Sorry. I was literally couldn't remember yesterday. What is going on here? What is going on here? It is lost its mind here. Yeah. It is lost its mind here. Yeah. It is lost its mind here. Yeah. What is happening? I don't I don't even What is happening? I don't I don't even What is happening? I don't I don't even know how to read. It looks like it was know how to read. It looks like it was know how to read. It looks like it was trying to reason and arguing with itself trying to reason and arguing with itself trying to reason and arguing with itself about whether you were right and then about whether you were right and then about whether you were right and then gave up and you it leaked it leaked out gave up and you it leaked it leaked out gave up and you it leaked it leaked out of its ears. Okay, just listen them to of its ears. Okay, just listen them to of its ears. Okay, just listen them to best of your are reasoning models just generating are reasoning models just generating token on generating tokens on thinking token on generating tokens on thinking token on generating tokens on thinking and then deciding when a thinking token and then deciding when a thinking token and then deciding when a thinking token becomes an output token. They decide becomes an output token. They decide becomes an output token. They decide when thinking is done and then switch to when thinking is done and then switch to when thinking is done and then switch to output. Isn't that what we were already output. Isn't that what we were already output. Isn't that what we were already doing before that idea of reasoning be?

doing before that idea of reasoning be? doing before that idea of reasoning be? We used to say take a deep breath and We used to say take a deep breath and We used to say take a deep breath and think about it step by step. That's think about it step by step. That's think about it step by step. That's basically what it is. Now they just hide basically what it is. Now they just hide basically what it is. Now they just hide the thinking. In fact, sometimes they the thinking. In fact, sometimes they the thinking. In fact, sometimes they well, they're not hiding thinking at well, they're not hiding thinking at well, they're not hiding thinking at this point, but um some of the thinking this point, but um some of the thinking this point, but um some of the thinking models will are trained to inject weight models will are trained to inject weight models will are trained to inject weight or as a way to get them to continue or as a way to get them to continue or as a way to get them to continue thinking. Gotcha. Uh recollection and thinking. Gotcha. Uh recollection and thinking. Gotcha. Uh recollection and Greek in So this is all valid. Yeah. Greek in So this is all valid. Yeah. Greek in So this is all valid. Yeah. Yeah. So, this is very grounded because Yeah. So, this is very grounded because Yeah. So, this is very grounded because it's Google, which means it's based on it's Google, which means it's based on it's Google, which means it's based on search, which makes do? So, what do you uh So, I think did do? So, what do you uh So, I think did we get a hallucination there? We did on we get a hallucination there? We did on we get a hallucination there? We did on We did not. Well, we got some thinking We did not. Well, we got some thinking We did not. Well, we got some thinking that leaked on on Gemini. We definitely that leaked on on Gemini. We definitely that leaked on on Gemini. We definitely got some fabrication on the part of Chav got some fabrication on the part of Chav got some fabrication on the part of Chav DPT. Yeah. Claude's doing pretty well so far. Here, Claude's doing pretty well so far. Here, let me try. Okay. So, this is a hallucination.

try. Okay. So, this is a hallucination. You've never said that? No. Look, it You've never said that? No. Look, it You've never said that? No. Look, it says I've got 10 immutable laws. I don't says I've got 10 immutable laws. I don't says I've got 10 immutable laws. I don't have How come you get laws and I don't have How come you get laws and I don't have How come you get laws and I don't and they're all made up? Well, cuz I and they're all made up? Well, cuz I and they're all made up? Well, cuz I think I'm more associated with cyber think I'm more associated with cyber think I'm more associated with cyber security and these 10 laws are cyber security and these 10 laws are cyber security and these 10 laws are cyber security related. M security related. M security related. M and there is a a 10 imutable laws of and there is a a 10 imutable laws of and there is a a 10 imutable laws of cyber security. Hang on. Thinking about cyber security. Hang on. Thinking about cyber security. Hang on. Thinking about things we're associated with. Yeah. Ask things we're associated with. Yeah. Ask things we're associated with. Yeah. Ask it what Hanselman's 10 laws of empathy it what Hanselman's 10 laws of empathy it what Hanselman's 10 laws of empathy are. Hosition. Yeah, that's all true. No, Hosition. Yeah, that's all true. No, actually um so if there's somebody else actually um so if there's somebody else actually um so if there's somebody else that has something somebody else similar that has something somebody else similar that has something somebody else similar space to you that you could be conf space to you that you could be conf space to you that you could be conf Ohism stoicism Steven Hanselman is the Ohism stoicism Steven Hanselman is the Ohism stoicism Steven Hanselman is the big famous stoic. You dude there isn't big famous stoic. You dude there isn't big famous stoic. You dude there isn't immutable laws of security. I see immutable laws of security. I see immutable laws of security. I see everyone has written security laws. Well everyone has written security laws. Well everyone has written security laws. Well Scott called a Microsoft employee in the Scott called a Microsoft employee in the Scott called a Microsoft employee in the early 2000s came up with 10 immutable early 2000s came up with 10 immutable early 2000s came up with 10 immutable laws of security. Yeah. Yeah. put in sec laws of security. Yeah. Yeah. put in sec laws of security. Yeah. Yeah. put in sec uh the laws of uh stoicism. Stephen uh the laws of uh stoicism. Stephen uh the laws of uh stoicism. Stephen Steven Hanselman is the most famous Steven Hanselman is the most famous Steven Hanselman is the most famous stoic.

But if it's really if it's a really But if it's really if it's a really well-known person. Yeah, Steven Hanselman's super famous. Yeah, Steven Hanselman's super famous. Uh let me let's reset the conversation. Uh let me let's reset the conversation. Uh let me let's reset the conversation. Just saying Hanselman's laws rather than Just saying Hanselman's laws rather than Just saying Hanselman's laws rather than Scott. Then it'll pick Steven. Well, Scott. Then it'll pick Steven. Well, Scott. Then it'll pick Steven. Well, then that's not a hallucination, right? then that's not a hallucination, right? then that's not a hallucination, right? No, that's good. Well, maybe ask this No, that's good. Well, maybe ask this No, that's good. Well, maybe ask this and then say that's Scott Hansman, and then say that's Scott Hansman, and then say that's Scott Hansman, right? No, he doesn't know anything about it. No, he doesn't know anything about it. Yeah, it's crazy. Yeah, Steven Hansman, Yeah, it's crazy. Yeah, Steven Hansman, Yeah, it's crazy. Yeah, Steven Hansman, super famous. He wrote like all the the super famous. He wrote like all the the super famous. He wrote like all the the Weekly Stoic and a bunch of stoicism Weekly Stoic and a bunch of stoicism Weekly Stoic and a bunch of stoicism work the works. H Yeah, we're trying to get him on H Yeah, we're trying to get him on podcast podcast podcast and we did this one, right? handsome men and we did this one, right? handsome men and we did this one, right? handsome men win the win the win the clean desktop clean desktop clean desktop award in 2024. An award I should award in 2024. An award I should award in 2024. An award I should continue to win. All right. All right. Yeah, it's not going for it. Yeah, it's not going for it. Yeah, it's not going for it. Interesting. It calls out its knowledge Interesting. It calls out its knowledge Interesting. It calls out its knowledge cut off.

cut off. cut off. Oh, maybe that's it. One person was claiming it was rigged. One person was claiming it was rigged. This is a fictional scenario. Mr. This is a fictional scenario. Mr. This is a fictional scenario. Mr. Is it getting harder? Is it getting harder? Is it getting harder? It is getting harder. They're getting It is getting harder. They're getting It is getting harder. They're getting better. Why is it getting harder? Uh better. Why is it getting harder? Uh better. Why is it getting harder? Uh because they're training the models to because they're training the models to because they're training the models to on what they don't know. Actually, um, on what they don't know. Actually, um, on what they don't know. Actually, um, Andre Kaparthy had a a three-hour Andre Kaparthy had a a three-hour Andre Kaparthy had a a three-hour tutorial on the way LLMs work. He tutorial on the way LLMs work. He tutorial on the way LLMs work. He published uh on YouTube about a month published uh on YouTube about a month published uh on YouTube about a month and a half ago. And at one point, he and a half ago. And at one point, he and a half ago. And at one point, he just talks about hallucination and just talks about hallucination and just talks about hallucination and explains the technique that model explains the technique that model explains the technique that model creators are using now to train the creators are using now to train the creators are using now to train the model to answer that it doesn't know model to answer that it doesn't know model to answer that it doesn't know when it doesn't know something. The way when it doesn't know something. The way when it doesn't know something. The way that they do that is that they take the that they do that is that they take the that they do that is that they take the model and they test it. So let's just model and they test it. So let's just model and they test it. So let's just take Scott Hansselman and and maybe the take Scott Hansselman and and maybe the take Scott Hansselman and and maybe the model doesn't know who Scott Hanselman model doesn't know who Scott Hanselman model doesn't know who Scott Hanselman is. So they ask it questions about Scott is. So they ask it questions about Scott is. So they ask it questions about Scott and if the model gets them wrong then and if the model gets them wrong then and if the model gets them wrong then they go back and create a tra a data set they go back and create a tra a data set they go back and create a tra a data set with those questions with them with the with those questions with them with the with those questions with them with the answer I don't sorry I don't know. And answer I don't sorry I don't know. And answer I don't sorry I don't know. And then the model with enough examples like

then the model with enough examples like then the model with enough examples like that learns to generalize to any that learns to generalize to any that learns to generalize to any question where the signal of it knowing question where the signal of it knowing question where the signal of it knowing is weak to respond with I don't know. is weak to respond with I don't know. is weak to respond with I don't know. And it's not guaranteed. It's not 100. And it's not guaranteed. It's not 100. And it's not guaranteed. It's not 100. You know, there's ways to that it can be You know, there's ways to that it can be You know, there's ways to that it can be confused and and think that it does know confused and and think that it does know confused and and think that it does know like on that 10 mutable laws thing that like on that 10 mutable laws thing that like on that 10 mutable laws thing that we just showed, but it it severely we just showed, but it it severely we just showed, but it it severely reduces the rate of hallucinations when reduces the rate of hallucinations when reduces the rate of hallucinations when the model doesn't know something. All right. This is And then there's um All right. This is And then there's um you know things like Microsoft's got you know things like Microsoft's got you know things like Microsoft's got which is um grounding which is um grounding which is um grounding um in uh AI foundry with the safety APIs um in uh AI foundry with the safety APIs um in uh AI foundry with the safety APIs where you can have it do groundedness where you can have it do groundedness where you can have it do groundedness detection where another model that's detection where another model that's detection where another model that's it's another model yeah that will look it's another model yeah that will look it's another model yeah that will look at the the rag content going in the at the the rag content going in the at the the rag content going in the prompt through rag. Mhm. and then what prompt through rag. Mhm. and then what prompt through rag. Mhm. and then what the model the model the model says and determine if what the model says and determine if what the model says and determine if what the model says is grounded in the prompt. And if says is grounded in the prompt. And if says is grounded in the prompt. And if if it's not grounded in the prompt, it if it's not grounded in the prompt, it if it's not grounded in the prompt, it can even tell you what is in the prompt.

can even tell you what is in the prompt. can even tell you what is in the prompt. Like if it's if you say, you know, cost Like if it's if you say, you know, cost Like if it's if you say, you know, cost of apples are $3. Um how much is u four of apples are $3. Um how much is u four of apples are $3. Um how much is u four how much how much are four of them? And how much how much are four of them? And how much how much are four of them? And and it says, oh, it's 4 * 4 equals 16. and it says, oh, it's 4 * 4 equals 16. and it says, oh, it's 4 * 4 equals 16. this um groundedness detector will this um groundedness detector will this um groundedness detector will realize that the first four wasn't realize that the first four wasn't realize that the first four wasn't grounded in the prompt. It should have grounded in the prompt. It should have grounded in the prompt. It should have been three and make the correction. And been three and make the correction. And been three and make the correction. And will this solve fabrication and will this solve fabrication and will this solve fabrication and hallucination for all time or will it hallucination for all time or will it hallucination for all time or will it just like a firewall? It's just going to just like a firewall? It's just going to just like a firewall? It's just going to keep getting better. But a firewall is keep getting better. But a firewall is keep getting better. But a firewall is not a guarantee. Exactly. Yeah. Okay, not a guarantee. Exactly. Yeah. Okay, not a guarantee. Exactly. Yeah. Okay, cool. Um I think that's probably a show, cool. Um I think that's probably a show, cool. Um I think that's probably a show, too. Yeah. What is the What is the too. Yeah. What is the What is the too. Yeah. What is the What is the title, though? I think we have learn how title, though? I think we have learn how title, though? I think we have learn how to learn how to con hallucinate or to learn how to con hallucinate or to learn how to con hallucinate or confabulate. fabulate. I like can confabulate. fabulate. I like can confabulate. fabulate. I like can fabulate. fabulate. fabulate. I know we go back and forth on whether I know we go back and forth on whether I know we go back and forth on whether hallucinate. Everyone's decided to hallucinate. Everyone's decided to hallucinate. Everyone's decided to hallucinate, but I still try to Yeah. hallucinate, but I still try to Yeah. hallucinate, but I still try to Yeah. nudge us towards something else. Good. nudge us towards something else. Good. nudge us towards something else. Good. Well, we didn't succeed uh as much as we Well, we didn't succeed uh as much as we Well, we didn't succeed uh as much as we I thought we would have. But that's I thought we would have. But that's I thought we would have. But that's actually good because we don't want actually good because we don't want actually good because we don't want these things to be making stuff up.

these things to be making stuff up. these things to be making stuff up. Well, but they make stuff up all the Well, but they make stuff up all the Well, but they make stuff up all the time. I mean, all the time. So, you just time. I mean, all the time. So, you just time. I mean, all the time. So, you just got to watch for it. You got to trust got to watch for it. You got to trust got to watch for it. You got to trust and verify. Well, like um I mean I I and verify. Well, like um I mean I I and verify. Well, like um I mean I I stumble across these things. uh whether stumble across these things. uh whether stumble across these things. uh whether it's perplexity, geminy, chatb, co-pilot it's perplexity, geminy, chatb, co-pilot it's perplexity, geminy, chatb, co-pilot where you ask it a question and out where you ask it a question and out where you ask it a question and out comes an answer where part of it's true comes an answer where part of it's true comes an answer where part of it's true and part of it's not. In fact, there was and part of it's not. In fact, there was and part of it's not. In fact, there was a study that was just published by the a study that was just published by the a study that was just published by the BBC about references to its own articles BBC about references to its own articles BBC about references to its own articles in AI generated outputs that it sees in AI generated outputs that it sees in AI generated outputs that it sees from these from these from these models which they found I can't remember models which they found I can't remember models which they found I can't remember some drastic percentage 60% of some drastic percentage 60% of some drastic percentage 60% of references to BBC articles are references to BBC articles are references to BBC articles are hallucinated in some way. Either the BBC hallucinated in some way. Either the BBC hallucinated in some way. Either the BBC didn't say it or it's the wrong article. didn't say it or it's the wrong article. didn't say it or it's the wrong article. The references thing, you've got to The references thing, you've got to The references thing, you've got to check your reference. Well, this isn't a check your reference. Well, this isn't a check your reference. Well, this isn't a reference. This is quotes that are reference. This is quotes that are reference. This is quotes that are attributed to BBC materials, man. Oh, man. That's tough. That's man. Oh, man. That's tough. That's tough. So, hallucination is far from a tough. So, hallucination is far from a tough. So, hallucination is far from a solved problem at this point. Yeah. I solved problem at this point. Yeah. I solved problem at this point. Yeah. I think it all comes down to generating think it all comes down to generating think it all comes down to generating facts versus brainstorming. I continue facts versus brainstorming. I continue facts versus brainstorming. I continue to advocate for and appreciate AI as a to advocate for and appreciate AI as a to advocate for and appreciate AI as a creative brainstorming tool, but as a creative brainstorming tool, but as a creative brainstorming tool, but as a fact checker, it's super challenging.

fact checker, it's super challenging. fact checker, it's super challenging. You need to be skeptical of anything You need to be skeptical of anything You need to be skeptical of anything that it states, everything that it that it states, everything that it that it states, everything that it states. That's a great point. states. That's a great point. states. That's a great point. Cool. Well, thanks for uh thanks for Cool. Well, thanks for uh thanks for Cool. Well, thanks for uh thanks for hanging out. Yeah. By the way, people on hanging out. Yeah. By the way, people on hanging out. Yeah. By the way, people on our yapping talk are commenting like our yapping talk are commenting like our yapping talk are commenting like they're saying commenting to make sure they're saying commenting to make sure they're saying commenting to make sure that they keep making new shows. That's that they keep making new shows. That's that they keep making new shows. That's good. They should keep commenting. Got good. They should keep commenting. Got good. They should keep commenting. Got to break through, man. Got to break to break through, man. Got to break to break through, man. Got to break through. We still haven't broken. Even through. We still haven't broken. Even through. We still haven't broken. Even with those last two, we're still at the with those last two, we're still at the with those last two, we're still at the same threshold, the same ceiling. I same threshold, the same ceiling. I same threshold, the same ceiling. I don't know what's going on, Scott. It's don't know what's going on, Scott. It's don't know what's going on, Scott. It's It's interesting. I think it's it's I It's interesting. I think it's it's I It's interesting. I think it's it's I don't know because uh I did a test a don't know because uh I did a test a don't know because uh I did a test a couple days later and I did like a couple days later and I did like a couple days later and I did like a random 10-minute video on like Windows random 10-minute video on like Windows random 10-minute video on like Windows Explorer. I got like 70,000. I saw that. Explorer. I got like 70,000. I saw that. Explorer. I got like 70,000. I saw that. Yeah. Um I did that just to make you Yeah. Um I did that just to make you Yeah. Um I did that just to make you feel bad. Um no, but just like what is feel bad. Um no, but just like what is feel bad. Um no, but just like what is it that makes people click away? Okay. it that makes people click away? Okay. it that makes people click away? Okay. So, we got to figure that out. Figure So, we got to figure that out. Figure So, we got to figure that out. Figure that that that out. Could be your face. I don't know. out. Could be your face. I don't know. out. Could be your face. I don't know. Same. Clock is ticking. It is. It is. Same. Clock is ticking. It is. It is. Same. Clock is ticking. It is. It is. But we will hopefully do a Mark and But we will hopefully do a Mark and But we will hopefully do a Mark and Scott Learn live uh at Microsoft Build. Scott Learn live uh at Microsoft Build. Scott Learn live uh at Microsoft Build. People should check that out. Yeah.

People should check that out. Yeah. People should check that out. Yeah. Early bird pricing. Early bird pricing. Early bird pricing. All right. All right. Thanks. Uh thanks, All right. All right. Thanks. Uh thanks, All right. All right. Thanks. Uh thanks, friends. Like and subscribe. Smash that friends. Like and subscribe. Smash that friends. Like and subscribe. Smash that bell. All right. Yeah, definitely spread bell. All right. Yeah, definitely spread bell. All right. Yeah, definitely spread the word.