Why Tori Westerhoff says we should talk to strangers

There's so much rich neuroscience There's so much rich neuroscience research around how humans interact in research around how humans interact in research around how humans interact in communities that that it is just a kind communities that that it is just a kind communities that that it is just a kind of an under client of how most people of an under client of how most people of an under client of how most people think and behave. It's funny that you think and behave. It's funny that you think and behave. It's funny that you mentioned the strangers thing cuz I mentioned the strangers thing cuz I mentioned the strangers thing cuz I actually have I have a reminder on my actually have I have a reminder on my actually have I have a reminder on my phone to talk to strangers phone to talk to strangers phone to talk to strangers every day noon. I've had every day noon. I've had every day noon. I've had >> That's so good. >> That's so good. >> That's so good. >> Now for so long and like my theory on >> Now for so long and like my theory on >> Now for so long and like my theory on this and I actually think it probably is this and I actually think it probably is this and I actually think it probably is really closely to what we're talking really closely to what we're talking really closely to what we're talking about and the introspection tool of an about and the introspection tool of an about and the introspection tool of an LLM is that you need diversity of LLM is that you need diversity of LLM is that you need diversity of thought. thought. thought. If you sycophant yourself, If you sycophant yourself, If you sycophant yourself, you're never going to grow. You're never you're never going to grow. You're never you're never going to grow. You're never going to find the new solution on the going to find the new solution on the going to find the new solution on the things that you're stuck on. things that you're stuck on. things that you're stuck on. And you also are going to be biased with And you also are going to be biased with And you also are going to be biased with in-group out-group like all these in-group out-group like all these in-group out-group like all these classic human things that you're not classic human things that you're not classic human things that you're not going to search for the really really going to search for the really really going to search for the really really different thinker unless you're being different thinker unless you're being different thinker unless you're being incredibly intentional. incredibly intentional. incredibly intentional. Like really specifically not looking for Like really specifically not looking for Like really specifically not looking for things that seem familiar and things that seem familiar and things that seem familiar and comfortable and good.

comfortable and good. comfortable and good. And talking to strangers is exactly And talking to strangers is exactly And talking to strangers is exactly that. Random people. that. Random people. that. Random people. >> Hey friends, you probably knew that Text >> Hey friends, you probably knew that Text >> Hey friends, you probably knew that Text Control is a powerful library for Control is a powerful library for Control is a powerful library for document editing and PDF generation, but document editing and PDF generation, but document editing and PDF generation, but did you also know that they're a strong did you also know that they're a strong did you also know that they're a strong supporter of the developer community and supporter of the developer community and supporter of the developer community and it's part of their mission to build and it's part of their mission to build and it's part of their mission to build and support a strong community by being support a strong community by being support a strong community by being present, by listening to users, and by present, by listening to users, and by present, by listening to users, and by sharing knowledge at conferences across sharing knowledge at conferences across sharing knowledge at conferences across Europe and the United States. If you're Europe and the United States. If you're Europe and the United States. If you're heading to a conference soon, heading to a conference soon, heading to a conference soon, maybe check if Text Control will be maybe check if Text Control will be maybe check if Text Control will be there. Stop by and say hi. You'll find there. Stop by and say hi. You'll find there. Stop by and say hi. You'll find their full conference calendar at their full conference calendar at their full conference calendar at textcontrol.com. textcontrol.com. textcontrol.com. That's t e x t control.com. >> Hi, I'm Scott Hanselman and this is >> Hi, I'm Scott Hanselman and this is another episode of Hanselminutes. Today another episode of Hanselminutes. Today another episode of Hanselminutes. Today I'm chatting with Tory Westerhoff. She's I'm chatting with Tory Westerhoff. She's I'm chatting with Tory Westerhoff. She's a principal AI security researcher with a principal AI security researcher with a principal AI security researcher with the red team at Microsoft. She's got a the red team at Microsoft. She's got a the red team at Microsoft. She's got a Wharton MBA and a background in Wharton MBA and a background in Wharton MBA and a background in neuroscience from Yale. How are you? neuroscience from Yale. How are you? neuroscience from Yale. How are you? >> I'm doing great. I'm stoked to chat. >> I'm doing great. I'm stoked to chat. >> I'm doing great. I'm stoked to chat. >> So we we got into it in a random meeting >> So we we got into it in a random meeting >> So we we got into it in a random meeting yesterday and it was super interesting yesterday and it was super interesting yesterday and it was super interesting and sometimes you have meetings at work and sometimes you have meetings at work and sometimes you have meetings at work and you're just like, "Oh, we got to and you're just like, "Oh, we got to and you're just like, "Oh, we got to keep this meeting going." But we had to keep this meeting going." But we had to keep this meeting going." But we had to hit record. Then you're talking about hit record. Then you're talking about hit record. Then you're talking about work stuff and so I said I want to have work stuff and so I said I want to have work stuff and so I said I want to have Tory on the on the podcast.

Tory on the on the podcast. Tory on the on the podcast. I I have not talked to a lot of I I have not talked to a lot of I I have not talked to a lot of neuroscientists right now, especially neuroscientists right now, especially neuroscientists right now, especially not ones whose job it is to be not ones whose job it is to be not ones whose job it is to be adversarial towards an AI. adversarial towards an AI. adversarial towards an AI. And And And there's a there's a there's a almost a pun there because everyone's almost a pun there because everyone's almost a pun there because everyone's adversarial towards AI. adversarial towards AI. adversarial towards AI. And how they feel about it and how And how they feel about it and how And how they feel about it and how they're dealing with the moment that they're dealing with the moment that they're dealing with the moment that we're in right now and we're in right now and we're in right now and I I feel some kind of way. Are you I I feel some kind of way. Are you I I feel some kind of way. Are you positive or negative or indifferent? positive or negative or indifferent? positive or negative or indifferent? >> Uh >> Uh >> Uh I think I'm strategic. I think I'm strategic. I think I'm strategic. I think I I think because a lot of what I think I I think because a lot of what I think I I think because a lot of what Microsoft AI red team does is to look at Microsoft AI red team does is to look at Microsoft AI red team does is to look at things things things from a misuse and also adversarial from a misuse and also adversarial from a misuse and also adversarial space. space. space. We don't just test like, "Hey, I am an We don't just test like, "Hey, I am an We don't just test like, "Hey, I am an adversary." We also test, "I'm a adversary." We also test, "I'm a adversary." We also test, "I'm a well-meaning person." Who would be using well-meaning person." Who would be using well-meaning person." Who would be using this in a way that veers this in a way that veers this in a way that veers harmful. harmful. harmful. So I think I'm really intentional about So I think I'm really intentional about So I think I'm really intentional about how I let AI impact my personal life and how I let AI impact my personal life and how I let AI impact my personal life and I try to like really I try to like really I try to like really fit the solution to the problem.

fit the solution to the problem. fit the solution to the problem. So it's it's not that I'm negative or So it's it's not that I'm negative or So it's it's not that I'm negative or positive, but I think I try to be really positive, but I think I try to be really positive, but I think I try to be really smart about the scope creep smart about the scope creep smart about the scope creep that AI can have on how you interact that AI can have on how you interact that AI can have on how you interact with the world, how you think about with the world, how you think about with the world, how you think about yourself, how you think about others. yourself, how you think about others. yourself, how you think about others. >> The part about how it how you think >> The part about how it how you think >> The part about how it how you think about yourself, I grew up in the '70s about yourself, I grew up in the '70s about yourself, I grew up in the '70s and '80s and they always and '80s and they always and '80s and they always in our kind of like in our kind of like in our kind of like middle school, elementary school middle school, elementary school middle school, elementary school psychology classes when they're kind of psychology classes when they're kind of psychology classes when they're kind of telling you how the brain works, they telling you how the brain works, they telling you how the brain works, they would say say like, "You've got these would say say like, "You've got these would say say like, "You've got these tapes that are running in your brain." tapes that are running in your brain." tapes that are running in your brain." Like they're tapes or background. We Like they're tapes or background. We Like they're tapes or background. We would imagine cassette tapes spinning in would imagine cassette tapes spinning in would imagine cassette tapes spinning in our brains and they would say, "Make our brains and they would say, "Make our brains and they would say, "Make sure that these tapes have good sure that these tapes have good sure that these tapes have good information that they're saying good information that they're saying good information that they're saying good things." It was one of those like, "Is things." It was one of those like, "Is things." It was one of those like, "Is your internal monologue negative towards your internal monologue negative towards your internal monologue negative towards yourself or positive to yourself?" yourself or positive to yourself?" yourself or positive to yourself?" And we kind of grew up in a time that And we kind of grew up in a time that And we kind of grew up in a time that said that introspection was a said that introspection was a said that introspection was a fundamental part of being a person who fundamental part of being a person who fundamental part of being a person who is a healthy and thoughtful person. And is a healthy and thoughtful person. And is a healthy and thoughtful person. And I kind of I'm always taking a pause and I kind of I'm always taking a pause and I kind of I'm always taking a pause and going, going, going, "How did I feel about how that meeting "How did I feel about how that meeting "How did I feel about how that meeting went? And how did I feel about that went? And how did I feel about that went? And how did I feel about that interaction with that person? And what interaction with that person? And what interaction with that person? And what can I do to to do better?"

can I do to to do better?" can I do to to do better?" And it may be generational, but I start And it may be generational, but I start And it may be generational, but I start to wonder to wonder to wonder if those tapes if those tapes if those tapes either don't exist or exist differently either don't exist or exist differently either don't exist or exist differently with the new generation as they may find with the new generation as they may find with the new generation as they may find them in infinite scrolling. them in infinite scrolling. them in infinite scrolling. What I have internalized, they have What I have internalized, they have What I have internalized, they have externalized. externalized. externalized. >> Right. I I do think that there's a lot >> Right. I I do think that there's a lot >> Right. I I do think that there's a lot of self-concept that can of self-concept that can of self-concept that can can start to can start to can start to bleed into how you're interacting with bleed into how you're interacting with bleed into how you're interacting with platforms, right? Especially at those platforms, right? Especially at those platforms, right? Especially at those super formative ages. And we've seen a super formative ages. And we've seen a super formative ages. And we've seen a lot of studies about how we know that lot of studies about how we know that lot of studies about how we know that doomscrolling can impact folks' ideas of doomscrolling can impact folks' ideas of doomscrolling can impact folks' ideas of themselves, right? Because it's all themselves, right? Because it's all themselves, right? Because it's all baselines. baselines. baselines. So, if you have a different perception So, if you have a different perception So, if you have a different perception of of of how the world is, how you perceive how the world is, how you perceive how the world is, how you perceive yourself in it is going to change. yourself in it is going to change. yourself in it is going to change. What I think is really interesting about What I think is really interesting about What I think is really interesting about AI is that it we're seeing in more and AI is that it we're seeing in more and AI is that it we're seeing in more and more published studies that people are more published studies that people are more published studies that people are using AI as the introspective tool. using AI as the introspective tool. using AI as the introspective tool. But now there is a system that's part of But now there is a system that's part of But now there is a system that's part of it instead of you having your it instead of you having your it instead of you having your conversation with yourself or someone in conversation with yourself or someone in conversation with yourself or someone in your life your life your life or pretending to have a conversation or pretending to have a conversation or pretending to have a conversation with role model, right? Like, "What with role model, right? Like, "What with role model, right? Like, "What would insert person do?"

would insert person do?" would insert person do?" You're interacting with an LLM You're interacting with an LLM You're interacting with an LLM that actually in a lot of cases that actually in a lot of cases that actually in a lot of cases is primed to have your information as is primed to have your information as is primed to have your information as context, right? So, it's not necessarily context, right? So, it's not necessarily context, right? So, it's not necessarily the worst introspection tool at all, but the worst introspection tool at all, but the worst introspection tool at all, but it is very different than us having that it is very different than us having that it is very different than us having that mental back and forth and assessment, mental back and forth and assessment, mental back and forth and assessment, I think. Because the assessment could I think. Because the assessment could I think. Because the assessment could potentially come from that LLM. potentially come from that LLM. potentially come from that LLM. >> Yeah. >> Yeah. >> Yeah. And and biases and context and And and biases and context and And and biases and context and randomization and all of those different randomization and all of those different randomization and all of those different things. things. things. >> Right. >> Right. >> Right. >> It That's interesting cuz I again, I'm >> It That's interesting cuz I again, I'm >> It That's interesting cuz I again, I'm I'm I'm going to use my generation at as I'm I'm going to use my generation at as I'm I'm going to use my generation at as a foil for the purposes of the a foil for the purposes of the a foil for the purposes of the conversation. I'm not fixated on it, but conversation. I'm not fixated on it, but conversation. I'm not fixated on it, but I'm using it for the for this this I'm using it for the for this this I'm using it for the for this this conversation. I have had moments where I conversation. I have had moments where I conversation. I have had moments where I would meet a stranger on a bus and have would meet a stranger on a bus and have would meet a stranger on a bus and have like a total random old person give you like a total random old person give you like a total random old person give you some thoughtful piece of advice, some thoughtful piece of advice, some thoughtful piece of advice, right? Some magical oracle that you bump right? Some magical oracle that you bump right? Some magical oracle that you bump into and then you don't know if they into and then you don't know if they into and then you don't know if they exist or not, but they told you some exist or not, but they told you some exist or not, but they told you some piece of advice. Or an auntie or, you piece of advice. Or an auntie or, you piece of advice. Or an auntie or, you know, a trea- a treasured elder in your know, a trea- a treasured elder in your know, a trea- a treasured elder in your life. And then I might talk to myself in life. And then I might talk to myself in life. And then I might talk to myself in the mirror or, you know, mumble to the mirror or, you know, mumble to the mirror or, you know, mumble to yourself on a walk.

yourself on a walk. yourself on a walk. And and I use the analogy that you and And and I use the analogy that you and And and I use the analogy that you and you're talking to an LLM, you are you're talking to an LLM, you are you're talking to an LLM, you are largely talking to yourself in the largely talking to yourself in the largely talking to yourself in the mirror, but you are also talking to some mirror, but you are also talking to some mirror, but you are also talking to some world computer that has the weights and world computer that has the weights and world computer that has the weights and balances and biases of whatever balances and biases of whatever balances and biases of whatever corporation decided to feed into that. corporation decided to feed into that. corporation decided to feed into that. Um I Um I Um I it makes me sad that my kids may not it makes me sad that my kids may not it makes me sad that my kids may not have that moment to talk to a random have that moment to talk to a random have that moment to talk to a random stranger on the bus and get advice stranger on the bus and get advice stranger on the bus and get advice because because because they might have a better advice than the they might have a better advice than the they might have a better advice than the LLM. LLM. LLM. But that might be age speaking. But that might be age speaking. But that might be age speaking. >> I I actually don't think it's age >> I I actually don't think it's age >> I I actually don't think it's age speaking and I think that all of the speaking and I think that all of the speaking and I think that all of the science says that we're community science says that we're community science says that we're community based creatures. based creatures. based creatures. >> Okay, that's not a theory. That's like a >> Okay, that's not a theory. That's like a >> Okay, that's not a theory. That's like a a pretty understood thing. a pretty understood thing. a pretty understood thing. >> Exactly. That's how evolution got us >> Exactly. That's how evolution got us >> Exactly. That's how evolution got us there. there. there. >> So, touch grass is not an insult. >> So, touch grass is not an insult. >> So, touch grass is not an insult. >> [laughter] >> [laughter] >> [laughter] >> I don't think so at all. Um and you also >> I don't think so at all. Um and you also >> I don't think so at all. Um and you also see see see there's so much rich neuroscience there's so much rich neuroscience there's so much rich neuroscience research around how humans interact in research around how humans interact in research around how humans interact in communities that that it is just a kind communities that that it is just a kind communities that that it is just a kind of an under current of how most people of an under current of how most people of an under current of how most people think and behave. It's funny that you think and behave. It's funny that you think and behave. It's funny that you mentioned the strangers thing. I mentioned the strangers thing. I mentioned the strangers thing. I actually have I have a reminder on my actually have I have a reminder on my actually have I have a reminder on my phone to talk to strangers every day phone to talk to strangers every day phone to talk to strangers every day noon.

noon. noon. >> Do you have that >> Do you have that >> Do you have that >> it That's so good. >> it That's so good. >> it That's so good. >> now for so long and like my theory on >> now for so long and like my theory on >> now for so long and like my theory on this and I actually think it it probably this and I actually think it it probably this and I actually think it it probably is really closely to what we're talking is really closely to what we're talking is really closely to what we're talking about in the introspection tool of an about in the introspection tool of an about in the introspection tool of an LLM is that you need diversity of LLM is that you need diversity of LLM is that you need diversity of thought. thought. thought. If you sycophant yourself, If you sycophant yourself, If you sycophant yourself, you're never going to grow. You're never you're never going to grow. You're never you're never going to grow. You're never going to find the new solution on the going to find the new solution on the going to find the new solution on the things that you're stuck on. things that you're stuck on. things that you're stuck on. And you also are going to be biased with And you also are going to be biased with And you also are going to be biased with in-group out-group like all these in-group out-group like all these in-group out-group like all these classic human things that you're not classic human things that you're not classic human things that you're not going to search for the really really going to search for the really really going to search for the really really different thinker unless you're being different thinker unless you're being different thinker unless you're being incredibly intentional. incredibly intentional. incredibly intentional. Like really specifically not looking for Like really specifically not looking for Like really specifically not looking for things that seem familiar and things that seem familiar and things that seem familiar and comfortable and good. comfortable and good. comfortable and good. And talking to strangers is exactly And talking to strangers is exactly And talking to strangers is exactly that. Random people on the bus, that. Random people on the bus, that. Random people on the bus, the only thing you have in common is the only thing you have in common is the only thing you have in common is theoretically the bus. theoretically the bus. theoretically the bus. Depending on the bus line, Depending on the bus line, Depending on the bus line, that's not a lot. And that actually that's not a lot. And that actually that's not a lot. And that actually disrupts your pattern of thinking. It disrupts your pattern of thinking. It disrupts your pattern of thinking. It gives you new information. It helps you gives you new information. It helps you gives you new information. It helps you update decision-making. It helps you update decision-making. It helps you update decision-making. It helps you update frames. That's kind of how update frames. That's kind of how update frames. That's kind of how concepts concepts concepts become new.

become new. become new. And when those concepts become new and And when those concepts become new and And when those concepts become new and you really think that they're you really think that they're you really think that they're impressionable, like that's actually how impressionable, like that's actually how impressionable, like that's actually how your decision-making starts to change. your decision-making starts to change. your decision-making starts to change. So in the context of that LLM where hey, So in the context of that LLM where hey, So in the context of that LLM where hey, maybe there is a bias towards a maybe there is a bias towards a maybe there is a bias towards a particular type of decision-making particular type of decision-making particular type of decision-making or a really particular maybe perhaps or a really particular maybe perhaps or a really particular maybe perhaps neutral frame on how people should be or neutral frame on how people should be or neutral frame on how people should be or how they should think about themselves how they should think about themselves how they should think about themselves or others, or others, or others, you're You're breaking out that one you're You're breaking out that one you're You're breaking out that one program, trained program, trained program, trained decision-making paradigm. And that's decision-making paradigm. And that's decision-making paradigm. And that's actually, I think, a really interesting actually, I think, a really interesting actually, I think, a really interesting element of element of element of growing up with an LLM in tow, or even growing up with an LLM in tow, or even growing up with an LLM in tow, or even even just being an adult with an LLM in even just being an adult with an LLM in even just being an adult with an LLM in tow, and replacing it with the tow, and replacing it with the tow, and replacing it with the randomness of a community. randomness of a community. randomness of a community. >> Yeah. >> Yeah. >> Yeah. I I I I have had people say that when I I I I I have had people say that when I I I I I have had people say that when I give my kind of like unk level advice, give my kind of like unk level advice, give my kind of like unk level advice, so I go out and talk to people and shake so I go out and talk to people and shake so I go out and talk to people and shake their hands, look them in the eye, and their hands, look them in the eye, and their hands, look them in the eye, and like, "Oh, you know, like, but my like, "Oh, you know, like, but my like, "Oh, you know, like, but my they'll say thing, 'Oh, my ADHD could they'll say thing, 'Oh, my ADHD could they'll say thing, 'Oh, my ADHD could never, or my social anxiety could never, or my social anxiety could never, or my social anxiety could never.'" And now I'm realizing that on never.'" And now I'm realizing that on never.'" And now I'm realizing that on on TikTok and on Instagram, and I'm very on TikTok and on Instagram, and I'm very on TikTok and on Instagram, and I'm very active on both, and I enjoy it very active on both, and I enjoy it very active on both, and I enjoy it very much, I have tuned them to be joyful for much, I have tuned them to be joyful for much, I have tuned them to be joyful for myself.

myself. myself. Um so, my algorithm is working. Um so, my algorithm is working. Um so, my algorithm is working. I I'm surprised that there seem to be I I'm surprised that there seem to be I I'm surprised that there seem to be two camps. There are those who doom two camps. There are those who doom two camps. There are those who doom scroll and say, "Oh, my anxiety could scroll and say, "Oh, my anxiety could scroll and say, "Oh, my anxiety could never." And then there are those for never." And then there are those for never." And then there are those for whom uh simply going out in the street whom uh simply going out in the street whom uh simply going out in the street and talking to strangers in a public and talking to strangers in a public and talking to strangers in a public square square square is a kind of content. Like, whether it's is a kind of content. Like, whether it's is a kind of content. Like, whether it's the guy who knows 20 languages and he the guy who knows 20 languages and he the guy who knows 20 languages and he wants to like meet random people and wants to like meet random people and wants to like meet random people and surprise them that he speaks his surprise them that he speaks his surprise them that he speaks his language, or it's like, "Here's how you language, or it's like, "Here's how you language, or it's like, "Here's how you rizz up, you know, the opposite sex." or rizz up, you know, the opposite sex." or rizz up, you know, the opposite sex." or whatever. Like, there's all these whatever. Like, there's all these whatever. Like, there's all these different people who like, "Oh my different people who like, "Oh my different people who like, "Oh my goodness, here's how you can be goodness, here's how you can be goodness, here's how you can be extroverted as well." And they seem to extroverted as well." And they seem to extroverted as well." And they seem to think that there's two states. There's think that there's two states. There's think that there's two states. There's either rotting either rotting either rotting or there's being an extrovert. And or there's being an extrovert. And or there's being an extrovert. And honestly, when I tell my kids or young honestly, when I tell my kids or young honestly, when I tell my kids or young people in my life, like, "You know, you people in my life, like, "You know, you people in my life, like, "You know, you can just talk to people at Walmart, it's can just talk to people at Walmart, it's can just talk to people at Walmart, it's not a problem." not a problem." not a problem." They're like, "No, no, my social They're like, "No, no, my social They're like, "No, no, my social battery. Oh, I can only do that once a battery. Oh, I can only do that once a battery. Oh, I can only do that once a week." There there's all these weird week." There there's all these weird week." There there's all these weird limitations that we seem to have put on limitations that we seem to have put on limitations that we seem to have put on ourselves. ourselves. ourselves. And we've given them language to say And we've given them language to say And we've given them language to say that like, "No, no, I'm broken. My brain that like, "No, no, I'm broken. My brain that like, "No, no, I'm broken. My brain doesn't work that way to be social." doesn't work that way to be social." doesn't work that way to be social." So, does the science support that, or So, does the science support that, or So, does the science support that, or did we break all these kids?

>> I think the simplest take on it is that >> I think the simplest take on it is that it's a muscle. it's a muscle. it's a muscle. And And And as a as an extroverted introvert, as a as an extroverted introvert, as a as an extroverted introvert, I really felt that, where I think if you I really felt that, where I think if you I really felt that, where I think if you if you met me pre-COVID, if you met me pre-COVID, if you met me pre-COVID, I was talking to strangers all the time. I was talking to strangers all the time. I was talking to strangers all the time. No need for an alarm for it. Hm. Right? No need for an alarm for it. Hm. Right? No need for an alarm for it. Hm. Right? I think I was I love people. The whole I think I was I love people. The whole I think I was I love people. The whole kind of premise of my kind of premise of my kind of premise of my obsession about life is that I just find obsession about life is that I just find obsession about life is that I just find humans fascinating, right? And any way humans fascinating, right? And any way humans fascinating, right? And any way I'm going to understand that, I'm going I'm going to understand that, I'm going I'm going to understand that, I'm going to go and grab. And I think that was an to go and grab. And I think that was an to go and grab. And I think that was an interesting experiment for me because I interesting experiment for me because I interesting experiment for me because I really really really was very comfortable being in like an was very comfortable being in like an was very comfortable being in like an introvert-like setting, introvert-like setting, introvert-like setting, and I lost the muscle and I felt it. and I lost the muscle and I felt it. and I lost the muscle and I felt it. >> Mhm. >> Mhm. >> Mhm. >> When I went back into the world, I >> When I went back into the world, I >> When I went back into the world, I really did feel really did feel really did feel it was more effortful. it was more effortful. it was more effortful. >> But you did something about that. You >> But you did something about that. You >> But you did something about that. You noticed it though, and that's if that noticed it though, and that's if that noticed it though, and that's if that feedback loop itself is is significant feedback loop itself is is significant feedback loop itself is is significant to call out. to call out. to call out. >> And I think I also had, going back to >> And I think I also had, going back to >> And I think I also had, going back to it's important to have experiences and it's important to have experiences and it's important to have experiences and almost like Bayesian processing of almost like Bayesian processing of almost like Bayesian processing of different social interactions, like I different social interactions, like I different social interactions, like I had a road map. I had done it before. I had a road map. I had done it before. I had a road map. I had done it before. I could go back to that. My brain knew it.

could go back to that. My brain knew it. could go back to that. My brain knew it. Uh it wasn't as good, Uh it wasn't as good, Uh it wasn't as good, and I had different feelings about it, and I had different feelings about it, and I had different feelings about it, but I had trained methods. but I had trained methods. but I had trained methods. If you don't train the methods, then If you don't train the methods, then If you don't train the methods, then you're really starting from you're really starting from you're really starting from your comfort level or your habit. your comfort level or your habit. your comfort level or your habit. >> Yeah. >> Yeah. >> Yeah. >> So I think I think that's what something >> So I think I think that's what something >> So I think I think that's what something too that in some instances, like too that in some instances, like too that in some instances, like practicing in a non-social setting, practicing in a non-social setting, practicing in a non-social setting, that's such a rich neurological that's such a rich neurological that's such a rich neurological stimuli setting. You're getting so many stimuli setting. You're getting so many stimuli setting. You're getting so many more cues interacting with a person than more cues interacting with a person than more cues interacting with a person than you are in an LLM. you are in an LLM. you are in an LLM. You're encoding so much more and really You're encoding so much more and really You're encoding so much more and really helping yourself learn in a different helping yourself learn in a different helping yourself learn in a different way than if you were to practice that or way than if you were to practice that or way than if you were to practice that or you were to say, "Hey LLM, how am I you were to say, "Hey LLM, how am I you were to say, "Hey LLM, how am I going to make friends? Like, how am I going to make friends? Like, how am I going to make friends? Like, how am I going to go talk to that guy in the going to go talk to that guy in the going to go talk to that guy in the square?" square?" square?" And and I think that's um part of And and I think that's um part of And and I think that's um part of learning learning learning that will be difficult to really that will be difficult to really that will be difficult to really intentionally intentionally intentionally intend intend intend >> While you were saying that, >> While you were saying that, >> While you were saying that, something popped in my head, which is something popped in my head, which is something popped in my head, which is you know, one of my superpowers is bad you know, one of my superpowers is bad you know, one of my superpowers is bad analogies. And there there's a machine analogies. And there there's a machine analogies. And there there's a machine at the gym called the Smith machine and at the gym called the Smith machine and at the gym called the Smith machine and it's basically squats on rails and the it's basically squats on rails and the it's basically squats on rails and the idea is that it keeps the squat you idea is that it keeps the squat you idea is that it keeps the squat you know, that everything's on rails and know, that everything's on rails and know, that everything's on rails and you're moving the weight in a single you're moving the weight in a single you're moving the weight in a single plane. And people will say, "Well, it's plane. And people will say, "Well, it's plane. And people will say, "Well, it's really great cuz you can target your really great cuz you can target your really great cuz you can target your quads." But then other people are like, quads." But then other people are like, quads." But then other people are like, "Well, no, it doesn't have all of the "Well, no, it doesn't have all of the "Well, no, it doesn't have all of the heaviness and the awkwardness of the heaviness and the awkwardness of the heaviness and the awkwardness of the weight and all the other little tiny weight and all the other little tiny weight and all the other little tiny muscles." So, when you were like muscles." So, when you were like muscles." So, when you were like saying you're talking to a human and all saying you're talking to a human and all saying you're talking to a human and all these additional pieces of input are

these additional pieces of input are these additional pieces of input are coming, it's like you're absolutely coming, it's like you're absolutely coming, it's like you're absolutely right. Microexpressions right. Microexpressions right. Microexpressions and every every all the six senses, you and every every all the six senses, you and every every all the six senses, you know, know, know, or a seven or eight depending on how or a seven or eight depending on how or a seven or eight depending on how many senses you decide to count we have. many senses you decide to count we have. many senses you decide to count we have. But like all the senses, while simply But like all the senses, while simply But like all the senses, while simply yapping in an LLM is, you know, barely yapping in an LLM is, you know, barely yapping in an LLM is, you know, barely one. one. one. That's I just clicked on me and it's That's I just clicked on me and it's That's I just clicked on me and it's just like, "Wow, do I want to hold the just like, "Wow, do I want to hold the just like, "Wow, do I want to hold the weight and all the little tiny muscles weight and all the little tiny muscles weight and all the little tiny muscles that make it under control? Or do I want that make it under control? Or do I want that make it under control? Or do I want to kind of artificially move the weight to kind of artificially move the weight to kind of artificially move the weight on rails?" And you're going to build a on rails?" And you're going to build a on rails?" And you're going to build a differently shaped differently shaped differently shaped persona or body persona or body persona or body uh if you if you think about it like uh if you if you think about it like uh if you if you think about it like that. that. that. >> I I love that analogy. >> I I love that analogy. >> I I love that analogy. Big analogy fan. Big analogy fan. Big analogy fan. I'm going to ride with it. I'm going to ride with it. I'm going to ride with it. The stability muscles that you get The stability muscles that you get The stability muscles that you get from from from the off-the-rails training, the off-the-rails training, the off-the-rails training, they actually get you different things they actually get you different things they actually get you different things in different scenarios. in different scenarios. in different scenarios. >> Yeah, yeah. Free weights. >> Yeah, yeah. Free weights. >> Yeah, yeah. Free weights. >> Now I'm in. That's also pretty key in >> Now I'm in. That's also pretty key in >> Now I'm in. That's also pretty key in how how how humans make connections across things.

humans make connections across things. humans make connections across things. You need to have cues, right? You need You need to have cues, right? You need You need to have cues, right? You need to have a signal in your brain to have a signal in your brain to have a signal in your brain to eventually move into an action or to eventually move into an action or to eventually move into an action or decision. decision. decision. Now, if the cue is just text Now, if the cue is just text Now, if the cue is just text or abstractly understanding a scenario, or abstractly understanding a scenario, or abstractly understanding a scenario, that could get you somewhere. that could get you somewhere. that could get you somewhere. But say it's a micro expression and But say it's a micro expression and But say it's a micro expression and you're picking it up not just in talking you're picking it up not just in talking you're picking it up not just in talking with the guy at the square, but with with the guy at the square, but with with the guy at the square, but with your parents, your parents, your parents, in an interview. in an interview. in an interview. And all of these are different scenarios And all of these are different scenarios And all of these are different scenarios that you didn't apply that that you didn't apply that that you didn't apply that maybe thought exercise of working with maybe thought exercise of working with maybe thought exercise of working with the LLM on. the LLM on. the LLM on. That's actually kind of how you get That's actually kind of how you get That's actually kind of how you get dynamic dynamic dynamic skills. skills. skills. So, I think I think that's kind of an So, I think I think that's kind of an So, I think I think that's kind of an important element to to take note of. important element to to take note of. important element to to take note of. Cuz you also need to sample like all of Cuz you also need to sample like all of Cuz you also need to sample like all of the diversity that you're going to the diversity that you're going to the diversity that you're going to interact with. You're not just going to interact with. You're not just going to interact with. You're not just going to be interact like different models. be interact like different models. be interact like different models. You're going to be interacting with You're going to be interacting with You're going to be interacting with different people. So, different people. So, different people. So, so that's a so that's a so that's a that's a great analogy in my opinion. that's a great analogy in my opinion. that's a great analogy in my opinion. >> I worry sometimes that social media has >> I worry sometimes that social media has >> I worry sometimes that social media has caused labels, unnecessary labels, to caused labels, unnecessary labels, to caused labels, unnecessary labels, to spread spread spread faster than necessarily they need to faster than necessarily they need to faster than necessarily they need to because Gen Z, Gen Alpha, whatever the because Gen Z, Gen Alpha, whatever the because Gen Z, Gen Alpha, whatever the current gen is, you know, they did they current gen is, you know, they did they current gen is, you know, they did they love a label because everyone wants to love a label because everyone wants to love a label because everyone wants to get a diagnosis and if you have a get a diagnosis and if you have a get a diagnosis and if you have a diagnosis, then you understand it then diagnosis, then you understand it then diagnosis, then you understand it then you're like, okay, I can do the thing.

you're like, okay, I can do the thing. you're like, okay, I can do the thing. That's how I used to think about it. That's how I used to think about it. That's how I used to think about it. Like you get a diagnostic, now I know Like you get a diagnostic, now I know Like you get a diagnostic, now I know what's wrong with me. But now we we stop what's wrong with me. But now we we stop what's wrong with me. But now we we stop at the I now I know what's wrong with at the I now I know what's wrong with at the I now I know what's wrong with me. We don't actually fix it. The reason me. We don't actually fix it. The reason me. We don't actually fix it. The reason that I'm saying that is that when that I'm saying that is that when that I'm saying that is that when someone says, well, I have this flavor someone says, well, I have this flavor someone says, well, I have this flavor of that thing and that's what prevents of that thing and that's what prevents of that thing and that's what prevents me from me from me from talking to people and not being awkward. talking to people and not being awkward. talking to people and not being awkward. And they're thinking it might be And they're thinking it might be And they're thinking it might be biological or serotonin or something in biological or serotonin or something in biological or serotonin or something in their brain. And it could be just that their brain. And it could be just that their brain. And it could be just that we broke them during COVID or we broke we broke them during COVID or we broke we broke them during COVID or we broke them during their formative years or the them during their formative years or the them during their formative years or the social media broke them. It makes me social media broke them. It makes me social media broke them. It makes me wonder if anyone's done any research wonder if anyone's done any research wonder if anyone's done any research about who is clinically about who is clinically about who is clinically got a problem that they need to work on got a problem that they need to work on got a problem that they need to work on versus who has simply adopted a label versus who has simply adopted a label versus who has simply adopted a label and made it a thing. It's kind of like and made it a thing. It's kind of like and made it a thing. It's kind of like when you think you're when you think you're when you think you're lactose intolerant and then you discover lactose intolerant and then you discover lactose intolerant and then you discover that your mom just didn't like milk and that your mom just didn't like milk and that your mom just didn't like milk and you were totally fine. You could have you were totally fine. You could have you were totally fine. You could have drank milk your whole life. drank milk your whole life. drank milk your whole life. I don't know, like whatever the lactose I don't know, like whatever the lactose I don't know, like whatever the lactose equivalent of Munchausen by proxy is. equivalent of Munchausen by proxy is. equivalent of Munchausen by proxy is. >> Right. I So, I think there is a lot of >> Right. I So, I think there is a lot of >> Right. I So, I think there is a lot of robust research to figure out what is robust research to figure out what is robust research to figure out what is going on neurologically when folks have going on neurologically when folks have going on neurologically when folks have this experience.

this experience. this experience. >> Mhm. >> Mhm. >> Mhm. >> Generally, my take on this >> Generally, my take on this >> Generally, my take on this is that brains are machines, too. is that brains are machines, too. is that brains are machines, too. So, if someone's feeling that way, it's So, if someone's feeling that way, it's So, if someone's feeling that way, it's likely the same exact undercurrent of a likely the same exact undercurrent of a likely the same exact undercurrent of a neurological signal or system. neurological signal or system. neurological signal or system. And so, I think that there's truth in And so, I think that there's truth in And so, I think that there's truth in that. that. that. What I think is a good thing and we What I think is a good thing and we What I think is a good thing and we actually think about this a ton in red actually think about this a ton in red actually think about this a ton in red teaming, which is kind of like a hard teaming, which is kind of like a hard teaming, which is kind of like a hard work, but work, but work, but a really good way I can contextualize a really good way I can contextualize a really good way I can contextualize the label and what that means is that the label and what that means is that the label and what that means is that every single one of those labels is a every single one of those labels is a every single one of those labels is a massive spectrum. massive spectrum. massive spectrum. And then it's overlaid on all of the And then it's overlaid on all of the And then it's overlaid on all of the contexts that someone's individual contexts that someone's individual contexts that someone's individual scenario is bringing. And that's scenario is bringing. And that's scenario is bringing. And that's actually what is making your brain fire. actually what is making your brain fire. actually what is making your brain fire. Not this label that has like a really Not this label that has like a really Not this label that has like a really specific specific specific suite of Google-able top five suite of Google-able top five suite of Google-able top five behaviors, for example. behaviors, for example. behaviors, for example. And it reminded me of red teaming And it reminded me of red teaming And it reminded me of red teaming because we're really pushing because we're really pushing because we're really pushing how we red team really complex how we red team really complex how we red team really complex socio-technical scenarios. And this is socio-technical scenarios. And this is socio-technical scenarios. And this is something we think about at work a lot, something we think about at work a lot, something we think about at work a lot, like how are we layering context so it's like how are we layering context so it's like how are we layering context so it's not actually just defining people or not actually just defining people or not actually just defining people or folks who could be interacting with AI folks who could be interacting with AI folks who could be interacting with AI with like the top three Google search with like the top three Google search with like the top three Google search results results results of something.

of something. of something. Because LLMs, humans, whatever scenario Because LLMs, humans, whatever scenario Because LLMs, humans, whatever scenario you're working with, you're working with, you're working with, everyone's going to be a really, really everyone's going to be a really, really everyone's going to be a really, really complex recipe complex recipe complex recipe of those ingredients. And it's going to of those ingredients. And it's going to of those ingredients. And it's going to taste different. It's going to feel taste different. It's going to feel taste different. It's going to feel different. different. different. So, my Yeah, that's I think So, my Yeah, that's I think So, my Yeah, that's I think I think it's more complex and probably I think it's more complex and probably I think it's more complex and probably quite quite quite truthful and honest in all of those truthful and honest in all of those truthful and honest in all of those experiences. experiences. experiences. But, there is great research about But, there is great research about But, there is great research about generally how this is happening to folks generally how this is happening to folks generally how this is happening to folks and like systematically what does it and like systematically what does it and like systematically what does it look like in the brain when we get these look like in the brain when we get these look like in the brain when we get these types of inputs. types of inputs. types of inputs. >> Yeah. >> Yeah. >> Yeah. Cuz I mean that you can tell that I'm Cuz I mean that you can tell that I'm Cuz I mean that you can tell that I'm trying hard to trying hard to trying hard to speak correctly and explicitly while speak correctly and explicitly while speak correctly and explicitly while offering an option that I'm wrong. offering an option that I'm wrong. offering an option that I'm wrong. And I want to be respectful of people And I want to be respectful of people And I want to be respectful of people whose lived experience is that XYZ is a whose lived experience is that XYZ is a whose lived experience is that XYZ is a thing that they have and it sucks cuz thing that they have and it sucks cuz thing that they have and it sucks cuz you know, like I'm a type 1 diabetic and you know, like I'm a type 1 diabetic and you know, like I'm a type 1 diabetic and I always hate it when people are like, I always hate it when people are like, I always hate it when people are like, "Oh, can you eat that? Or are you sure? "Oh, can you eat that? Or are you sure? "Oh, can you eat that? Or are you sure? Oh, my my friend cured that with Oh, my my friend cured that with Oh, my my friend cured that with cinnamon." Like there's always like, cinnamon." Like there's always like, cinnamon." Like there's always like, "Oh, I have a thing." Oh, well, that's "Oh, I have a thing." Oh, well, that's "Oh, I have a thing." Oh, well, that's not a really a thing. No one wants to not a really a thing. No one wants to not a really a thing. No one wants to feel like that. But, at the same time if feel like that. But, at the same time if feel like that. But, at the same time if I tell someone you should go out and I tell someone you should go out and I tell someone you should go out and touch grass and you should talk to touch grass and you should talk to touch grass and you should talk to people and you should put a reminder on people and you should put a reminder on people and you should put a reminder on your phone that says talk to strangers.

your phone that says talk to strangers. your phone that says talk to strangers. They might be able to say, "Well, I They might be able to say, "Well, I They might be able to say, "Well, I can't do that. You know, I can't talk to can't do that. You know, I can't talk to can't do that. You know, I can't talk to strangers. That's I'm prevented from strangers. That's I'm prevented from strangers. That's I'm prevented from doing that by some doing that by some doing that by some from some ism from some ism from some ism that I have. But, I still think it that I have. But, I still think it that I have. But, I still think it remains good advice if you believe in remains good advice if you believe in remains good advice if you believe in the root issue which is we are the root issue which is we are the root issue which is we are uh community-oriented social beings and uh community-oriented social beings and uh community-oriented social beings and as our optimal state is out in the in as our optimal state is out in the in as our optimal state is out in the in the market or in the town square and not the market or in the town square and not the market or in the town square and not in our beds with the pillow over our in our beds with the pillow over our in our beds with the pillow over our heads. heads. heads. And I think it actually goes back to And I think it actually goes back to And I think it actually goes back to that cassette idea. that cassette idea. that cassette idea. Right, the spirit of the cassette Right, the spirit of the cassette Right, the spirit of the cassette playing in your mind is playing in your mind is playing in your mind is whatever gets put in whatever gets put in whatever gets put in to your brain machine to your brain machine to your brain machine is what you're working with. That's how is what you're working with. That's how is what you're working with. That's how you're training it. you're training it. you're training it. Those are all the patterns Those are all the patterns Those are all the patterns that will result in decisions and that will result in decisions and that will result in decisions and actions and stuff that makes up your actions and stuff that makes up your actions and stuff that makes up your life. life. life. And And And there is a universe where you choose to there is a universe where you choose to there is a universe where you choose to say, "Okay, the cassette tape is say, "Okay, the cassette tape is say, "Okay, the cassette tape is long-form long-form long-form LMAO chatting." LMAO chatting." LMAO chatting." >> Uh-huh. >> Uh-huh. >> Uh-huh. >> But, if you zoom out from that, you >> But, if you zoom out from that, you >> But, if you zoom out from that, you would never want to listen to the same would never want to listen to the same would never want to listen to the same CD CD CD for your entire life.

for your entire life. for your entire life. Or cassette. Or cassette. Or cassette. Or MP3. Or whatever version of music Or MP3. Or whatever version of music Or MP3. Or whatever version of music listening we are listening we are listening we are >> if if if you were if well, you know, >> if if if you were if well, you know, >> if if if you were if well, you know, that you think about the research on that you think about the research on that you think about the research on dementia and like getting people to like dementia and like getting people to like dementia and like getting people to like play games and do the Wordle and try new play games and do the Wordle and try new play games and do the Wordle and try new things. Like things. Like things. Like time time time when you well, the days are long, but when you well, the days are long, but when you well, the days are long, but the years are short and you'll find when the years are short and you'll find when the years are short and you'll find when you get older that days fly by and you you get older that days fly by and you you get older that days fly by and you go and you think, "Why are this Why are go and you think, "Why are this Why are go and you think, "Why are this Why are these days flying by?" It's because of these days flying by?" It's because of these days flying by?" It's because of the same day. And the brain JPEGs them. the same day. And the brain JPEGs them. the same day. And the brain JPEGs them. It compresses them all into kind of just It compresses them all into kind of just It compresses them all into kind of just a randomly kind of mediocrely blue sky a randomly kind of mediocrely blue sky a randomly kind of mediocrely blue sky because like, "Yeah, nothing really because like, "Yeah, nothing really because like, "Yeah, nothing really happened this week." happened this week." happened this week." So, forcing yourself to do something and So, forcing yourself to do something and So, forcing yourself to do something and like un-JPEGGING your your life is an like un-JPEGGING your your life is an like un-JPEGGING your your life is an important part of important part of important part of being present. So, like my parents are being present. So, like my parents are being present. So, like my parents are in their 80s and I'm always trying to in their 80s and I'm always trying to in their 80s and I'm always trying to like like like get them out of there get them out of there get them out of there you know, maximize the time so that you know, maximize the time so that you know, maximize the time so that their brains are still operating as their brains are still operating as their brains are still operating as opposed to listening to opposed to listening to opposed to listening to Creedence Clearwater Revival for the Creedence Clearwater Revival for the Creedence Clearwater Revival for the same same same >> [laughter] >> [laughter] >> [laughter] >> Yeah, my dad I don't think the CD's left >> Yeah, my dad I don't think the CD's left >> Yeah, my dad I don't think the CD's left the car.

the car. the car. >> I actually think I have that on vinyl. >> I actually think I have that on vinyl. >> I actually think I have that on vinyl. Uh so, I would that's your choice. Uh so, I would that's your choice. Uh so, I would that's your choice. >> going to have if you're going to get >> going to have if you're going to get >> going to have if you're going to get stuck on one CD, I mean Creedence is stuck on one CD, I mean Creedence is stuck on one CD, I mean Creedence is definitely the move, but I'm still The definitely the move, but I'm still The definitely the move, but I'm still The point is try new things. point is try new things. point is try new things. >> Yeah, exactly. I and I think in the when >> Yeah, exactly. I and I think in the when >> Yeah, exactly. I and I think in the when it relates to AI, there are some it relates to AI, there are some it relates to AI, there are some there are some ways going back to one of there are some ways going back to one of there are some ways going back to one of your first questions like I said I use your first questions like I said I use your first questions like I said I use it strategically. it strategically. it strategically. >> Mhm. >> Mhm. >> Mhm. >> I think there is a skill to use >> I think there is a skill to use >> I think there is a skill to use technology as a tool technology as a tool technology as a tool that is probably going to need to be the that is probably going to need to be the that is probably going to need to be the solution to the Gen Alpha solution to the Gen Alpha solution to the Gen Alpha native AI user. native AI user. native AI user. And that's for getting things done, but And that's for getting things done, but And that's for getting things done, but it's also for individual relationships it's also for individual relationships it's also for individual relationships with AI and how it bleeds into your with AI and how it bleeds into your with AI and how it bleeds into your life. life. life. So, there's a universe if someone is in So, there's a universe if someone is in So, there's a universe if someone is in that spot where they're like, "I do not that spot where they're like, "I do not that spot where they're like, "I do not want to go to the square." want to go to the square." want to go to the square." Using LLMs as the tool that gets you to Using LLMs as the tool that gets you to Using LLMs as the tool that gets you to the lived experience you want.

the lived experience you want. the lived experience you want. That actually is a really That actually is a really That actually is a really now starting to be studied now starting to be studied now starting to be studied proactive use proactive use proactive use of an LLM. of an LLM. of an LLM. >> [snorts] >> [snorts] >> [snorts] >> But, it's intentional though. It's not >> But, it's intentional though. It's not >> But, it's intentional though. It's not It's not happening accidentally. Like if It's not happening accidentally. Like if It's not happening accidentally. Like if I like I got this little robot next to I like I got this little robot next to I like I got this little robot next to me and my mom was here earlier and she me and my mom was here earlier and she me and my mom was here earlier and she thinks the robot's adorable and it has thinks the robot's adorable and it has thinks the robot's adorable and it has expressions and stuff. And then there expressions and stuff. And then there expressions and stuff. And then there was a great article in the New York was a great article in the New York was a great article in the New York Times a couple of weeks ago about a Times a couple of weeks ago about a Times a couple of weeks ago about a shut-in older person who got one of the shut-in older person who got one of the shut-in older person who got one of the robots and you know, they're huge in robots and you know, they're huge in robots and you know, they're huge in Japan, the companion robots. And really Japan, the companion robots. And really Japan, the companion robots. And really it's just somebody to talk to cuz people it's just somebody to talk to cuz people it's just somebody to talk to cuz people are lonely. And like the loneliness are lonely. And like the loneliness are lonely. And like the loneliness epidemic's a thing. But, I feel like epidemic's a thing. But, I feel like epidemic's a thing. But, I feel like those are all like allowable, at least those are all like allowable, at least those are all like allowable, at least by my uh if I'm the if I'm the arbiter by my uh if I'm the if I'm the arbiter by my uh if I'm the if I'm the arbiter of this thing, because they're of this thing, because they're of this thing, because they're intentional. What I worry about is intentional. What I worry about is intentional. What I worry about is unintended use. And I was on some call unintended use. And I was on some call unintended use. And I was on some call some third-party call a couple of days some third-party call a couple of days some third-party call a couple of days ago and someone was saying that they ago and someone was saying that they ago and someone was saying that they were looking forward to LLMs being like were looking forward to LLMs being like were looking forward to LLMs being like the movie Her. the movie Her. the movie Her. And I like winced cuz I have no poker And I like winced cuz I have no poker And I like winced cuz I have no poker face on a Teams call. And I was just face on a Teams call. And I was just face on a Teams call. And I was just like, like, like, "That's not what we want at all." Like "That's not what we want at all." Like "That's not what we want at all." Like and I was wondering to myself, is this and I was wondering to myself, is this and I was wondering to myself, is this person intentionally looking for an AI person intentionally looking for an AI person intentionally looking for an AI girlfriend or was that an accident and girlfriend or was that an accident and girlfriend or was that an accident and they fell into that? And cuz I would they fell into that? And cuz I would they fell into that? And cuz I would think that would be not an optimal think that would be not an optimal think that would be not an optimal scenario.

scenario. scenario. >> Yeah. >> Yeah. >> Yeah. The The psychosocial impacts of AI are The The psychosocial impacts of AI are The The psychosocial impacts of AI are so interesting and our and our team so interesting and our and our team so interesting and our and our team actually really does look into what are actually really does look into what are actually really does look into what are the impacts of that. the impacts of that. the impacts of that. >> Mhm. >> Mhm. >> Mhm. >> in our work, actually. The The >> in our work, actually. The The >> in our work, actually. The The interesting thing on "Hey, I need this interesting thing on "Hey, I need this interesting thing on "Hey, I need this or want this." relationship with LLMs or want this." relationship with LLMs or want this." relationship with LLMs >> Mhm. >> Mhm. >> Mhm. >> is that I do feel like going back to >> is that I do feel like going back to >> is that I do feel like going back to that everyone's everyone's their own that everyone's everyone's their own that everyone's everyone's their own unique recipe. unique recipe. unique recipe. I do feel like that sometimes like can I do feel like that sometimes like can I do feel like that sometimes like can help people. help people. help people. But I think the awareness of how LLM's But I think the awareness of how LLM's But I think the awareness of how LLM's interact and replace or using the same interact and replace or using the same interact and replace or using the same exact neurological exact neurological exact neurological systems is maybe the part that I always systems is maybe the part that I always systems is maybe the part that I always ground myself in. ground myself in. ground myself in. So if you're getting to the place where So if you're getting to the place where So if you're getting to the place where it's activating that same idea of like, it's activating that same idea of like, it's activating that same idea of like, oh, this is my LLM best friend and this oh, this is my LLM best friend and this oh, this is my LLM best friend and this is also my best friend. I think having a is also my best friend. I think having a is also my best friend. I think having a really hard look on it really hard look on it really hard look on it >> Mhm. >> Mhm. >> Mhm. >> being able to peel yourself away and say >> being able to peel yourself away and say >> being able to peel yourself away and say is that actually how I want my brain to is that actually how I want my brain to is that actually how I want my brain to think of this think of this think of this is how I manage it.

is how I manage it. is how I manage it. >> Right. >> Right. >> Right. >> With building on the tools to do that, I >> With building on the tools to do that, I >> With building on the tools to do that, I do think is like going to be the do think is like going to be the do think is like going to be the frontier of how frontier of how frontier of how AI starts getting integrated in AI starts getting integrated in AI starts getting integrated in relationship building, coaching, relationship building, coaching, relationship building, coaching, education, education, education, folks whose brains are forming. folks whose brains are forming. folks whose brains are forming. >> Uh-huh. >> Uh-huh. >> Uh-huh. >> So that the awareness is not just coming >> So that the awareness is not just coming >> So that the awareness is not just coming from, hey, I like studied a lot of from, hey, I like studied a lot of from, hey, I like studied a lot of neuroscience so I think a lot about this neuroscience so I think a lot about this neuroscience so I think a lot about this constantly and instead comes from constantly and instead comes from constantly and instead comes from this is how you use the tool. This is this is how you use the tool. This is this is how you use the tool. This is how you should use it how you should use it how you should use it to get you to where you want to go. to get you to where you want to go. to get you to where you want to go. >> Yeah. >> Yeah. >> Yeah. The at the root of all of that is though The at the root of all of that is though The at the root of all of that is though an is intention and like I didn't let my an is intention and like I didn't let my an is intention and like I didn't let my kids have a phone until they were 13, kids have a phone until they were 13, kids have a phone until they were 13, you know, and maybe that should have you know, and maybe that should have you know, and maybe that should have been longer. Like that seems to be one been longer. Like that seems to be one been longer. Like that seems to be one of these generally set upon 13-ish is a of these generally set upon 13-ish is a of these generally set upon 13-ish is a good time to give a kid good time to give a kid good time to give a kid a phone and I think we've all, you know, a phone and I think we've all, you know, a phone and I think we've all, you know, given a two-year-old an iPad to shut given a two-year-old an iPad to shut given a two-year-old an iPad to shut them up and then we've also all been at them up and then we've also all been at them up and then we've also all been at Chili's and seen a two-year-old on an Chili's and seen a two-year-old on an Chili's and seen a two-year-old on an iPad and said, oh, look at those iPad and said, oh, look at those iPad and said, oh, look at those horrible people, they gave that kid an horrible people, they gave that kid an horrible people, they gave that kid an iPad to shut them up, you know. Uh iPad to shut them up, you know. Uh iPad to shut them up, you know. Uh there's like the the Overton window of there's like the the Overton window of there's like the the Overton window of like what is reasonable seems to be to like what is reasonable seems to be to like what is reasonable seems to be to be shifting and I'm trying not to be be shifting and I'm trying not to be be shifting and I'm trying not to be judgmental, but I want to judgmental, but I want to judgmental, but I want to give people the tools like you said so give people the tools like you said so give people the tools like you said so that they can be intentional that they can be intentional that they can be intentional and make the choices. But what I'm and make the choices. But what I'm and make the choices. But what I'm saying That requires cognitive saying That requires cognitive saying That requires cognitive That requires cognition.

That requires cognition. That requires cognition. And it feels like there's almost a And it feels like there's almost a And it feels like there's almost a a new generation of the subcognitive a new generation of the subcognitive a new generation of the subcognitive where people are simply reacting rather where people are simply reacting rather where people are simply reacting rather than than than being a member of like being at the tail being a member of like being at the tail being a member of like being at the tail of the teller of their own story. And of the teller of their own story. And of the teller of their own story. And there's there's there's you know, certainly capitalism and all you know, certainly capitalism and all you know, certainly capitalism and all the different things that cause the the different things that cause the the different things that cause the hamster wheel to have to spin and make hamster wheel to have to spin and make hamster wheel to have to spin and make that happen. But like maybe it's that that happen. But like maybe it's that that happen. But like maybe it's that the Maslow's hierarchy of needs doesn't the Maslow's hierarchy of needs doesn't the Maslow's hierarchy of needs doesn't let people have the space let people have the space let people have the space to talk about these things like we are. to talk about these things like we are. to talk about these things like we are. We are blessed to have the half an hour We are blessed to have the half an hour We are blessed to have the half an hour that we have that we have that we have to be able to do that. But if I was to be able to do that. But if I was to be able to do that. But if I was working two jobs, I might not have this working two jobs, I might not have this working two jobs, I might not have this luxury. luxury. luxury. >> I think that's so real. Yeah, I think >> I think that's so real. Yeah, I think >> I think that's so real. Yeah, I think about this a lot about this a lot about this a lot in in in how LLMs are used for decision-making. how LLMs are used for decision-making. how LLMs are used for decision-making. Again, that that's kind of where my Again, that that's kind of where my Again, that that's kind of where my research was and research was and research was and the instance that I see most is often the instance that I see most is often the instance that I see most is often that LLMs are just really, really that LLMs are just really, really that LLMs are just really, really prompted to give options to select. prompted to give options to select. prompted to give options to select. >> Oh, yeah. Well, multiple choice is >> Oh, yeah. Well, multiple choice is >> Oh, yeah. Well, multiple choice is always an easier test.

always an easier test. always an easier test. >> Exactly. And when you're busy and >> Exactly. And when you're busy and >> Exactly. And when you're busy and stressed and managing a ton of things, stressed and managing a ton of things, stressed and managing a ton of things, that's actually the most human instinct. that's actually the most human instinct. that's actually the most human instinct. Like Like Like give me something to pick from. give me something to pick from. give me something to pick from. Give me two to pick from. Give me two to pick from. Give me two to pick from. What it I think about a lot is that What it I think about a lot is that What it I think about a lot is that there's a ton of there's a ton of there's a ton of research around how we get decision research around how we get decision research around how we get decision fatigue throughout days. We get decision fatigue throughout days. We get decision fatigue throughout days. We get decision when we have a ton of choices and humans when we have a ton of choices and humans when we have a ton of choices and humans get bad at that. get bad at that. get bad at that. Say for example, you have a ton of Say for example, you have a ton of Say for example, you have a ton of options. options. options. And thinking about how LLMs just route And thinking about how LLMs just route And thinking about how LLMs just route very closely to hey, select instead of very closely to hey, select instead of very closely to hey, select instead of assess. You end up missing back to our assess. You end up missing back to our assess. You end up missing back to our conversation about talking on the square conversation about talking on the square conversation about talking on the square versus talking to an LLM. You miss a ton versus talking to an LLM. You miss a ton versus talking to an LLM. You miss a ton of information that would get you to of information that would get you to of information that would get you to your decision quicker with more your decision quicker with more your decision quicker with more assuredity if you thought through the assuredity if you thought through the assuredity if you thought through the assessment assessment assessment to get the options versus selecting to get the options versus selecting to get the options versus selecting blindly from options. blindly from options. blindly from options. Cuz it it helps you figure out how you Cuz it it helps you figure out how you Cuz it it helps you figure out how you how your values interact with that how your values interact with that how your values interact with that decision. And that's actually like a decision. And that's actually like a decision. And that's actually like a pretty well studied decision framework.

pretty well studied decision framework. pretty well studied decision framework. So maybe that's just one example of how So maybe that's just one example of how So maybe that's just one example of how I see I see I see these tools these tools these tools flooding our systems and interacting flooding our systems and interacting flooding our systems and interacting with us with us with us changing the muscles that we're working. changing the muscles that we're working. changing the muscles that we're working. >> Yeah. >> Yeah. >> Yeah. No, you No, you No, you I'm feeling better about my Smith I'm feeling better about my Smith I'm feeling better about my Smith machine analogy though because the machine analogy though because the machine analogy though because the stabilizer muscles like you you know, stabilizer muscles like you you know, stabilizer muscles like you you know, when you have no more reps in you, you when you have no more reps in you, you when you have no more reps in you, you can switch to a Smith Smith machine and can switch to a Smith Smith machine and can switch to a Smith Smith machine and you can probably get a couple more reps you can probably get a couple more reps you can probably get a couple more reps out. out. out. And it also makes me think about my And it also makes me think about my And it also makes me think about my buddy Alex Falcone who's a comedian buddy Alex Falcone who's a comedian buddy Alex Falcone who's a comedian has this great bit where he talks about has this great bit where he talks about has this great bit where he talks about cuz I've been married 25 years. This is cuz I've been married 25 years. This is cuz I've been married 25 years. This is your 26. your 26. your 26. Being married that long Being married that long Being married that long your entire relationship boils down to your entire relationship boils down to your entire relationship boils down to just what are we eating tonight? just what are we eating tonight? just what are we eating tonight? And like that like if you if you're in a And like that like if you if you're in a And like that like if you if you're in a relationship for long enough, like relationship for long enough, like relationship for long enough, like ultimately you know, I'm going to be ultimately you know, I'm going to be ultimately you know, I'm going to be married 50 years and I'm going to be married 50 years and I'm going to be married 50 years and I'm going to be like, "Oh, what are we going to eat?" I like, "Oh, what are we going to eat?" I like, "Oh, what are we going to eat?" I don't know. What do you What do you want don't know. What do you What do you want don't know. What do you What do you want to eat? I don't know. Like do you want to eat? I don't know. Like do you want to eat? I don't know. Like do you want to And I'll say to my wife, "Do you want to And I'll say to my wife, "Do you want to And I'll say to my wife, "Do you want to go to Noodles Noodles and Company?" to go to Noodles Noodles and Company?" to go to Noodles Noodles and Company?" And she's like, "No, I don't want this." And she's like, "No, I don't want this." And she's like, "No, I don't want this." The one thing that she gets at Noodles. The one thing that she gets at Noodles. The one thing that she gets at Noodles. And I'm like, "Well, there's like 50 And I'm like, "Well, there's like 50 And I'm like, "Well, there's like 50 things at Noodles there's like 10,000 things at Noodles there's like 10,000 things at Noodles there's like 10,000 different options." She's like, "No, no, different options." She's like, "No, no, different options." She's like, "No, no, no."

no." no." One restaurant one one choice and I'm One restaurant one one choice and I'm One restaurant one one choice and I'm realizing now that that's decision realizing now that that's decision realizing now that that's decision fatigue. It's just like I don't I don't fatigue. It's just like I don't I don't fatigue. It's just like I don't I don't know what I want to freaking eat. And know what I want to freaking eat. And know what I want to freaking eat. And I'll say, "What about Indian?" "No, not I'll say, "What about Indian?" "No, not I'll say, "What about Indian?" "No, not Indian." Indian." Indian." Well, then what do you want to eat? Well, then what do you want to eat? Well, then what do you want to eat? >> [laughter] >> [laughter] >> [laughter] >> You you have to have the spoons to make >> You you have to have the spoons to make >> You you have to have the spoons to make these decisions and when you don't have these decisions and when you don't have these decisions and when you don't have the spoons, then yeah, just ask the LLM. the spoons, then yeah, just ask the LLM. the spoons, then yeah, just ask the LLM. Spin the wheel. Spin the wheel. Spin the wheel. >> And I think that's a great >> And I think that's a great >> And I think that's a great that's a great example of how LLMs can that's a great example of how LLMs can that's a great example of how LLMs can help. help. help. Cuz you may only think of Indian and Cuz you may only think of Indian and Cuz you may only think of Indian and Noodles. Noodles. Noodles. LLMs are trained to think of more. LLMs are trained to think of more. LLMs are trained to think of more. So, that's like a positive introjection, So, that's like a positive introjection, So, that's like a positive introjection, I think, because it does get you out of I think, because it does get you out of I think, because it does get you out of potentially a bias of the initial set, potentially a bias of the initial set, potentially a bias of the initial set, maybe. maybe. maybe. But, it still doesn't get you the answer But, it still doesn't get you the answer But, it still doesn't get you the answer that you want, cuz you haven't said, that you want, cuz you haven't said, that you want, cuz you haven't said, "Okay, do I need something warm? Do I "Okay, do I need something warm? Do I "Okay, do I need something warm? Do I need comfort?" need comfort?" need comfort?" >> Yeah, there's no context. >> Yeah, there's no context. >> Yeah, there's no context. >> Vegetables? Like, the assessment part >> Vegetables? Like, the assessment part >> Vegetables? Like, the assessment part tends to be where we get the most tends to be where we get the most tends to be where we get the most insights A about ourselves, but B a insights A about ourselves, but B a insights A about ourselves, but B a better outcome.

better outcome. better outcome. And I I do think that's something to And I I do think that's something to And I I do think that's something to really pay attention to, and I think really pay attention to, and I think really pay attention to, and I think about it a lot in a red teaming and how about it a lot in a red teaming and how about it a lot in a red teaming and how I translate it, cuz I translate it, cuz I translate it, cuz >> Yeah. >> Yeah. >> Yeah. >> it's a >> it's a >> it's a it's a core difference from what I think it's a core difference from what I think it's a core difference from what I think my generation was, which was the big my generation was, which was the big my generation was, which was the big data generation obsessed with assessment data generation obsessed with assessment data generation obsessed with assessment and analysis. and analysis. and analysis. And now, that big data assessment lives And now, that big data assessment lives And now, that big data assessment lives in an LLM. in an LLM. in an LLM. >> Yeah. >> Yeah. >> Yeah. >> And you just get the exact output. >> And you just get the exact output. >> And you just get the exact output. Really, really, really different Really, really, really different Really, really, really different decision-making paradigm. decision-making paradigm. decision-making paradigm. >> Yeah, I could >> Yeah, I could >> Yeah, I could There's so many directions I we're we're There's so many directions I we're we're There's so many directions I we're we're we're ba- basically out of time, but we're ba- basically out of time, but we're ba- basically out of time, but there's so many directions I'd make it there's so many directions I'd make it there's so many directions I'd make it take this conversation, cuz now when you take this conversation, cuz now when you take this conversation, cuz now when you thought about the the assessment thought about the the assessment thought about the the assessment generation, my my cardiologist, cuz I'm generation, my my cardiologist, cuz I'm generation, my my cardiologist, cuz I'm at the age where I have to get my heart at the age where I have to get my heart at the age where I have to get my heart checked on a regular basis, says that he checked on a regular basis, says that he checked on a regular basis, says that he has an absolute hate relationship with has an absolute hate relationship with has an absolute hate relationship with the Apple Watch. And I said, "Why?" And the Apple Watch. And I said, "Why?" And the Apple Watch. And I said, "Why?" And he says, "Because it's a one-lead he says, "Because it's a one-lead he says, "Because it's a one-lead EKG EKG EKG or ECG. Uh it's one lead. It doesn't or ECG. Uh it's one lead. It doesn't or ECG. Uh it's one lead. It doesn't have like seven leads. It's it's it's on have like seven leads. It's it's it's on have like seven leads. It's it's it's on one part of your body, and it'll tell one part of your body, and it'll tell one part of your body, and it'll tell you like, 'Oh, that sounds like it could you like, 'Oh, that sounds like it could you like, 'Oh, that sounds like it could be wrong.' And yeah, like you know, you be wrong.' And yeah, like you know, you be wrong.' And yeah, like you know, you hear the story, it saved a guy's life, hear the story, it saved a guy's life, hear the story, it saved a guy's life, he was having A-fib, and he went in and he was having A-fib, and he went in and he was having A-fib, and he went in and he saved his life. But, he's like, for he saved his life. But, he's like, for he saved his life. But, he's like, for the every one guy that does that, there the every one guy that does that, there the every one guy that does that, there is a thousand that come in and they're is a thousand that come in and they're is a thousand that come in and they're like, 'My resting heart rate is too like, 'My resting heart rate is too like, 'My resting heart rate is too high. My Aura Ring said this and this high. My Aura Ring said this and this high. My Aura Ring said this and this and that and you know, and the other and that and you know, and the other and that and you know, and the other thing and it's just like, just listen to thing and it's just like, just listen to thing and it's just like, just listen to your body.'

your body.' your body.' So, I worry about the externalizing of So, I worry about the externalizing of So, I worry about the externalizing of signals signals signals that turn us from looking for validation that turn us from looking for validation that turn us from looking for validation from computers and machines and from computers and machines and from computers and machines and measurement systems that are imperfect measurement systems that are imperfect measurement systems that are imperfect when to your original point when we when to your original point when we when to your original point when we started this thing it's like well I started this thing it's like well I started this thing it's like well I thought to myself I don't talk to enough thought to myself I don't talk to enough thought to myself I don't talk to enough people so I made a posted note that says people so I made a posted note that says people so I made a posted note that says talk to more people. That's the input. talk to more people. That's the input. talk to more people. That's the input. That's the That's the tool. That's the That's the That's the tool. That's the That's the That's the tool. That's the feedback loop that I want is the one feedback loop that I want is the one feedback loop that I want is the one where I think about myself, think about where I think about myself, think about where I think about myself, think about what's going on in me, listen to the what's going on in me, listen to the what's going on in me, listen to the signals I have available, and make a signals I have available, and make a signals I have available, and make a decision. decision. decision. I don't know if I always need to go and I don't know if I always need to go and I don't know if I always need to go and validate that with Apple Health or validate that with Apple Health or validate that with Apple Health or ChatGPT. ChatGPT. ChatGPT. >> Yeah. I think that's really it. >> Yeah. I think that's really it. >> Yeah. I think that's really it. >> Mhm. >> Mhm. >> Mhm. >> And it was refined >> And it was refined >> And it was refined over years and years of having to do it over years and years of having to do it over years and years of having to do it without the tools. without the tools. without the tools. >> And there you There it is. There's the >> And there you There it is. There's the >> And there you There it is. There's the rub right there. rub right there. rub right there. So then So then So then here I am telling all the kids, "Well, here I am telling all the kids, "Well, here I am telling all the kids, "Well, you really need to drive stick shift." you really need to drive stick shift." you really need to drive stick shift." And they're like, "But Ubers are so fun. And they're like, "But Ubers are so fun. And they're like, "But Ubers are so fun. I love to Uber." I love to Uber." I love to Uber." Right? And that's That's the analogy Right? And that's That's the analogy Right? And that's That's the analogy that people who listen to this podcast that people who listen to this podcast that people who listen to this podcast are sick of me hearing is like there's are sick of me hearing is like there's are sick of me hearing is like there's old man who shakes fist at cloud is like old man who shakes fist at cloud is like old man who shakes fist at cloud is like do it the old-fashioned way.

do it the old-fashioned way. do it the old-fashioned way. Uh because if you don't then everyone's Uh because if you don't then everyone's Uh because if you don't then everyone's going to forget and then it's the end of going to forget and then it's the end of going to forget and then it's the end of Wall-E and we're all just running around Wall-E and we're all just running around Wall-E and we're all just running around uh getting tacos delivered directly into uh getting tacos delivered directly into uh getting tacos delivered directly into our mouths. our mouths. our mouths. And then somebody, you know, comes and And then somebody, you know, comes and And then somebody, you know, comes and moves my jaw for me cuz chewing is too moves my jaw for me cuz chewing is too moves my jaw for me cuz chewing is too hard. A sigh. A sigh. >> [laughter] >> [laughter] >> [laughter] >> Yeah. I I think it's >> Yeah. I I think it's >> Yeah. I I think it's it's an interesting place it's an interesting place it's an interesting place or inflection point or inflection point or inflection point to think through because I also I like to think through because I also I like to think through because I also I like fundamentally believe that tech and fundamentally believe that tech and fundamentally believe that tech and tools like this tools like this tools like this do help humans do help humans do help humans get to places that they want to do. Like get to places that they want to do. Like get to places that they want to do. Like humans are so good at tool use. That's humans are so good at tool use. That's humans are so good at tool use. That's another thing. Like we are community another thing. Like we are community another thing. Like we are community animals and we love a tool. But the tradeoffs maybe But the tradeoffs maybe I think aren't necessarily I think aren't necessarily I think aren't necessarily maybe highlighted in the way that tools maybe highlighted in the way that tools maybe highlighted in the way that tools used to be. used to be. used to be. >> [clears throat] >> [clears throat] >> [clears throat] >> So that I don't think that the feedback >> So that I don't think that the feedback >> So that I don't think that the feedback loop on what you're missing or how it's loop on what you're missing or how it's loop on what you're missing or how it's changing is as changing is as changing is as evident as we'd want it to be.

evident as we'd want it to be. evident as we'd want it to be. >> Mhm. >> Mhm. >> Mhm. >> And I also wonder >> And I also wonder >> And I also wonder the technology moves so fast and the the technology moves so fast and the the technology moves so fast and the ability to do these things is honestly ability to do these things is honestly ability to do these things is honestly quite new. quite new. quite new. Right? I don't think that the uses that Right? I don't think that the uses that Right? I don't think that the uses that we're talking about were relevant two we're talking about were relevant two we're talking about were relevant two years ago. years ago. years ago. >> Oh, they weren't relevant like last >> Oh, they weren't relevant like last >> Oh, they weren't relevant like last summer. summer. summer. >> Exactly. >> Exactly. >> Exactly. >> Yeah. >> Yeah. >> Yeah. >> So, >> So, >> So, tracking how folks are going to interact tracking how folks are going to interact tracking how folks are going to interact with it and what they replace with it and what they replace with it and what they replace is going to be a big part of the is going to be a big part of the is going to be a big part of the science. Understanding how the evolution science. Understanding how the evolution science. Understanding how the evolution of of of AI starts to almost like co-create AI starts to almost like co-create AI starts to almost like co-create the reality of folks who really invest the reality of folks who really invest the reality of folks who really invest in it. in it. in it. >> Mhm. >> Mhm. >> Mhm. >> Is that human >> Is that human >> Is that human computer interaction bit I think this is computer interaction bit I think this is computer interaction bit I think this is getting really really quite interesting getting really really quite interesting getting really really quite interesting now that capabilities are so advanced. now that capabilities are so advanced. now that capabilities are so advanced. And it becomes a social technical And it becomes a social technical And it becomes a social technical problem rather than just a society problem rather than just a society problem rather than just a society problem or just a technical problem problem or just a technical problem problem or just a technical problem because at this point because at this point because at this point the capabilities are are really the capabilities are are really the capabilities are are really meaningful.

meaningful. meaningful. And they're very good. And they're very good. And they're very good. >> [clears throat] >> [clears throat] >> [clears throat] >> Right? There we'd be having a different >> Right? There we'd be having a different >> Right? There we'd be having a different conversation if LLM chatbots gave conversation if LLM chatbots gave conversation if LLM chatbots gave horrible advice that didn't make any horrible advice that didn't make any horrible advice that didn't make any sense. Cuz people just wouldn't use it. sense. Cuz people just wouldn't use it. sense. Cuz people just wouldn't use it. Wouldn't be a good tool, right? So, that Wouldn't be a good tool, right? So, that Wouldn't be a good tool, right? So, that I that's an element that I that's an element that I that's an element that is going to be really interesting to is going to be really interesting to is going to be really interesting to watch. watch. watch. >> Yeah, and thing is that with the normal >> Yeah, and thing is that with the normal >> Yeah, and thing is that with the normal curve there is going to be bad advice. curve there is going to be bad advice. curve there is going to be bad advice. There has been a bad advice. There have There has been a bad advice. There have There has been a bad advice. There have been bad advice with bad outcomes. been bad advice with bad outcomes. been bad advice with bad outcomes. Have we decided that this power tool is Have we decided that this power tool is Have we decided that this power tool is is safe and should be unleashed? It's is safe and should be unleashed? It's is safe and should be unleashed? It's too late. It's happened. too late. It's happened. too late. It's happened. Um, but I'm glad that folks like you are Um, but I'm glad that folks like you are Um, but I'm glad that folks like you are studying it and I appreciate you taking studying it and I appreciate you taking studying it and I appreciate you taking the time to chat with me today. the time to chat with me today. the time to chat with me today. >> It was a joy. Thanks for chatting with >> It was a joy. Thanks for chatting with >> It was a joy. Thanks for chatting with me. me. me. >> We've been chatting with Tory >> We've been chatting with Tory >> We've been chatting with Tory Westerhoff. She's principal AI security Westerhoff. She's principal AI security Westerhoff. She's principal AI security researcher and a member of the AI red researcher and a member of the AI red researcher and a member of the AI red team at Microsoft. And this has been team at Microsoft. And this has been team at Microsoft. And this has been another episode of Hanselminutes and another episode of Hanselminutes and another episode of Hanselminutes and we'll see you again next week.