What is AI's UI? with Maggie Appleton

hi I'm Scott Hansel and this is another hi I'm Scott Hansel and this is another episode of Hansel minutes today I'm episode of Hansel minutes today I'm episode of Hansel minutes today I'm chatting with Maggie Appleton she sits chatting with Maggie Appleton she sits chatting with Maggie Appleton she sits at the intersection of design at the intersection of design at the intersection of design anthropology and programming which is anthropology and programming which is anthropology and programming which is three things that I think are fantastic three things that I think are fantastic three things that I think are fantastic and she's the lead design engineer at and she's the lead design engineer at and she's the lead design engineer at normal a research and Design Studio how normal a research and Design Studio how normal a research and Design Studio how are you I'm really good thank you for are you I'm really good thank you for are you I'm really good thank you for having me on yeah thanks for hanging out having me on yeah thanks for hanging out having me on yeah thanks for hanging out so you you've got such a lovely website so you you've got such a lovely website so you you've got such a lovely website at maggie.com you've got blogs you've at maggie.com you've got blogs you've at maggie.com you've got blogs you've got notes you've got essays you've got got notes you've got essays you've got got notes you've got essays you've got long form narrative writing and it is uh long form narrative writing and it is uh long form narrative writing and it is uh it is opinionated and I love that um you it is opinionated and I love that um you it is opinionated and I love that um you uh but your opinions are are based in a uh but your opinions are are based in a uh but your opinions are are based in a kind of like you said an intersection of kind of like you said an intersection of kind of like you said an intersection of anthropology and design and programming anthropology and design and programming anthropology and design and programming you're not just a person on the internet you're not just a person on the internet you're not just a person on the internet with opinions how do you form these with opinions how do you form these with opinions how do you form these these ideas around how you think things these ideas around how you think things these ideas around how you think things should be around ux or visual interface should be around ux or visual interface should be around ux or visual interface oh uh that's a very good question maybe oh uh that's a very good question maybe oh uh that's a very good question maybe that I'm hoping I don't go too deep or that I'm hoping I don't go too deep or that I'm hoping I don't go too deep or like ramble too much on it because I'm like ramble too much on it because I'm like ramble too much on it because I'm thinking I do try to base everything in thinking I do try to base everything in thinking I do try to base everything in anthropology um we were just chatting anthropology um we were just chatting anthropology um we were just chatting before that I went to school near where before that I went to school near where before that I went to school near where you live uh Whitman College in you live uh Whitman College in you live uh Whitman College in Washington state and I studied cultural Washington state and I studied cultural Washington state and I studied cultural anthropology there and it was a it was anthropology there and it was a it was anthropology there and it was a it was so wonderful I mean let's say it was not so wonderful I mean let's say it was not so wonderful I mean let's say it was not employable the day after I graduated employable the day after I graduated employable the day after I graduated like my lot and I had no career guidance like my lot and I had no career guidance like my lot and I had no career guidance so they're not winning points for that so they're not winning points for that so they're not winning points for that but my professors there gave me such a but my professors there gave me such a but my professors there gave me such a good theoretical grounding in cultural good theoretical grounding in cultural good theoretical grounding in cultural anthropology and how to think about anthropology and how to think about anthropology and how to think about society and culture and just I would say society and culture and just I would say society and culture and just I would say look at the world from a very look at the world from a very look at the world from a very interesting lens and give me lots of

interesting lens and give me lots of interesting lens and give me lots of interesting tools to kind of pull on and interesting tools to kind of pull on and interesting tools to kind of pull on and interesting thinkers so I had this interesting thinkers so I had this interesting thinkers so I had this really strong grounding that I've tried really strong grounding that I've tried really strong grounding that I've tried to keep up with over the years I briefly to keep up with over the years I briefly to keep up with over the years I briefly went back for a master's degree um but went back for a master's degree um but went back for a master's degree um but Co cut it off um in digital anthropology Co cut it off um in digital anthropology Co cut it off um in digital anthropology but I still love that as a a found found but I still love that as a a found found but I still love that as a a found found for how to think about things um but I for how to think about things um but I for how to think about things um but I also love the level of applying it to also love the level of applying it to also love the level of applying it to the real world of you know I'm very the real world of you know I'm very the real world of you know I'm very immersed in in the web development world immersed in in the web development world immersed in in the web development world as you are that's kind of the place I as you are that's kind of the place I as you are that's kind of the place I live and breathe all my friends are live and breathe all my friends are live and breathe all my friends are developers that's the only people I developers that's the only people I developers that's the only people I really talk to so I try to look at all really talk to so I try to look at all really talk to so I try to look at all the issues or or themes that we deal the issues or or themes that we deal the issues or or themes that we deal with in our in our web development jobs with in our in our web development jobs with in our in our web development jobs through the lens of culture anthropology through the lens of culture anthropology through the lens of culture anthropology which turns out to be I guess a thing which turns out to be I guess a thing which turns out to be I guess a thing not many other people do so it fills a not many other people do so it fills a not many other people do so it fills a good Niche yeah and I think it is a good Niche yeah and I think it is a good Niche yeah and I think it is a useful thing to call out what useful thing to call out what useful thing to call out what anthropology is for those who may have anthropology is for those who may have anthropology is for those who may have forgotten right it's this study of forgotten right it's this study of forgotten right it's this study of humanity but it thinks about culture humanity but it thinks about culture humanity but it thinks about culture Society Linguistics biology all of these Society Linguistics biology all of these Society Linguistics biology all of these things because when I think of a ux things because when I think of a ux things because when I think of a ux person as a as as a developer myself it person as a as as a developer myself it person as a as as a developer myself it seems like they're the person that knows seems like they're the person that knows seems like they're the person that knows why the button goes over there but why the button goes over there but why the button goes over there but you're you're thinking about like why do you're you're thinking about like why do you're you're thinking about like why do we as humans want the button over there we as humans want the button over there we as humans want the button over there like what is the you know animus behind like what is the you know animus behind like what is the you know animus behind that like why this goes on the right and that like why this goes on the right and that like why this goes on the right and that goes on the left there's a whole that goes on the left there's a whole that goes on the left there's a whole conversation there that's more than just conversation there that's more than just conversation there that's more than just it looks better on the right yes yeah it looks better on the right yes yeah it looks better on the right yes yeah cultural anthropologies are very weird cultural anthropologies are very weird cultural anthropologies are very weird discipline and kind of yeah Niche and discipline and kind of yeah Niche and discipline and kind of yeah Niche and maybe not growing that well I will say maybe not growing that well I will say maybe not growing that well I will say they're not very good at self-promotion they're not very good at self-promotion they're not very good at self-promotion as as a discipline um not very popular as as a discipline um not very popular as as a discipline um not very popular um and hard to Define but yes there are um and hard to Define but yes there are um and hard to Define but yes there are um a mix between like practical on the

um a mix between like practical on the um a mix between like practical on the ground research like going genuinely ground research like going genuinely ground research like going genuinely spending time with people often years or spending time with people often years or spending time with people often years or months if you're doing ethnographic work months if you're doing ethnographic work months if you're doing ethnographic work um and trying to sort of see the um and trying to sort of see the um and trying to sort of see the invisible rules that govern how people invisible rules that govern how people invisible rules that govern how people behave in their societies and then behave in their societies and then behave in their societies and then mixing that with philosophy so also mixing that with philosophy so also mixing that with philosophy so also trying to ask meaningful questions not trying to ask meaningful questions not trying to ask meaningful questions not just saying okay why do developers sort just saying okay why do developers sort just saying okay why do developers sort of use certain language and why do they of use certain language and why do they of use certain language and why do they act certain ways but what does that tell act certain ways but what does that tell act certain ways but what does that tell us about the wells and how we should us about the wells and how we should us about the wells and how we should live our lives like there's definitely a live our lives like there's definitely a live our lives like there's definitely a philosophical element to it hence not philosophical element to it hence not philosophical element to it hence not being super employable but being very being super employable but being very being super employable but being very good for writing long form good for writing long form good for writing long form content yeah and you've been thinking content yeah and you've been thinking content yeah and you've been thinking about this stuff at depth on some of about this stuff at depth on some of about this stuff at depth on some of your articles uh you say uh how many your articles uh you say uh how many your articles uh you say uh how many years ago they were they were done like years ago they were they were done like years ago they were they were done like I just read an article that you did I just read an article that you did I just read an article that you did called The Dark Forest in generative AI called The Dark Forest in generative AI called The Dark Forest in generative AI it could have been written last week and it could have been written last week and it could have been written last week and I read it with the lens of having you I read it with the lens of having you I read it with the lens of having you know I'm currently in the present and know I'm currently in the present and know I'm currently in the present and I'm thinking about last week and I'm I'm thinking about last week and I'm I'm thinking about last week and I'm like oh she wrote this two years like oh she wrote this two years like oh she wrote this two years ago it's very preent um do you find ago it's very preent um do you find ago it's very preent um do you find yourself in that situation a lot yeah I yourself in that situation a lot yeah I yourself in that situation a lot yeah I guess I don't know how I couldn't guess I don't know how I couldn't guess I don't know how I couldn't explain to someone I think sometimes explain to someone I think sometimes explain to someone I think sometimes people message me and they go how do you people message me and they go how do you people message me and they go how do you sort of have insights ahead of time not sort of have insights ahead of time not sort of have insights ahead of time not that I'm really over here you know that I'm really over here you know that I'm really over here you know predicting anything useful like the predicting anything useful like the predicting anything useful like the stock market but I there are are certain stock market but I there are are certain stock market but I there are are certain things that I think I've been quite things that I think I've been quite things that I think I've been quite early to like I I I will say I'm not a early to like I I I will say I'm not a early to like I I I will say I'm not a big fan of web 3 but I got really big fan of web 3 but I got really big fan of web 3 but I got really interested in the concept of web 3 when interested in the concept of web 3 when interested in the concept of web 3 when I first came across protocol Labs Works I first came across protocol Labs Works I first came across protocol Labs Works uh protocols Labs work um in 2018 which uh protocols Labs work um in 2018 which uh protocols Labs work um in 2018 which at the time isn't that early but was

at the time isn't that early but was at the time isn't that early but was early for the web 3 movement and I early for the web 3 movement and I early for the web 3 movement and I luckily I I got out of it or was like luckily I I got out of it or was like luckily I I got out of it or was like not affiliated with it by the time it not affiliated with it by the time it not affiliated with it by the time it became nfts and a whole bunch of other became nfts and a whole bunch of other became nfts and a whole bunch of other stuff um but the same happened for stuff um but the same happened for stuff um but the same happened for language models and AI I joined my old language models and AI I joined my old language models and AI I joined my old company elicit in um Mid 2020 but I company elicit in um Mid 2020 but I company elicit in um Mid 2020 but I started talking no not mid 2020 mid 2022 started talking no not mid 2020 mid 2022 started talking no not mid 2020 mid 2022 years a strange at this point um but I years a strange at this point um but I years a strange at this point um but I started looking at language models in started looking at language models in started looking at language models in late 2021 this is like late 2021 this is like late 2021 this is like gpt3 um we you know chat gbt went viral gpt3 um we you know chat gbt went viral gpt3 um we you know chat gbt went viral a year later to give some like timeline a year later to give some like timeline a year later to give some like timeline context but I joined this company and context but I joined this company and context but I joined this company and got really up to speed on language got really up to speed on language got really up to speed on language models and like the theory of AGI and a models and like the theory of AGI and a models and like the theory of AGI and a lot of AI safety work about nine months lot of AI safety work about nine months lot of AI safety work about nine months before the rest of the world had this before the rest of the world had this before the rest of the world had this huge explosion of interest when chat GPT huge explosion of interest when chat GPT huge explosion of interest when chat GPT came out so for some reason I've been came out so for some reason I've been came out so for some reason I've been able to get interested in things before able to get interested in things before able to get interested in things before other people you know before they kind other people you know before they kind other people you know before they kind of hit critical mass of popularity which of hit critical mass of popularity which of hit critical mass of popularity which has been a bit of an advantage in has been a bit of an advantage in has been a bit of an advantage in writing yeah this is going to be an writing yeah this is going to be an writing yeah this is going to be an extended analogy but you at the extended analogy but you at the extended analogy but you at the beginning before we started recording beginning before we started recording beginning before we started recording you know we have the little chat back you know we have the little chat back you know we have the little chat back and forth and you mentioned that you and forth and you mentioned that you and forth and you mentioned that you went to school in the Northwest and I went to school in the Northwest and I went to school in the Northwest and I said oh I'm from Portland and you're said oh I'm from Portland and you're said oh I'm from Portland and you're like I love Portland you know Portland's like I love Portland you know Portland's like I love Portland you know Portland's this weird little town Portland is is is this weird little town Portland is is is this weird little town Portland is is is the Cozy web It's the weird you know the Cozy web It's the weird you know the Cozy web It's the weird you know everyone's like make Portland Weird everyone's like make Portland Weird everyone's like make Portland Weird again it's like Austin like there's again it's like Austin like there's again it's like Austin like there's there's self-identifying weird sections there's self-identifying weird sections there's self-identifying weird sections of of the US and little weird towns and of of the US and little weird towns and of of the US and little weird towns and the Cozy web is this fun little place the Cozy web is this fun little place the Cozy web is this fun little place where Niche people do weird stuff and where Niche people do weird stuff and where Niche people do weird stuff and they're just goofy and weird and people

they're just goofy and weird and people they're just goofy and weird and people have been saying amongst My Generation have been saying amongst My Generation have been saying amongst My Generation I'm a Gen X make the web weird again I'm a Gen X make the web weird again I'm a Gen X make the web weird again like where's Tumblr where's geoc cities like where's Tumblr where's geoc cities like where's Tumblr where's geoc cities and you have called out and others have and you have called out and others have and you have called out and others have said that the dark Forest is this part said that the dark Forest is this part said that the dark Forest is this part of the web that is eerily devoid of of the web that is eerily devoid of of the web that is eerily devoid of human life it's this of human life it's this of human life it's this of like like I don't know how to spell that like like I don't know how to spell that like like I don't know how to spell that but like it's just this quiet Eerie but like it's just this quiet Eerie but like it's just this quiet Eerie place and it feels like AI is is is place and it feels like AI is is is place and it feels like AI is is is going to create a big dark sad going to create a big dark sad going to create a big dark sad empty algorithmic section of the empty algorithmic section of the empty algorithmic section of the internet um and that makes me sad as a internet um and that makes me sad as a internet um and that makes me sad as a portlander yeah uh that's a good that's portlander yeah uh that's a good that's portlander yeah uh that's a good that's a good point that I think that anal fits a good point that I think that anal fits a good point that I think that anal fits that Portland is the Cozy web it is very that Portland is the Cozy web it is very that Portland is the Cozy web it is very just strange the whole Northwest that's just strange the whole Northwest that's just strange the whole Northwest that's kind of why I love it I wish London had kind of why I love it I wish London had kind of why I love it I wish London had a little more of that weirdness frankly a little more of that weirdness frankly a little more of that weirdness frankly they're a bit more traditionalist they're a bit more traditionalist they're a bit more traditionalist British um but this yeah this theory British um but this yeah this theory British um but this yeah this theory around the dark web was and I I mean around the dark web was and I I mean around the dark web was and I I mean again I had given this talk oh God I again I had given this talk oh God I again I had given this talk oh God I think of years again it must have been think of years again it must have been think of years again it must have been two years ago I gave this talk on the two years ago I gave this talk on the two years ago I gave this talk on the expanding Dark Forest in gener AI just expanding Dark Forest in gener AI just expanding Dark Forest in gener AI just because I had started working in this AI because I had started working in this AI because I had started working in this AI startup and seeing what was possible I startup and seeing what was possible I startup and seeing what was possible I just I can't tell you how it came to me just I can't tell you how it came to me just I can't tell you how it came to me it's sort of like I think my ideas build it's sort of like I think my ideas build it's sort of like I think my ideas build up very slowly over time I see lots and up very slowly over time I see lots and up very slowly over time I see lots and lots of evidence Fly By Me on what you lots of evidence Fly By Me on what you lots of evidence Fly By Me on what you know used to be Twitter now blue sky and know used to be Twitter now blue sky and know used to be Twitter now blue sky and on blogs and on Hacker News and I just I on blogs and on Hacker News and I just I on blogs and on Hacker News and I just I read a lot and slowly you get these read a lot and slowly you get these read a lot and slowly you get these things of like oh I'm so sure that we're things of like oh I'm so sure that we're things of like oh I'm so sure that we're just about to get this flood of AI just about to get this flood of AI just about to get this flood of AI generated generated generated content and then you know I'm able to I

content and then you know I'm able to I content and then you know I'm able to I have some Frameworks that I was taught have some Frameworks that I was taught have some Frameworks that I was taught from culture anthropology and from from culture anthropology and from from culture anthropology and from certain ways of thinking like um design certain ways of thinking like um design certain ways of thinking like um design Futures and um Rec speculative design Futures and um Rec speculative design Futures and um Rec speculative design about how to sort of think about the about how to sort of think about the about how to sort of think about the repercussions or follow on implications repercussions or follow on implications repercussions or follow on implications of certain things you're seeing and one of certain things you're seeing and one of certain things you're seeing and one of the things I thought through just of the things I thought through just of the things I thought through just quite extensively was like okay let's quite extensively was like okay let's quite extensively was like okay let's say there's a web where the vast say there's a web where the vast say there's a web where the vast majority of content is made not by what majority of content is made not by what majority of content is made not by what we used to know as Bots but by much more we used to know as Bots but by much more we used to know as Bots but by much more sophisticated AI agents who have very sophisticated AI agents who have very sophisticated AI agents who have very particular um goals they've been set by particular um goals they've been set by particular um goals they've been set by humans like promote this product or make humans like promote this product or make humans like promote this product or make it seem like this idea is popular among it seem like this idea is popular among it seem like this idea is popular among people or make this political candidate people or make this political candidate people or make this political candidate popular what happens when that um works popular what happens when that um works popular what happens when that um works at scale what happens when you are able at scale what happens when you are able at scale what happens when you are able to release a million AI agents to cover to release a million AI agents to cover to release a million AI agents to cover Pinterest and Twitter and Linkedin with Pinterest and Twitter and Linkedin with Pinterest and Twitter and Linkedin with all of this fake but you know very all of this fake but you know very all of this fake but you know very convincing humanlike content and then convincing humanlike content and then convincing humanlike content and then just kind of think through interesting just kind of think through interesting just kind of think through interesting implications of that so that was what implications of that so that was what implications of that so that was what that whole talk was yeah it has me that whole talk was yeah it has me that whole talk was yeah it has me thinking about how I feel about thinking about how I feel about thinking about how I feel about existence and how I feel about moving existence and how I feel about moving existence and how I feel about moving around and how I am on even when I'm in around and how I am on even when I'm in around and how I am on even when I'm in a digital wall Garden like Instagram as a digital wall Garden like Instagram as a digital wall Garden like Instagram as an example the things I end up following an example the things I end up following an example the things I end up following are cozy things like one of the things are cozy things like one of the things are cozy things like one of the things I'm excited about now is meet cutes New I'm excited about now is meet cutes New I'm excited about now is meet cutes New York in Meet cutes London where this York in Meet cutes London where this York in Meet cutes London where this gentleman says are you a couple would gentleman says are you a couple would gentleman says are you a couple would you mind telling us the story about how you mind telling us the story about how you mind telling us the story about how you met and then it it basically takes you met and then it it basically takes you met and then it it basically takes all these people who are effectively all these people who are effectively all these people who are effectively NPCs like the non-player characters of

NPCs like the non-player characters of NPCs like the non-player characters of the people that you pass and then it the people that you pass and then it the people that you pass and then it humanizes them in a moment and you go humanizes them in a moment and you go humanizes them in a moment and you go wow that couple that just walked by has wow that couple that just walked by has wow that couple that just walked by has a really interesting story a really interesting story a really interesting story the thing that makes me sad about these the thing that makes me sad about these the thing that makes me sad about these large language models that to your this large language models that to your this large language models that to your this is a quote can instantly generate is a quote can instantly generate is a quote can instantly generate coherent swats of humanlike coherent swats of humanlike coherent swats of humanlike text uh is that they are able to create text uh is that they are able to create text uh is that they are able to create NPCs which make what should be a very NPCs which make what should be a very NPCs which make what should be a very excited excited excited um dense world of people of whom um dense world of people of whom um dense world of people of whom everyone has a back story into just a everyone has a back story into just a everyone has a back story into just a bunch of like I don't know kind of Elden bunch of like I don't know kind of Elden bunch of like I don't know kind of Elden ring uh you know you know you know what ring uh you know you know you know what ring uh you know you know you know what Elder Scrolls kind of like NPCs like H Elder Scrolls kind of like NPCs like H Elder Scrolls kind of like NPCs like H these aren't real people which is a very these aren't real people which is a very these aren't real people which is a very lonely feeling you're surrounded but lonely feeling you're surrounded but lonely feeling you're surrounded but also lonely and that's what the Web is also lonely and that's what the Web is also lonely and that's what the Web is starting to feel like yeah it's it's starting to feel like yeah it's it's starting to feel like yeah it's it's genuinely happening now again like as I genuinely happening now again like as I genuinely happening now again like as I said it was sort of I had had this you said it was sort of I had had this you said it was sort of I had had this you know thought experiment what if this know thought experiment what if this know thought experiment what if this happens and then the sad thing is I've happens and then the sad thing is I've happens and then the sad thing is I've done that same conference talk I think done that same conference talk I think done that same conference talk I think five or six times since I first gave it five or six times since I first gave it five or six times since I first gave it over two years ago and now when I update over two years ago and now when I update over two years ago and now when I update the talk every time I'm just adding all the talk every time I'm just adding all the talk every time I'm just adding all this evidence to it the first time I this evidence to it the first time I this evidence to it the first time I gave it I had had theories and no gave it I had had theories and no gave it I had had theories and no evidence whatsoever to back them up I evidence whatsoever to back them up I evidence whatsoever to back them up I couldn't find a single like generated AI couldn't find a single like generated AI couldn't find a single like generated AI post on LinkedIn or Twitter at the time post on LinkedIn or Twitter at the time post on LinkedIn or Twitter at the time it wasn't happening and now half the it wasn't happening and now half the it wasn't happening and now half the talk is just like horrifying AI gen talk is just like horrifying AI gen talk is just like horrifying AI gen photos from Facebook you know with like photos from Facebook you know with like photos from Facebook you know with like thousands of AI comments on them so it thousands of AI comments on them so it thousands of AI comments on them so it was kind of awful to have that proved was kind of awful to have that proved was kind of awful to have that proved correct yeah it is it is preent you you correct yeah it is it is preent you you correct yeah it is it is preent you you said we are about to drown in a sea of

said we are about to drown in a sea of said we are about to drown in a sea of pedestrian takes everything is just mid pedestrian takes everything is just mid pedestrian takes everything is just mid you know yeah sucks yeah and it really you know yeah sucks yeah and it really you know yeah sucks yeah and it really matters um I don't know if you felt this matters um I don't know if you felt this matters um I don't know if you felt this when even listeners might have that when when even listeners might have that when when even listeners might have that when you come across a whole slew of Twitter you come across a whole slew of Twitter you come across a whole slew of Twitter comments or or LinkedIn posts that you comments or or LinkedIn posts that you comments or or LinkedIn posts that you can just tell chat gbt wrote them you can just tell chat gbt wrote them you can just tell chat gbt wrote them you just get this this yeah this lonely just get this this yeah this lonely just get this this yeah this lonely sense this empty sense like I was trying sense this empty sense like I was trying sense this empty sense like I was trying to look up you know interior design to look up you know interior design to look up you know interior design stuff on Pinterest earli because I have stuff on Pinterest earli because I have stuff on Pinterest earli because I have no interior design sense but I have to no interior design sense but I have to no interior design sense but I have to decorate a house or something um and decorate a house or something um and decorate a house or something um and over half the images were AI generated over half the images were AI generated over half the images were AI generated you just know the vibe of them and you just know the vibe of them and you just know the vibe of them and you're like okay listen I really need to you're like okay listen I really need to you're like okay listen I really need to understand like physical materials understand like physical materials understand like physical materials texture light to like make a room feel texture light to like make a room feel texture light to like make a room feel nice and oh I've got are fake images of nice and oh I've got are fake images of nice and oh I've got are fake images of non places that are not grounded in the non places that are not grounded in the non places that are not grounded in the real world so this is all useless to me real world so this is all useless to me real world so this is all useless to me this is not a technical problem though this is not a technical problem though this is not a technical problem though everything that we've talked about is everything that we've talked about is everything that we've talked about is all people all people all people problems how do we solve that because problems how do we solve that because problems how do we solve that because right now I'm seeing in all companies right now I'm seeing in all companies right now I'm seeing in all companies not just the one that I work for in my not just the one that I work for in my not just the one that I work for in my day job um a laziness as we apply AI to day job um a laziness as we apply AI to day job um a laziness as we apply AI to things where it's just co-pilot for things where it's just co-pilot for things where it's just co-pilot for blank text box over thing and that seems blank text box over thing and that seems blank text box over thing and that seems like the lazy maybe maybe too too crisp like the lazy maybe maybe too too crisp like the lazy maybe maybe too too crisp of of a word but it's just it's the of of a word but it's just it's the of of a word but it's just it's the simple as possible thing is to put a simple as possible thing is to put a simple as possible thing is to put a text box over there and have it fill up text box over there and have it fill up text box over there and have it fill up the whole right side and it'll be a the whole right side and it'll be a the whole right side and it'll be a co-pilot for the thing yeah um so this co-pilot for the thing yeah um so this co-pilot for the thing yeah um so this gets into I know the motivations and

gets into I know the motivations and gets into I know the motivations and incentives for why people build certain incentives for why people build certain incentives for why people build certain kinds of products so at the moment you kinds of products so at the moment you kinds of products so at the moment you know I think there's a lot easy money to know I think there's a lot easy money to know I think there's a lot easy money to be made by building co-pilot or chatbot be made by building co-pilot or chatbot be made by building co-pilot or chatbot style things on top of you know insert style things on top of you know insert style things on top of you know insert domain or insert problem here that won't domain or insert problem here that won't domain or insert problem here that won't last forever though I think chat Bots I last forever though I think chat Bots I last forever though I think chat Bots I you could get into it it's like they are you could get into it it's like they are you could get into it it's like they are fairly good um low barrier to entry fairly good um low barrier to entry fairly good um low barrier to entry interfaces to most things like if you interfaces to most things like if you interfaces to most things like if you want to do a really generic task like want to do a really generic task like want to do a really generic task like search the web a chat bar might actually search the web a chat bar might actually search the web a chat bar might actually be I don't think it's the optimal be I don't think it's the optimal be I don't think it's the optimal interface but it will do an okay job of interface but it will do an okay job of interface but it will do an okay job of it and I think a chatbot will do a sort it and I think a chatbot will do a sort it and I think a chatbot will do a sort of okay job of a large range of tasks of okay job of a large range of tasks of okay job of a large range of tasks that you might want to do on a computer that you might want to do on a computer that you might want to do on a computer but that definitely doesn't mean that it but that definitely doesn't mean that it but that definitely doesn't mean that it is the optimal interface for any one of is the optimal interface for any one of is the optimal interface for any one of those tasks there's almost always a kind those tasks there's almost always a kind those tasks there's almost always a kind of interface or a kind of visual of interface or a kind of visual of interface or a kind of visual presentation that's going to be a much presentation that's going to be a much presentation that's going to be a much better fit that will be much more better fit that will be much more better fit that will be much more effective in getting the person to do effective in getting the person to do effective in getting the person to do the task that will give them more the task that will give them more the task that will give them more information that will be a richer more information that will be a richer more information that will be a richer more Dynamic experience but that takes effort Dynamic experience but that takes effort Dynamic experience but that takes effort to figure out what that specific to figure out what that specific to figure out what that specific interface is and that's just classic interface is and that's just classic interface is and that's just classic interface interaction ux design work interface interaction ux design work interface interaction ux design work that I think for a moment a lot of that I think for a moment a lot of that I think for a moment a lot of people got quite excited that we could people got quite excited that we could people got quite excited that we could ditch that they were like oh great we ditch that they were like oh great we ditch that they were like oh great we don't have to do all that work anymore don't have to do all that work anymore don't have to do all that work anymore we don't have to do all that user we don't have to do all that user we don't have to do all that user research we don't do have to do all that research we don't do have to do all that research we don't do have to do all that domain research we don't have to domain research we don't have to domain research we don't have to understand the user needs we don't have understand the user needs we don't have understand the user needs we don't have to think through how to display complex to think through how to display complex to think through how to display complex information in dense ways we just are information in dense ways we just are information in dense ways we just are going to put a chat bot on it um and going to put a chat bot on it um and going to put a chat bot on it um and that was a theory for a year or so and I that was a theory for a year or so and I that was a theory for a year or so and I think people are starting to realize think people are starting to realize think people are starting to realize that theory is

that theory is that theory is untrue the when when the web first untrue the when when the web first untrue the when when the web first started started started and and uh right right in the moment and and uh right right in the moment and and uh right right in the moment when JavaScript be started being used when JavaScript be started being used when JavaScript be started being used for uh textbox validation uh we would for uh textbox validation uh we would for uh textbox validation uh we would run around and teach this stuff to run around and teach this stuff to run around and teach this stuff to people this is in the mid90s early 90s people this is in the mid90s early 90s people this is in the mid90s early 90s and we'd say don't trust user input and and we'd say don't trust user input and and we'd say don't trust user input and we build these amazing forums just text we build these amazing forums just text we build these amazing forums just text boxes over data deep deep deep text boxes over data deep deep deep text boxes over data deep deep deep text boxes over data and now it's one text boxes over data and now it's one text boxes over data and now it's one text box and apparently we're supposed to box and apparently we're supposed to box and apparently we're supposed to just trust it and all the users that just trust it and all the users that just trust it and all the users that type into it it seems the irony is thick type into it it seems the irony is thick type into it it seems the irony is thick yeah for every task in the world like yeah for every task in the world like yeah for every task in the world like you want to make an image like where you you want to make an image like where you you want to make an image like where you would traditionally paint you know do would traditionally paint you know do would traditionally paint you know do sketches and like lay out the the um sketches and like lay out the the um sketches and like lay out the the um composition of it and think about composition of it and think about composition of it and think about lighting and think about uh flow and lighting and think about uh flow and lighting and think about uh flow and structure and color you no longer do structure and color you no longer do structure and color you no longer do that you just type into a box right you that you just type into a box right you that you just type into a box right you no longer need to control where objects no longer need to control where objects no longer need to control where objects are in the scene or what color things are in the scene or what color things are in the scene or what color things are um just you know mid Journey will are um just you know mid Journey will are um just you know mid Journey will generate it or like oh you want to yeah generate it or like oh you want to yeah generate it or like oh you want to yeah run a scientific experiment here just run a scientific experiment here just run a scientific experiment here just type into the box and it's like okay type into the box and it's like okay type into the box and it's like okay sure again for for something like I have sure again for for something like I have sure again for for something like I have a single question with a single answer a single question with a single answer a single question with a single answer that the the Wikipedia could answer that that the the Wikipedia could answer that that the the Wikipedia could answer that is maybe a decent use case but for is maybe a decent use case but for is maybe a decent use case but for almost every other I will say serious almost every other I will say serious almost every other I will say serious complex human task that needs to be done complex human task that needs to be done complex human task that needs to be done in the world the kinds of things that in the world the kinds of things that in the world the kinds of things that we're being promised AI will actually we're being promised AI will actually we're being promised AI will actually help with like it's going to solve help with like it's going to solve help with like it's going to solve medicine and health care and you know medicine and health care and you know medicine and health care and you know public infrastructure and it's going to public infrastructure and it's going to public infrastructure and it's going to solve climate change and you think well solve climate change and you think well solve climate change and you think well all of those are extremely complex all of those are extremely complex all of those are extremely complex specific you know expert driven stuff

specific you know expert driven stuff specific you know expert driven stuff there is absolutely no reason to believe there is absolutely no reason to believe there is absolutely no reason to believe a chat but we'll serve the needs of the a chat but we'll serve the needs of the a chat but we'll serve the needs of the people who are doing those jobs exactly people who are doing those jobs exactly people who are doing those jobs exactly so then people who canot Express so then people who canot Express so then people who canot Express themselves clearly and we all know we're themselves clearly and we all know we're themselves clearly and we all know we're in a podcast extemporaneous speech is in a podcast extemporaneous speech is in a podcast extemporaneous speech is difficult why do they why do we think difficult why do they why do we think difficult why do they why do we think that the lack of clarity that they're that the lack of clarity that they're that the lack of clarity that they're able to express off the top of their able to express off the top of their able to express off the top of their head is going to turn into a great squl head is going to turn into a great squl head is going to turn into a great squl query or a really solid you know Dom uh query or a really solid you know Dom uh query or a really solid you know Dom uh this is where everything is perfectly this is where everything is perfectly this is where everything is perfectly fine like they love to show you a demo fine like they love to show you a demo fine like they love to show you a demo where like look I wrote pong in react by where like look I wrote pong in react by where like look I wrote pong in react by just asking for it but they they forget just asking for it but they they forget just asking for it but they they forget that there's 20,000 example pongs out that there's 20,000 example pongs out that there's 20,000 example pongs out there and it just kind of cribbed It there and it just kind of cribbed It there and it just kind of cribbed It kind of kind of kind of cheated this is kind of tough in a world cheated this is kind of tough in a world cheated this is kind of tough in a world where everyone is recommending the word where everyone is recommending the word where everyone is recommending the word uh Rec recommending the book deep work uh Rec recommending the book deep work uh Rec recommending the book deep work everyone's like yeah you should read everyone's like yeah you should read everyone's like yeah you should read deep work deep work we're asking people deep work deep work we're asking people deep work deep work we're asking people to do the least yeah yeah think was a to do the least yeah yeah think was a to do the least yeah yeah think was a thing I'm trying to figure out something thing I'm trying to figure out something thing I'm trying to figure out something to write about this or or do a talk on to write about this or or do a talk on to write about this or or do a talk on um as like an upcoming piece and um as like an upcoming piece and um as like an upcoming piece and something around this thing of complex something around this thing of complex something around this thing of complex work is really what I wish more people work is really what I wish more people work is really what I wish more people would be focused on um I'm also part of would be focused on um I'm also part of would be focused on um I'm also part of this community uh called tools for this community uh called tools for this community uh called tools for thought that uh as a as a philosophical thought that uh as a as a philosophical thought that uh as a as a philosophical dream is a really good idea like how do dream is a really good idea like how do dream is a really good idea like how do we build better tools that help humans we build better tools that help humans we build better tools that help humans think better kinds of thoughts which think better kinds of thoughts which think better kinds of thoughts which sounds very vague um but in recent years sounds very vague um but in recent years sounds very vague um but in recent years has become very affiliated with sort of has become very affiliated with sort of has become very affiliated with sort of note- taking apps like obsidian or log note- taking apps like obsidian or log note- taking apps like obsidian or log seek or Rome research was a very popular seek or Rome research was a very popular seek or Rome research was a very popular one um that tried to help people see one um that tried to help people see one um that tried to help people see connections between ideas and organize connections between ideas and organize connections between ideas and organize information and like you know very um

information and like you know very um information and like you know very um noble goals like good philosophical noble goals like good philosophical noble goals like good philosophical goals um but ended up sort of being goals um but ended up sort of being goals um but ended up sort of being fancy text editors in the end and now fancy text editors in the end and now fancy text editors in the end and now that we're sort of at a moment where that we're sort of at a moment where that we're sort of at a moment where this dream of tools for thought I wish this dream of tools for thought I wish this dream of tools for thought I wish people would be more ambitious with it people would be more ambitious with it people would be more ambitious with it and think about well genuinely if you're and think about well genuinely if you're and think about well genuinely if you're going to say what kinds of tools would going to say what kinds of tools would going to say what kinds of tools would make humans think more important complex make humans think more important complex make humans think more important complex you know higher level kinds of thought you know higher level kinds of thought you know higher level kinds of thought you have to move Beyond text editors and you have to move Beyond text editors and you have to move Beyond text editors and think really about cognitive science and think really about cognitive science and think really about cognitive science and anthropology um and and if we're going anthropology um and and if we're going anthropology um and and if we're going to say that AI really is this great to say that AI really is this great to say that AI really is this great thinking tool this great cognitive aid thinking tool this great cognitive aid thinking tool this great cognitive aid for all of us I want people to be a bit for all of us I want people to be a bit for all of us I want people to be a bit more ambitious and take more seriously more ambitious and take more seriously more ambitious and take more seriously what kind of work you would need to do what kind of work you would need to do what kind of work you would need to do to apply that to to real problems that to apply that to to real problems that to apply that to to real problems that I'm not just organizing text on a page I'm not just organizing text on a page I'm not just organizing text on a page yeah I like that that term cognitive Aid yeah I like that that term cognitive Aid yeah I like that that term cognitive Aid um I'm very uh aspirational and very um I'm very uh aspirational and very um I'm very uh aspirational and very optimistic almost like toxically optimistic almost like toxically optimistic almost like toxically positive and I feel like I was promised positive and I feel like I was promised positive and I feel like I was promised the Star Trek the Next Generation future the Star Trek the Next Generation future the Star Trek the Next Generation future and like when I got an iPad I'm like and like when I got an iPad I'm like and like when I got an iPad I'm like look it's like a pad like Captain peard look it's like a pad like Captain peard look it's like a pad like Captain peard had one and now I have one um and when I had one and now I have one um and when I had one and now I have one um and when I want to talk to a computer I want to be want to talk to a computer I want to be want to talk to a computer I want to be like Scotty in Star Trek 4 in which I like Scotty in Star Trek 4 in which I like Scotty in Star Trek 4 in which I saw at the theater in 1984 when he picks saw at the theater in 1984 when he picks saw at the theater in 1984 when he picks up the mouse and he goes computer and up the mouse and he goes computer and up the mouse and he goes computer and thought he was going to have a whole thought he was going to have a whole thought he was going to have a whole conversation with the computer and they conversation with the computer and they conversation with the computer and they said just use the keyboard and the mouse said just use the keyboard and the mouse said just use the keyboard and the mouse and he's like oh how quaint but in Star and he's like oh how quaint but in Star and he's like oh how quaint but in Star Trek Discovery and they they they Vibe Trek Discovery and they they they Vibe Trek Discovery and they they they Vibe they brainstorm it's a partner it's a they brainstorm it's a partner it's a they brainstorm it's a partner it's a rubber duck rubber duck rubber duck um is is rubber ducking a reasonable

um is is rubber ducking a reasonable um is is rubber ducking a reasonable thing to do with an llm given that I'm thing to do with an llm given that I'm thing to do with an llm given that I'm just talking to myself like I'm not just talking to myself like I'm not just talking to myself like I'm not really talking to an entity yeah this is really talking to an entity yeah this is really talking to an entity yeah this is my favorite use case for it or the thing my favorite use case for it or the thing my favorite use case for it or the thing that I argue should be the primary use that I argue should be the primary use that I argue should be the primary use for it at in its current state um and for it at in its current state um and for it at in its current state um and again this is one of those like I'm not again this is one of those like I'm not again this is one of those like I'm not saying I was that early to this but I saying I was that early to this but I saying I was that early to this but I have this unpublished post from three have this unpublished post from three have this unpublished post from three years ago called language models of years ago called language models of years ago called language models of rubber ducks that I never published rubber ducks that I never published rubber ducks that I never published because I don't know what happened it because I don't know what happened it because I don't know what happened it just like never made it through editing just like never made it through editing just like never made it through editing and now I'm like so but I just but I and now I'm like so but I just but I and now I'm like so but I just but I never published it I now it's still never published it I now it's still never published it I now it's still relevant um but the the argument was relevant um but the the argument was relevant um but the the argument was everyone's trying to treat them as an everyone's trying to treat them as an everyone's trying to treat them as an oracle but instead they should be a oracle but instead they should be a oracle but instead they should be a rubber duck because that is their best rubber duck because that is their best rubber duck because that is their best use case right now um and this was even use case right now um and this was even use case right now um and this was even back when we were using chat g g uh back when we were using chat g g uh back when we were using chat g g uh sorry gbg3 but it's still true today I sorry gbg3 but it's still true today I sorry gbg3 but it's still true today I feel where they are amazing thought feel where they are amazing thought feel where they are amazing thought Partners to just this is the Primo way I Partners to just this is the Primo way I Partners to just this is the Primo way I use that sounds like you do too is is use that sounds like you do too is is use that sounds like you do too is is just bouncing ideas off them I'm like just bouncing ideas off them I'm like just bouncing ideas off them I'm like hey I'm trying to work through this hey I'm trying to work through this hey I'm trying to work through this problem I don't think it has the answer problem I don't think it has the answer problem I don't think it has the answer for me at all I think I have the answer for me at all I think I have the answer for me at all I think I have the answer but I have to figure out how to get to but I have to figure out how to get to but I have to figure out how to get to that answer and I need someone to talk that answer and I need someone to talk that answer and I need someone to talk it through with and prompt me and ask me it through with and prompt me and ask me it through with and prompt me and ask me questions and it's great for that that questions and it's great for that that questions and it's great for that that is for me is how I think is the most is for me is how I think is the most is for me is how I think is the most responsible and useful use case for the responsible and useful use case for the responsible and useful use case for the the the the moment hey friends for the next couple moment hey friends for the next couple moment hey friends for the next couple of episodes while I learn more about AI of episodes while I learn more about AI of episodes while I learn more about AI in the middle of the show I'm going to in the middle of the show I'm going to in the middle of the show I'm going to chat with ler Van Dorne from Qualcomm chat with ler Van Dorne from Qualcomm chat with ler Van Dorne from Qualcomm and look at some of the great work that and look at some of the great work that and look at some of the great work that they're doing over there I've got a a they're doing over there I've got a a they're doing over there I've got a a Snapdragon based co-pilot PC right now Snapdragon based co-pilot PC right now Snapdragon based co-pilot PC right now and the battery life is like it's the

and the battery life is like it's the and the battery life is like it's the battery life that they always promised battery life that they always promised battery life that they always promised me how is that possible what changes me how is that possible what changes me how is that possible what changes were made to make the battery life so were made to make the battery life so were made to make the battery life so good yeah so first of all you're good yeah so first of all you're good yeah so first of all you're absolutely right right if I look at the absolutely right right if I look at the absolutely right right if I look at the Dell XPS I think it's 19 hours HPS Omni Dell XPS I think it's 19 hours HPS Omni Dell XPS I think it's 19 hours HPS Omni book I think has 26 hours is absolutely book I think has 26 hours is absolutely book I think has 26 hours is absolutely amazing B it's the old day compute that amazing B it's the old day compute that amazing B it's the old day compute that we will be there going for uh the way we will be there going for uh the way we will be there going for uh the way we're getting there is first of all a we're getting there is first of all a we're getting there is first of all a very efficient CPU right so we have a very efficient CPU right so we have a very efficient CPU right so we have a CPU that gives you phenomenal power CPU that gives you phenomenal power CPU that gives you phenomenal power right that's so it makes it a very right that's so it makes it a very right that's so it makes it a very Snappy platform but it also does that at Snappy platform but it also does that at Snappy platform but it also does that at very very efficiently the other part of very very efficiently the other part of very very efficiently the other part of it too we have lots of dedicated it too we have lots of dedicated it too we have lots of dedicated accelerators including the npu which is accelerators including the npu which is accelerators including the npu which is a dedicated AI accelerator it just runs a dedicated AI accelerator it just runs a dedicated AI accelerator it just runs the AI inference algorithms and it does the AI inference algorithms and it does the AI inference algorithms and it does that better and more power efficient that better and more power efficient that better and more power efficient than anybody else right so it's a than anybody else right so it's a than anybody else right so it's a combination of these two things it's a combination of these two things it's a combination of these two things it's a very efficient CPU and then a hetro very efficient CPU and then a hetro very efficient CPU and then a hetro genius accelerator strategy which genius accelerator strategy which genius accelerator strategy which includes an AI machine learning includes an AI machine learning includes an AI machine learning accelerator that is accelerator that is accelerator that is absolutely wow that's so cool it's it's absolutely wow that's so cool it's it's absolutely wow that's so cool it's it's really happening uh the move to a new really happening uh the move to a new really happening uh the move to a new instruction set architecture is seeing instruction set architecture is seeing instruction set architecture is seeing some really extraordinary improvements some really extraordinary improvements some really extraordinary improvements I'm really enjoying my my co-pilot PC I'm really enjoying my my co-pilot PC I'm really enjoying my my co-pilot PC folks can learn a lot more at folks can learn a lot more at folks can learn a lot more at qualcomm.com developer if you want to qualcomm.com developer if you want to qualcomm.com developer if you want to build your software for this great new build your software for this great new build your software for this great new platform I love that you said right now and at I love that you said right now and at the moment three times in that like

the moment three times in that like the moment three times in that like speech in that moment because you're speech in that moment because you're speech in that moment because you're being conscious and focused on like this being conscious and focused on like this being conscious and focused on like this is the State of Affairs this is the is the State of Affairs this is the is the State of Affairs this is the state of the tech maybe tomorrow will be state of the tech maybe tomorrow will be state of the tech maybe tomorrow will be different maybe five years ago was different maybe five years ago was different maybe five years ago was different but at the moment they are different but at the moment they are different but at the moment they are extraordinarily good rubber ducks and extraordinarily good rubber ducks and extraordinarily good rubber ducks and and and you're you're couching that in a and and you're you're couching that in a and and you're you're couching that in a for now and I love that I love that it's for now and I love that I love that it's for now and I love that I love that it's good that you did that is that conscious good that you did that is that conscious good that you did that is that conscious yes yeah I think uh I I did I learned a yes yeah I think uh I I did I learned a yes yeah I think uh I I did I learned a good amount of context about the current good amount of context about the current good amount of context about the current state of like AI capabilities and where state of like AI capabilities and where state of like AI capabilities and where we might be able to take them by working we might be able to take them by working we might be able to take them by working for this company elicit previously where for this company elicit previously where for this company elicit previously where the founders had been involved in in the founders had been involved in in the founders had been involved in in researching language models for 10 years researching language models for 10 years researching language models for 10 years you know way ahead of the curve kind of you know way ahead of the curve kind of you know way ahead of the curve kind of thing um and I was very useful talking thing um and I was very useful talking thing um and I was very useful talking to them about what they think is going to them about what they think is going to them about what they think is going to be possible soon they were very much to be possible soon they were very much to be possible soon they were very much on the side of like we're all going to on the side of like we're all going to on the side of like we're all going to have AGI in 5 years which is not the have AGI in 5 years which is not the have AGI in 5 years which is not the camp I'm in for sure like I'm definitely camp I'm in for sure like I'm definitely camp I'm in for sure like I'm definitely quite skeptical of only text based tools quite skeptical of only text based tools quite skeptical of only text based tools getting us to any sort of humanlike getting us to any sort of humanlike getting us to any sort of humanlike intelligence just because I've studied intelligence just because I've studied intelligence just because I've studied enough embodied cognition and cognitive enough embodied cognition and cognitive enough embodied cognition and cognitive science to be like our bodies in science to be like our bodies in science to be like our bodies in physical space are incredibly important physical space are incredibly important physical space are incredibly important and the fact that they don't have any and the fact that they don't have any and the fact that they don't have any physical inputs is just a a huge blind physical inputs is just a a huge blind physical inputs is just a a huge blind spot like maybe we can make robot bodies spot like maybe we can make robot bodies spot like maybe we can make robot bodies and teach them space and time and color and teach them space and time and color and teach them space and time and color and texture and emotion but I don't see and texture and emotion but I don't see and texture and emotion but I don't see any plans for us to do that so until we any plans for us to do that so until we any plans for us to do that so until we do very skeptical um but I am you know do very skeptical um but I am you know do very skeptical um but I am you know like we always have to think about what like we always have to think about what like we always have to think about what might be possible if there's a new

might be possible if there's a new might be possible if there's a new technical breakthrough maybe someone technical breakthrough maybe someone technical breakthrough maybe someone does invent something that's akin to the does invent something that's akin to the does invent something that's akin to the Transformer where we make some huge Transformer where we make some huge Transformer where we make some huge jumping capabilities but at the moment jumping capabilities but at the moment jumping capabilities but at the moment I'm not seeing it I'm but I'm you know I'm not seeing it I'm but I'm you know I'm not seeing it I'm but I'm you know what do I know I don't work for deep what do I know I don't work for deep what do I know I don't work for deep mind I don't know what they're doing mind I don't know what they're doing mind I don't know what they're doing yeah yeah the the treating it as an yeah yeah the the treating it as an yeah yeah the the treating it as an oracle is a problem so just before this oracle is a problem so just before this oracle is a problem so just before this talk I was at the school district uh and talk I was at the school district uh and talk I was at the school district uh and I presented my perspective on AI to 90 I presented my perspective on AI to 90 I presented my perspective on AI to 90 uh teachers all line line teachers like uh teachers all line line teachers like uh teachers all line line teachers like they they left the meeting and went to they they left the meeting and went to they they left the meeting and went to teach immediately afterwards they teach immediately afterwards they teach immediately afterwards they weren't like school board members and um weren't like school board members and um weren't like school board members and um they're afraid and I think they're they're afraid and I think they're they're afraid and I think they're afraid because they don't understand the afraid because they don't understand the afraid because they don't understand the thing so I explained the thing and they thing so I explained the thing and they thing so I explained the thing and they were like oh well now I am both less were like oh well now I am both less were like oh well now I am both less afraid and also more afraid like you afraid and also more afraid like you afraid and also more afraid like you know like now I get it but like also know like now I get it but like also know like now I get it but like also we're just talking to a dice role at we're just talking to a dice role at we're just talking to a dice role at this point and one of the this point and one of the this point and one of the recommendations that I made although I recommendations that I made although I recommendations that I made although I didn't phrase it as nicely as you did didn't phrase it as nicely as you did didn't phrase it as nicely as you did like treat it as a a partner a writing like treat it as a a partner a writing like treat it as a a partner a writing partner rather than Oracle was they said partner rather than Oracle was they said partner rather than Oracle was they said kids are going and they're writing kids are going and they're writing kids are going and they're writing entire essays uh on Abe Lincoln and we entire essays uh on Abe Lincoln and we entire essays uh on Abe Lincoln and we did a prompt we did a prompt exercise did a prompt we did a prompt exercise did a prompt we did a prompt exercise where I told Chad GPT 4 do not write where I told Chad GPT 4 do not write where I told Chad GPT 4 do not write this for me we explicitly came out of this for me we explicitly came out of this for me we explicitly came out of said I want to talk to you about said I want to talk to you about said I want to talk to you about organizing my thoughts about my article organizing my thoughts about my article organizing my thoughts about my article I have to write a thing about Abe I have to write a thing about Abe I have to write a thing about Abe Lincoln do not write it for me Lincoln do not write it for me Lincoln do not write it for me and I tried to take it off topic I tried and I tried to take it off topic I tried and I tried to take it off topic I tried to have it talk about other stuff it to have it talk about other stuff it to have it talk about other stuff it kept bringing me hey let's let's get kept bringing me hey let's let's get kept bringing me hey let's let's get back into AB Lincoln let's not talk back into AB Lincoln let's not talk back into AB Lincoln let's not talk about anime right now let's let's focus

about anime right now let's let's focus about anime right now let's let's focus on you know how anime relates to a on you know how anime relates to a on you know how anime relates to a Lincoln or whatever and it was such a Lincoln or whatever and it was such a Lincoln or whatever and it was such a fun way to tell it explicitly not to do fun way to tell it explicitly not to do fun way to tell it explicitly not to do the thing and some of the teachers the thing and some of the teachers the thing and some of the teachers thought that they might use that thought that they might use that thought that they might use that technique to actually say all right kids technique to actually say all right kids technique to actually say all right kids we know you're using chat gbt here's the we know you're using chat gbt here's the we know you're using chat gbt here's the prompt we want you to prompt we want you to prompt we want you to use and then it became like okay this is use and then it became like okay this is use and then it became like okay this is a that can help these kids get better a that can help these kids get better a that can help these kids get better and it'll be with them patient when I'm and it'll be with them patient when I'm and it'll be with them patient when I'm not there to sit with them yeah um not there to sit with them yeah um not there to sit with them yeah um actually I just seen I'm yeah I'm really actually I just seen I'm yeah I'm really actually I just seen I'm yeah I'm really interested in this in how it applies to interested in this in how it applies to interested in this in how it applies to education I haven't researched this education I haven't researched this education I haven't researched this extensively I know there are people extensively I know there are people extensively I know there are people whose whole lives right now is just whose whole lives right now is just whose whole lives right now is just thinking about this problem and my thinking about this problem and my thinking about this problem and my mother's a teacher so I hear a lot about mother's a teacher so I hear a lot about mother's a teacher so I hear a lot about it what how they're tackling it in her it what how they're tackling it in her it what how they're tackling it in her school like moving back to oral exams school like moving back to oral exams school like moving back to oral exams right trying to get teachers to right trying to get teachers to right trying to get teachers to integrate it into their lesson plans and integrate it into their lesson plans and integrate it into their lesson plans and use like various models not just chat use like various models not just chat use like various models not just chat GPT but teaching them about there these GPT but teaching them about there these GPT but teaching them about there these different models and different companies different models and different companies different models and different companies and let's talk about the motivations and let's talk about the motivations and let's talk about the motivations behind why people are building these behind why people are building these behind why people are building these things like really try to give them a things like really try to give them a things like really try to give them a holistic understanding of of what holistic understanding of of what holistic understanding of of what language models are which is I think the language models are which is I think the language models are which is I think the best thing you could do for a young kid best thing you could do for a young kid best thing you could do for a young kid at the moment um but I had seen even at the moment um but I had seen even at the moment um but I had seen even that here in the UK um a school has that here in the UK um a school has that here in the UK um a school has launched a program to have um I think launched a program to have um I think launched a program to have um I think it's just for the first two years of it's just for the first two years of it's just for the first two years of high school we have these weird like high school we have these weird like high school we have these weird like gcsc exam splits that people do um gcsc exam splits that people do um gcsc exam splits that people do um they're having a cohort of kids go they're having a cohort of kids go they're having a cohort of kids go through with mostly an AI tutor leading through with mostly an AI tutor leading through with mostly an AI tutor leading them through the program they'll have them through the program they'll have them through the program they'll have like human I guess support people but like human I guess support people but like human I guess support people but for the most part they're trying to test for the most part they're trying to test for the most part they're trying to test whether um an AI system can adjust to whether um an AI system can adjust to whether um an AI system can adjust to each kids's learning level and where each kids's learning level and where each kids's learning level and where they are so they're all working through

they are so they're all working through they are so they're all working through the same curriculum but it's adapting the same curriculum but it's adapting the same curriculum but it's adapting it's the way it's teaching and what it's the way it's teaching and what it's the way it's teaching and what content it's giving them at what time to content it's giving them at what time to content it's giving them at what time to each kid's level which I'm a little each kid's level which I'm a little each kid's level which I'm a little skeptical that it would like really be skeptical that it would like really be skeptical that it would like really be as good as teachers but I'm also like as good as teachers but I'm also like as good as teachers but I'm also like wow if that could work and everyone gets wow if that could work and everyone gets wow if that could work and everyone gets a more personalized education that's a more personalized education that's a more personalized education that's like suited to their pace of things like suited to their pace of things like suited to their pace of things um maybe it's actually pretty um maybe it's actually pretty um maybe it's actually pretty promising it sounds interesting I mean promising it sounds interesting I mean promising it sounds interesting I mean it's certainly scary to be practicing it's certainly scary to be practicing it's certainly scary to be practicing that on anyone but I think that the the that on anyone but I think that the the that on anyone but I think that the the issue is the lack of long-term memory issue is the lack of long-term memory issue is the lack of long-term memory like it's it's the it's a new Fresh like it's it's the it's a new Fresh like it's it's the it's a new Fresh person every time I wake up in the person every time I wake up in the person every time I wake up in the morning and ask it a question so unless morning and ask it a question so unless morning and ask it a question so unless you're keeping track and truly you're keeping track and truly you're keeping track and truly adjusting uh on a day byday and a adjusting uh on a day byday and a adjusting uh on a day byday and a year-by-year basis they're going to end year-by-year basis they're going to end year-by-year basis they're going to end up talking to the same mirror every day up talking to the same mirror every day up talking to the same mirror every day yes yeah and I was I was reading the the yes yeah and I was I was reading the the yes yeah and I was I was reading the the whole pitch on this on this this like AI whole pitch on this on this this like AI whole pitch on this on this this like AI T thing I went I hope you're really T thing I went I hope you're really T thing I went I hope you're really sophisticated architecture built around sophisticated architecture built around sophisticated architecture built around this where you've just got databases this where you've just got databases this where you've just got databases you're saving things to and doing really you're saving things to and doing really you're saving things to and doing really clever rag tricks to pull back the data clever rag tricks to pull back the data clever rag tricks to pull back the data at the right time so that was where I at the right time so that was where I at the right time so that was where I was like I doubt capabilities are good was like I doubt capabilities are good was like I doubt capabilities are good enough to do this at the moment but it's enough to do this at the moment but it's enough to do this at the moment but it's a nice future idea yeah the you say a nice future idea yeah the you say a nice future idea yeah the you say clever rag tricks but I'm starting to clever rag tricks but I'm starting to clever rag tricks but I'm starting to see and realize in my own usage of AI see and realize in my own usage of AI see and realize in my own usage of AI that the recency bias is such a problem that the recency bias is such a problem that the recency bias is such a problem that I'm leading it down the Primrose that I'm leading it down the Primrose that I'm leading it down the Primrose path and it's giving me answers because path and it's giving me answers because path and it's giving me answers because it feels like it's like it's almost like it feels like it's like it's almost like it feels like it's like it's almost like it knows I want the answer so it's it knows I want the answer so it's it knows I want the answer so it's trying to get me there but it's like trying to get me there but it's like trying to get me there but it's like it's not willing to challenge me oh yeah it's not willing to challenge me oh yeah it's not willing to challenge me oh yeah if it's not grounded so it'll just go if it's not grounded so it'll just go if it's not grounded so it'll just go with whatever bias that I have and head

with whatever bias that I have and head with whatever bias that I have and head in that direction it's a really eager in that direction it's a really eager in that direction it's a really eager improv partner that's willing to say yes improv partner that's willing to say yes improv partner that's willing to say yes and to anything right yeah that's true and to anything right yeah that's true and to anything right yeah that's true it's not very good at being critical it's not very good at being critical it's not very good at being critical I've been trying to build I've been I've been trying to build I've been I've been trying to build I've been trying to think about them as as ways to trying to think about them as as ways to trying to think about them as as ways to like embed certain kinds of thinking like embed certain kinds of thinking like embed certain kinds of thinking into into a tool that could guide you into into a tool that could guide you into into a tool that could guide you through a process so I'm at the moment through a process so I'm at the moment through a process so I'm at the moment I'm trying to build a prototype where um I'm trying to build a prototype where um I'm trying to build a prototype where um there's like a certain kind of um there's like a certain kind of um there's like a certain kind of um advisor I want when I'm writing an essay advisor I want when I'm writing an essay advisor I want when I'm writing an essay it would be like a critical College it would be like a critical College it would be like a critical College advisor like usually a professor who you advisor like usually a professor who you advisor like usually a professor who you would give them your essay and they would give them your essay and they would give them your essay and they would RP it apart they would also say would RP it apart they would also say would RP it apart they would also say you know okay you're trying to talk you know okay you're trying to talk you know okay you're trying to talk about these things here you really need about these things here you really need about these things here you really need to make sure you've read so and so's to make sure you've read so and so's to make sure you've read so and so's work well like if you if you're reading work well like if you if you're reading work well like if you if you're reading so and so's work you need to know that so and so's work you need to know that so and so's work you need to know that they were influenced by this person like they were influenced by this person like they were influenced by this person like that is the really valuable thing about that is the really valuable thing about that is the really valuable thing about good teachers and professors is they good teachers and professors is they good teachers and professors is they have all the domain context and then have all the domain context and then have all the domain context and then they can ask really good critical you they can ask really good critical you they can ask really good critical you know like youve also you know this isn't know like youve also you know this isn't know like youve also you know this isn't a convincing point and like you failed a convincing point and like you failed a convincing point and like you failed to support this with evidence that that to support this with evidence that that to support this with evidence that that kind of that kind of feedback I don't kind of that kind of feedback I don't kind of that kind of feedback I don't expect so this prototyp I'm building is expect so this prototyp I'm building is expect so this prototyp I'm building is trying to recreate that it's trying to trying to recreate that it's trying to trying to recreate that it's trying to get it to be a really critical reader of get it to be a really critical reader of get it to be a really critical reader of things I've written and to suggest holes things I've written and to suggest holes things I've written and to suggest holes in my knowledge or holes in the domain in my knowledge or holes in the domain in my knowledge or holes in the domain that I'm working in um but I'm trying to that I'm working in um but I'm trying to that I'm working in um but I'm trying to like guide it very narrowly to do that like guide it very narrowly to do that like guide it very narrowly to do that so I'm sort of saying okay I need you to so I'm sort of saying okay I need you to so I'm sort of saying okay I need you to like identify every single claim I've like identify every single claim I've like identify every single claim I've made in this piece of writing okay now I made in this piece of writing okay now I made in this piece of writing okay now I want you to check whether I've given want you to check whether I've given want you to check whether I've given evidence for every single one of those evidence for every single one of those evidence for every single one of those claims so I'm giving it in fact a very claims so I'm giving it in fact a very claims so I'm giving it in fact a very limited set of things it can do I'm not

limited set of things it can do I'm not limited set of things it can do I'm not just being like okay one prompt to the just being like okay one prompt to the just being like okay one prompt to the model and like see what it says because model and like see what it says because model and like see what it says because I expect it will just say those are I expect it will just say those are I expect it will just say those are great ideas you know go ahead and great ideas you know go ahead and great ideas you know go ahead and publish it or something I'm trying to publish it or something I'm trying to publish it or something I'm trying to instead ask it to just do very narrow instead ask it to just do very narrow instead ask it to just do very narrow specific tasks based on a critical specific tasks based on a critical specific tasks based on a critical framework that I'm coming up with it framework that I'm coming up with it framework that I'm coming up with it feels like that's right on the edge of feels like that's right on the edge of feels like that's right on the edge of what we're able to do without including what we're able to do without including what we're able to do without including agents like that's asking a lot of a agents like that's asking a lot of a agents like that's asking a lot of a model on its own to do when and I've model on its own to do when and I've model on its own to do when and I've been saying this declaratively when I been saying this declaratively when I been saying this declaratively when I give these presentations is it cannot give these presentations is it cannot give these presentations is it cannot reason like I think we just all need to reason like I think we just all need to reason like I think we just all need to accept that GPT is is an interesting and accept that GPT is is an interesting and accept that GPT is is an interesting and a powerful um architecture but it it it a powerful um architecture but it it it a powerful um architecture but it it it doesn't reason and there's a lot of doesn't reason and there's a lot of doesn't reason and there's a lot of argument like people seem split on that argument like people seem split on that argument like people seem split on that that that simple statement oh I think it that that simple statement oh I think it that that simple statement oh I think it does reason I'm like it makes me wonder does reason I'm like it makes me wonder does reason I'm like it makes me wonder then wait a second if it's just rolling then wait a second if it's just rolling then wait a second if it's just rolling a D20 to get to the next word am I also a D20 to get to the next word am I also a D20 to get to the next word am I also doing the same right do I reason or do I doing the same right do I reason or do I doing the same right do I reason or do I just like guess the next word yeah yeah just like guess the next word yeah yeah just like guess the next word yeah yeah like philosophy has never been so like philosophy has never been so like philosophy has never been so relevant to the programming Community as relevant to the programming Community as relevant to the programming Community as as with language models because this as with language models because this as with language models because this just feels like an endless debate I feel just feels like an endless debate I feel just feels like an endless debate I feel there's every other week some paper on there's every other week some paper on there's every other week some paper on Archive of people trying to prove that Archive of people trying to prove that Archive of people trying to prove that models do or don't Reason by you know models do or don't Reason by you know models do or don't Reason by you know conducting some experiment or they get conducting some experiment or they get conducting some experiment or they get it to like design a board game or it to like design a board game or it to like design a board game or extract from the rules of something um extract from the rules of something um extract from the rules of something um and and then no one can agree on what and and then no one can agree on what and and then no one can agree on what reasoning is we never agreed on it in reasoning is we never agreed on it in reasoning is we never agreed on it in the first place yeah uh I was I was I the first place yeah uh I was I was I the first place yeah uh I was I was I gave a talk about um about Ai and I use gave a talk about um about Ai and I use gave a talk about um about Ai and I use the analogy that I've been married to my the analogy that I've been married to my the analogy that I've been married to my wife now for 25 years and we finish each

wife now for 25 years and we finish each wife now for 25 years and we finish each other's sandwiches and like you just other's sandwiches and like you just other's sandwiches and like you just don't know what the next word is but she don't know what the next word is but she don't know what the next word is but she does because she's got 25 years of does because she's got 25 years of does because she's got 25 years of context so now I'm starting to wonder am context so now I'm starting to wonder am context so now I'm starting to wonder am I a good husband or am I just the I a good husband or am I just the I a good husband or am I just the statistical model of the most statistical model of the most statistical model of the most like likely next thing hanselman's going like likely next thing hanselman's going like likely next thing hanselman's going to say people are listening to this show to say people are listening to this show to say people are listening to this show and they're like ah it's classic and they're like ah it's classic and they're like ah it's classic Hanselman that's exactly what he would Hanselman that's exactly what he would Hanselman that's exactly what he would say because they've listened to enough say because they've listened to enough say because they've listened to enough episodes that they know that I'm just episodes that they know that I'm just episodes that they know that I'm just predictable which is not a good thing predictable which is not a good thing predictable which is not a good thing right but I think if we replaced you right but I think if we replaced you right but I think if we replaced you with a model that like always predicted with a model that like always predicted with a model that like always predicted the correct next word that you would the correct next word that you would the correct next word that you would have said based on all your previous have said based on all your previous have said based on all your previous records I mean I just can't imagine that records I mean I just can't imagine that records I mean I just can't imagine that being as compelling it gets back to the being as compelling it gets back to the being as compelling it gets back to the humanity thing it it matters that you're humanity thing it it matters that you're humanity thing it it matters that you're an actual living being with like Dynamic an actual living being with like Dynamic an actual living being with like Dynamic thoughts and complex opinions and a thoughts and complex opinions and a thoughts and complex opinions and a whole physical emotional life outside of whole physical emotional life outside of whole physical emotional life outside of the words you say on a podcast yeah what the words you say on a podcast yeah what the words you say on a podcast yeah what do you think about that term I think it do you think about that term I think it do you think about that term I think it was timet GBU and her uh friends that was timet GBU and her uh friends that was timet GBU and her uh friends that said stochastic parrot like that's a said stochastic parrot like that's a said stochastic parrot like that's a like it's a parrot parrots are great and like it's a parrot parrots are great and like it's a parrot parrots are great and it's crazy when parrots talk but the it's crazy when parrots talk but the it's crazy when parrots talk but the parrot is not as smart as you are giving parrot is not as smart as you are giving parrot is not as smart as you are giving it you know credit for yeah I know that it you know credit for yeah I know that it you know credit for yeah I know that was such a um controversial is the word was such a um controversial is the word was such a um controversial is the word I'm looking for phrase I feel that I'm looking for phrase I feel that I'm looking for phrase I feel that people reacted so strongly to that people reacted so strongly to that people reacted so strongly to that because even I thought it was a great a because even I thought it was a great a because even I thought it was a great a great line stochastic power and does great line stochastic power and does great line stochastic power and does sort of represent what's happening but sort of represent what's happening but sort of represent what's happening but then I feel like it also tries to then I feel like it also tries to then I feel like it also tries to simplify it in a way that makes it simplify it in a way that makes it simplify it in a way that makes it difficult to reason about the difficult to reason about the difficult to reason about the capabilities of models because even when capabilities of models because even when capabilities of models because even when we think of like oh all it's doing is we think of like oh all it's doing is we think of like oh all it's doing is predicting the next word we would predicting the next word we would predicting the next word we would imagine that's not a very capable imagine that's not a very capable imagine that's not a very capable machine but then it turns out it is machine but then it turns out it is machine but then it turns out it is quite capable in all these interesting

quite capable in all these interesting quite capable in all these interesting ways if you're clever about the design ways if you're clever about the design ways if you're clever about the design and the architecture and like the ux and the architecture and like the ux and the architecture and like the ux around it and finding interesting ways around it and finding interesting ways around it and finding interesting ways to use word next word completion to like to use word next word completion to like to use word next word completion to like apply to real tasks because I don't apply to real tasks because I don't apply to real tasks because I don't think anyone would say okay if you can think anyone would say okay if you can think anyone would say okay if you can predict the next word in a sentence predict the next word in a sentence predict the next word in a sentence you'll be able to like yeah build you'll be able to like yeah build you'll be able to like yeah build everything that we've built so far with everything that we've built so far with everything that we've built so far with it it's yeah yeah that is the part it it's yeah yeah that is the part it it's yeah yeah that is the part that's hard is that people want to and that's hard is that people want to and that's hard is that people want to and I'm the worst of worst of them people I'm the worst of worst of them people I'm the worst of worst of them people want to simplify things and come up with want to simplify things and come up with want to simplify things and come up with analogies and say just I talk about this analogies and say just I talk about this analogies and say just I talk about this a lot on the show oh it's just whatever a lot on the show oh it's just whatever a lot on the show oh it's just whatever okay so you're just taking the work of okay so you're just taking the work of okay so you're just taking the work of 50 years of machine learning and data 50 years of machine learning and data 50 years of machine learning and data science and statistics and math and science and statistics and math and science and statistics and math and you're putting it into a phrase and you're putting it into a phrase and you're putting it into a phrase and stochastic parrot like is like two solid stochastic parrot like is like two solid stochastic parrot like is like two solid words that are taking thousands of pages words that are taking thousands of pages words that are taking thousands of pages of people's PhD thesis and saying oh of people's PhD thesis and saying oh of people's PhD thesis and saying oh it's just tastic pair that's very it's just tastic pair that's very it's just tastic pair that's very minimizing yeah very minimizing I know minimizing yeah very minimizing I know minimizing yeah very minimizing I know because then we just wouldn't predict because then we just wouldn't predict because then we just wouldn't predict that it would be capable at all and then that it would be capable at all and then that it would be capable at all and then we underestimate it but we're in this I we underestimate it but we're in this I we underestimate it but we're in this I feel like we're in this like um I don't feel like we're in this like um I don't feel like we're in this like um I don't know the hard to quite to describe it know the hard to quite to describe it know the hard to quite to describe it but I say some say being like but I say some say being like but I say some say being like capabilities gaslighting is kind of capabilities gaslighting is kind of capabilities gaslighting is kind of what's Happening like we like that's a what's Happening like we like that's a what's Happening like we like that's a phrase sometimes use in talks because phrase sometimes use in talks because phrase sometimes use in talks because it's like we both are looking at it and it's like we both are looking at it and it's like we both are looking at it and going okay it looks like it's not going okay it looks like it's not going okay it looks like it's not actually that capable it's just a actually that capable it's just a actually that capable it's just a stochastic parrot but also it seems like stochastic parrot but also it seems like stochastic parrot but also it seems like it's incredibly capable and can like it's incredibly capable and can like it's incredibly capable and can like pass the GRE at the 99th percentile so pass the GRE at the 99th percentile so pass the GRE at the 99th percentile so like what in the world is going on here like what in the world is going on here like what in the world is going on here and like no one can figure it out we're and like no one can figure it out we're and like no one can figure it out we're just sort of being gas lit by it yeah just sort of being gas lit by it yeah just sort of being gas lit by it yeah it's it's it's difficult because they

it's it's it's difficult because they it's it's it's difficult because they say any sufficiently advanced technology say any sufficiently advanced technology say any sufficiently advanced technology is indistinguishable from Magic and like is indistinguishable from Magic and like is indistinguishable from Magic and like I could say oh yeah well this computer I could say oh yeah well this computer I could say oh yeah well this computer that you and I are talking on right now that you and I are talking on right now that you and I are talking on right now is just a rock that we Infuse with is just a rock that we Infuse with is just a rock that we Infuse with lightning and and you know lightning and and you know lightning and and you know it's really yeah but it's complicated uh it's really yeah but it's complicated uh it's really yeah but it's complicated uh I love that you're talking about these I love that you're talking about these I love that you're talking about these complicated things and that you have complicated things and that you have complicated things and that you have long form and I'm I'm shocked that long form and I'm I'm shocked that long form and I'm I'm shocked that there's long form material that you have there's long form material that you have there's long form material that you have in drafts post your drafts Maggie I want in drafts post your drafts Maggie I want in drafts post your drafts Maggie I want to I want to read this article that you to I want to read this article that you to I want to read this article that you never published I know I wish I had I never published I know I wish I had I never published I know I wish I had I knew at the time I had only gotten into knew at the time I had only gotten into knew at the time I had only gotten into the language model language model World the language model language model World the language model language model World a few months previously which often is a few months previously which often is a few months previously which often is like you have perfect beginner's mind like you have perfect beginner's mind like you have perfect beginner's mind you are can kind of see things fresh you are can kind of see things fresh you are can kind of see things fresh that other people can't see but I was so that other people can't see but I was so that other people can't see but I was so insecure because I was meeting all these insecure because I was meeting all these insecure because I was meeting all these people who had been working in it for people who had been working in it for people who had been working in it for years these phds and people working at years these phds and people working at years these phds and people working at open Ai and I was just like oh my gosh I open Ai and I was just like oh my gosh I open Ai and I was just like oh my gosh I cannot publicly say anything about cannot publicly say anything about cannot publicly say anything about language models I know nothing and now language models I know nothing and now language models I know nothing and now three years down the line I'm like oh my three years down the line I'm like oh my three years down the line I'm like oh my gosh I knew so gosh I knew so gosh I knew so much well I'm glad that you got to spend much well I'm glad that you got to spend much well I'm glad that you got to spend time with us today and I appreciate you time with us today and I appreciate you time with us today and I appreciate you coming on the podcast yeah thank you so coming on the podcast yeah thank you so coming on the podcast yeah thank you so much for having me you can learn more much for having me you can learn more much for having me you can learn more about Maggie Appleton and her work at about Maggie Appleton and her work at about Maggie Appleton and her work at Maggie aon.com check out her blog and Maggie aon.com check out her blog and Maggie aon.com check out her blog and her long form writings her long form writings her long form writings and this has been another episode of and this has been another episode of and this has been another episode of Hansel minutes and we'll see you again Hansel minutes and we'll see you again Hansel minutes and we'll see you again next week next week next week [Music]