The AI Revolution in Medicine with Dr. Peter Lee, President, Microsoft Research

You know, I love my terminal. I have You know, I love my terminal. I have completely customized my terminal with completely customized my terminal with completely customized my terminal with open source tools like Oh my Posh. I open source tools like Oh my Posh. I open source tools like Oh my Posh. I love the Windows terminal. I've been love the Windows terminal. I've been love the Windows terminal. I've been talking about it for a long time. But talking about it for a long time. But talking about it for a long time. But recently, Warp has launched on Windows. recently, Warp has launched on Windows. recently, Warp has launched on Windows. This is a new way of thinking about the This is a new way of thinking about the This is a new way of thinking about the terminal and I'm excited to see this terminal and I'm excited to see this terminal and I'm excited to see this newcomer into the Windows developer newcomer into the Windows developer newcomer into the Windows developer ecosystem. I think you should check it ecosystem. I think you should check it ecosystem. I think you should check it out. I definitely push back on a lot of out. I definitely push back on a lot of out. I definitely push back on a lot of uh sprinkling AI onto an application, uh sprinkling AI onto an application, uh sprinkling AI onto an application, but Warp is really recreating the but Warp is really recreating the but Warp is really recreating the command line. It's bringing AI and command line. It's bringing AI and command line. It's bringing AI and collaboration and a modern user collaboration and a modern user collaboration and a modern user interface to an essential tool. interface to an essential tool. interface to an essential tool. Historically, it's only been on Mac, but Historically, it's only been on Mac, but Historically, it's only been on Mac, but now Warp is bringing itself to Windows now Warp is bringing itself to Windows now Warp is bringing itself to Windows as a native application. It's a really, as a native application. It's a really, as a native application. It's a really, really interesting, really interesting really interesting, really interesting really interesting, really interesting product. I've spent a lot of time with product. I've spent a lot of time with product. I've spent a lot of time with the team uh personally spending the team uh personally spending the team uh personally spending literally hours with them to help them literally hours with them to help them literally hours with them to help them improve their experience. They're really improve their experience. They're really improve their experience. They're really passionate about delivering a great passionate about delivering a great passionate about delivering a great product. I think the experience is product. I think the experience is product. I think the experience is really good and it's it's a new take on really good and it's it's a new take on really good and it's it's a new take on the terminal. I think it adds a lot to the terminal. I think it adds a lot to the terminal. I think it adds a lot to the Windows developer experience. And if the Windows developer experience. And if the Windows developer experience. And if you maybe haven't been driving stick you maybe haven't been driving stick you maybe haven't been driving stick shift as much as I have, you haven't shift as much as I have, you haven't shift as much as I have, you haven't spent a lot of time at the terminal and spent a lot of time at the terminal and spent a lot of time at the terminal and you want a terminal with a modern user you want a terminal with a modern user you want a terminal with a modern user interface that lets you talk to it in interface that lets you talk to it in interface that lets you talk to it in plain language, you should check out plain language, you should check out plain language, you should check out Warp. It's really, really interesting Warp. It's really, really interesting Warp. It's really, really interesting and I've had a lot of fun running it on and I've had a lot of fun running it on and I've had a lot of fun running it on Windows. It's out now on Windows. You Windows. It's out now on Windows. You Windows. It's out now on Windows. You can get started for free at can get started for free at can get started for free at warp.dev/hancelminut. And as a listener, warp.dev/hancelminut. And as a listener, warp.dev/hancelminut. And as a listener, you get 50% off your first two months of you get 50% off your first two months of you get 50% off your first two months of Warp Pro with the code Hansel minutes.

Warp Pro with the code Hansel minutes. Warp Pro with the code Hansel minutes. Just all caps Hansel minutes. And Pro Just all caps Hansel minutes. And Pro Just all caps Hansel minutes. And Pro gives you over a thousand AI requests a gives you over a thousand AI requests a gives you over a thousand AI requests a month, which is certainly more than month, which is certainly more than month, which is certainly more than enough to uh to do some damage. Very, enough to uh to do some damage. Very, enough to uh to do some damage. Very, very cool product. I encourage you to very cool product. I encourage you to very cool product. I encourage you to check out Warp, the intelligent terminal check out Warp, the intelligent terminal check out Warp, the intelligent terminal with AI. And I want to thank them for with AI. And I want to thank them for with AI. And I want to thank them for sponsoring Hansel Minutes. You know, I sponsoring Hansel Minutes. You know, I sponsoring Hansel Minutes. You know, I don't bring a ton of sponsors on and I don't bring a ton of sponsors on and I don't bring a ton of sponsors on and I certainly don't bring them on as guests, certainly don't bring them on as guests, certainly don't bring them on as guests, but I thought that this was a cool tool but I thought that this was a cool tool but I thought that this was a cool tool that you all should check out. That's that you all should check out. That's that you all should check out. That's warp.dev/hancel warp.dev/hancel warp.dev/hancel [Music] [Music] [Music] minutes. Hi, I'm Scott Hansel and this minutes. Hi, I'm Scott Hansel and this minutes. Hi, I'm Scott Hansel and this is another episode of Hansel Minutes in is another episode of Hansel Minutes in is another episode of Hansel Minutes in association with the ACM Bikecast. Today association with the ACM Bikecast. Today association with the ACM Bikecast. Today I'm chatting with Dr. Peter Lee, I'm chatting with Dr. Peter Lee, I'm chatting with Dr. Peter Lee, president of Microsoft Research. You've president of Microsoft Research. You've president of Microsoft Research. You've got a resume as long as my arm, and it got a resume as long as my arm, and it got a resume as long as my arm, and it is an absolute joy to chat with you is an absolute joy to chat with you is an absolute joy to chat with you today, sir. Oh, uh, it's a joy for me to today, sir. Oh, uh, it's a joy for me to today, sir. Oh, uh, it's a joy for me to be here. I really appreciate it. Um, I be here. I really appreciate it. Um, I be here. I really appreciate it. Um, I want to go in a little bit of a want to go in a little bit of a want to go in a little bit of a controversial direction because I want controversial direction because I want controversial direction because I want to start out with the fact that I had a to start out with the fact that I had a to start out with the fact that I had a birthday last week and in this year, if birthday last week and in this year, if birthday last week and in this year, if I understand correctly, you're going to I understand correctly, you're going to I understand correctly, you're going to turn 65. Is that correct, sir? Yes. I turn 65. Is that correct, sir? Yes. I turn 65. Is that correct, sir? Yes. I Which I can't believe. Yeah. Yeah. I Which I can't believe. Yeah. Yeah. I Which I can't believe. Yeah. Yeah. I turned 51 last week and I have been turned 51 last week and I have been turned 51 last week and I have been reflecting on the the space that I've reflecting on the the space that I've reflecting on the the space that I've been in in the last, you know, I've got been in in the last, you know, I've got been in in the last, you know, I've got 32 years of software experience and and 32 years of software experience and and 32 years of software experience and and you have more and what does that feel you have more and what does that feel you have more and what does that feel like? Because like I've got I don't if like? Because like I've got I don't if like? Because like I've got I don't if you can see behind me, I've got a PDP11

you can see behind me, I've got a PDP11 you can see behind me, I've got a PDP11 uh that I built from a Raspberry Pi and uh that I built from a Raspberry Pi and uh that I built from a Raspberry Pi and I've been learning about that stuff. I've been learning about that stuff. I've been learning about that stuff. I've got my Commodore 64 that I had as a I've got my Commodore 64 that I had as a I've got my Commodore 64 that I had as a child, you know, like I'm really child, you know, like I'm really child, you know, like I'm really reflecting on how far this has come and reflecting on how far this has come and reflecting on how far this has come and with the experience and the space that with the experience and the space that with the experience and the space that you have occupied in computer science you have occupied in computer science you have occupied in computer science over the last, you know, 40 plus years. over the last, you know, 40 plus years. over the last, you know, 40 plus years. What does that feel like as we sit here What does that feel like as we sit here What does that feel like as we sit here on this m this AI moment? Yeah. U first on this m this AI moment? Yeah. U first on this m this AI moment? Yeah. U first off, um since you mentioned PDB11s and off, um since you mentioned PDB11s and off, um since you mentioned PDB11s and Commodore Amiggas, there's a lot of Commodore Amiggas, there's a lot of Commodore Amiggas, there's a lot of fondness in my heart for those. Um, I fondness in my heart for those. Um, I fondness in my heart for those. Um, I actually uh my first real paying job uh actually uh my first real paying job uh actually uh my first real paying job uh was a system administrator for uh a was a system administrator for uh a was a system administrator for uh a PDP11. And so I remember learning how to PDP11. And so I remember learning how to PDP11. And so I remember learning how to wire wrap the core wire wrap the core wire wrap the core memories there. And then uh for memories there. And then uh for memories there. And then uh for Commodore um Amigga uh I I um Amigga not Commodore um Amigga uh I I um Amigga not Commodore um Amigga uh I I um Amigga not 64. I um took uh time away from my 64. I um took uh time away from my 64. I um took uh time away from my graduate studies at Michigan to be part graduate studies at Michigan to be part graduate studies at Michigan to be part of a startup. And um uh and in that of a startup. And um uh and in that of a startup. And um uh and in that startup, one thing we were trying to do startup, one thing we were trying to do startup, one thing we were trying to do was um put productivity software like was um put productivity software like was um put productivity software like word processing and spreadsheets and word processing and spreadsheets and word processing and spreadsheets and paint programs onto Apple 2e and paint programs onto Apple 2e and paint programs onto Apple 2e and Commodore Amigga. So if you ever used Commodore Amigga. So if you ever used Commodore Amigga. So if you ever used Amigga, write Yeah. has my code. Uh Amigga, write Yeah. has my code. Uh Amigga, write Yeah. has my code. Uh really and um you know that company really and um you know that company really and um you know that company didn't really uh succeed. Um but you didn't really uh succeed. Um but you didn't really uh succeed. Um but you know we had to do things like make our know we had to do things like make our know we had to do things like make our own mouse because of course those own mouse because of course those own mouse because of course those computers didn't have mice at that time

computers didn't have mice at that time computers didn't have mice at that time but you needed a mouse to use those but you needed a mouse to use those but you needed a mouse to use those sorts of modern productivity programs. sorts of modern productivity programs. sorts of modern productivity programs. And so really a lot of fondness for And so really a lot of fondness for And so really a lot of fondness for those. Yeah. So, you know, if I I those. Yeah. So, you know, if I I those. Yeah. So, you know, if I I reflected about this and um you know, reflected about this and um you know, reflected about this and um you know, Scott, I assume you have a mobile phone, Scott, I assume you have a mobile phone, Scott, I assume you have a mobile phone, you know, in your pocket and of course, you know, in your pocket and of course, you know, in your pocket and of course, and the reason I think you have that is and the reason I think you have that is and the reason I think you have that is you look pretty comfortable and relaxed you look pretty comfortable and relaxed you look pretty comfortable and relaxed because, you know, I think nowadays if because, you know, I think nowadays if because, you know, I think nowadays if you don't if like you forgot it at home, you don't if like you forgot it at home, you don't if like you forgot it at home, um or you know, left it in a hotel room, um or you know, left it in a hotel room, um or you know, left it in a hotel room, you feel you feel bad. you feel naked you feel you feel bad. you feel naked you feel you feel bad. you feel naked and vulnerable uh like you can't really and vulnerable uh like you can't really and vulnerable uh like you can't really function properly and from a research function properly and from a research function properly and from a research perspective I count six major triumphs perspective I count six major triumphs perspective I count six major triumphs of computer science research that are in of computer science research that are in of computer science research that are in that mobile phone uh there's VSI design that mobile phone uh there's VSI design that mobile phone uh there's VSI design that emerged in the 1970s um that emerged in the 1970s um that emerged in the 1970s um there's a Linux or Unix inspired there's a Linux or Unix inspired there's a Linux or Unix inspired operating system kernel uh there's a operating system kernel uh there's a operating system kernel uh there's a softwaredefined mobile wireless softwaredefined mobile wireless softwaredefined mobile wireless radio and and so on. And and these radio and and so on. And and these radio and and so on. And and these things were hardcore academic research things were hardcore academic research things were hardcore academic research in computer science departments, great in computer science departments, great in computer science departments, great ones around the world, in great ones around the world, in great ones around the world, in great laboratories, you know, like Bell Labs laboratories, you know, like Bell Labs laboratories, you know, like Bell Labs and others. Uh they got published and and others. Uh they got published and and others. Uh they got published and then they made it then they made it then they made it into a suite of technologies that you into a suite of technologies that you into a suite of technologies that you literally feel like you can't live literally feel like you can't live literally feel like you can't live without all day, every day.

without all day, every day. without all day, every day. And you know, it's kind of amazing when And you know, it's kind of amazing when And you know, it's kind of amazing when you reflect on things like that. And so you reflect on things like that. And so you reflect on things like that. And so when I look at, for example, what's when I look at, for example, what's when I look at, for example, what's happening today with LLMs and generative happening today with LLMs and generative happening today with LLMs and generative AI, the question in my mind isn't one AI, the question in my mind isn't one AI, the question in my mind isn't one about AGI or not. It's about when and about AGI or not. It's about when and about AGI or not. It's about when and if, say, large language model technology if, say, large language model technology if, say, large language model technology will become yet another example of a will become yet another example of a will become yet another example of a technology that you can't bear to be technology that you can't bear to be technology that you can't bear to be without at any point of your waking without at any point of your waking without at any point of your waking hours. And I think it might happen. But, hours. And I think it might happen. But, hours. And I think it might happen. But, you know, just, you know, as I get you know, just, you know, as I get you know, just, you know, as I get closer to retirement at some point and closer to retirement at some point and closer to retirement at some point and just reflect on everything that's just reflect on everything that's just reflect on everything that's happened, it it's really incredible uh happened, it it's really incredible uh happened, it it's really incredible uh how much, you know, how much things have how much, you know, how much things have how much, you know, how much things have evolved to the point where we really evolved to the point where we really evolved to the point where we really literally depend on them all the time literally depend on them all the time literally depend on them all the time for our mental health and sanity. Yeah, for our mental health and sanity. Yeah, for our mental health and sanity. Yeah, that that idea of like not being able to that that idea of like not being able to that that idea of like not being able to live without it is such an interesting live without it is such an interesting live without it is such an interesting way of phrasing it. Uh, I have a very way of phrasing it. Uh, I have a very way of phrasing it. Uh, I have a very personal relationship with my phone personal relationship with my phone personal relationship with my phone because it also runs my artificial because it also runs my artificial because it also runs my artificial pancreas and I was in Johannesburg last pancreas and I was in Johannesburg last pancreas and I was in Johannesburg last week for two weeks and I bought a backup week for two weeks and I bought a backup week for two weeks and I bought a backup phone because even though I can drive phone because even though I can drive phone because even though I can drive stick shift and just take needles and stick shift and just take needles and stick shift and just take needles and take blood sugar, you know, with my take blood sugar, you know, with my take blood sugar, you know, with my finger like I could do that and I did finger like I could do that and I did finger like I could do that and I did that for 20 years. Yes. Why is it now that for 20 years. Yes. Why is it now that for 20 years. Yes. Why is it now that I feel like I cannot survive that I feel like I cannot survive that I feel like I cannot survive without I have to have the phone within without I have to have the phone within without I have to have the phone within Bluetooth range in, you know, near me 24 Bluetooth range in, you know, near me 24 Bluetooth range in, you know, near me 24 hours a day. And if I lost the phone or hours a day. And if I lost the phone or hours a day. And if I lost the phone or if it was dropped or stolen, I would if it was dropped or stolen, I would if it was dropped or stolen, I would immediately need to hook another phone

immediately need to hook another phone immediately need to hook another phone up because I want my continuous glucose up because I want my continuous glucose up because I want my continuous glucose meter. And my wife, who's not diabetic, meter. And my wife, who's not diabetic, meter. And my wife, who's not diabetic, but is a nurse said, "Well, you know, but is a nurse said, "Well, you know, but is a nurse said, "Well, you know, you could just take needles and and you could just take needles and and you could just take needles and and stick your finger like you did back in stick your finger like you did back in stick your finger like you did back in the 90s." And I was like, "That's the 90s." And I was like, "That's the 90s." And I was like, "That's unthinkable. How dare you?" Um that's unthinkable. How dare you?" Um that's unthinkable. How dare you?" Um that's actually something called the flip. Um actually something called the flip. Um actually something called the flip. Um and um you see the flip u most vividly and um you see the flip u most vividly and um you see the flip u most vividly uh in the technology adoption history in uh in the technology adoption history in uh in the technology adoption history in healthcare and medicine. And my favorite healthcare and medicine. And my favorite healthcare and medicine. And my favorite example of that is um ultrasound. So, example of that is um ultrasound. So, example of that is um ultrasound. So, you know, ultrasound had been invented you know, ultrasound had been invented you know, ultrasound had been invented um quite a long time ago, but it wasn't um quite a long time ago, but it wasn't um quite a long time ago, but it wasn't until the late until the late until the late 1950s that people technologists proposed 1950s that people technologists proposed 1950s that people technologists proposed uh that ultrasound might be a good uh that ultrasound might be a good uh that ultrasound might be a good diagnostic tool in the practice of diagnostic tool in the practice of diagnostic tool in the practice of medicine. And so you know if you're a medicine. And so you know if you're a medicine. And so you know if you're a pregnant woman and you go to see your pregnant woman and you go to see your pregnant woman and you go to see your gynecologist or obstitrician gynecologist or obstitrician gynecologist or obstitrician um you um you um you will see ultrasound used to examine the will see ultrasound used to examine the will see ultrasound used to examine the your uh unborn fetus and the your uh unborn fetus and the your uh unborn fetus and the reproductive system of the of the reproductive system of the of the reproductive system of the of the mother. But in the late 1950s when this mother. But in the late 1950s when this mother. But in the late 1950s when this was first proposed, it scared people. Is was first proposed, it scared people. Is was first proposed, it scared people. Is this going to damage my baby? is it this going to damage my baby? is it this going to damage my baby? is it going to damage my ability uh to going to damage my ability uh to going to damage my ability uh to reproduce? Uh our doctors trained to reproduce? Uh our doctors trained to reproduce? Uh our doctors trained to read these grainy images and and all read these grainy images and and all read these grainy images and and all sorts of problems. And so it took about sorts of problems. And so it took about sorts of problems. And so it took about a decade a decade a decade before uh ultrasound became the standard before uh ultrasound became the standard before uh ultrasound became the standard of care. And so now what do I mean by

of care. And so now what do I mean by of care. And so now what do I mean by the flip? Well, today if you are a the flip? Well, today if you are a the flip? Well, today if you are a pregnant woman and then you go to your pregnant woman and then you go to your pregnant woman and then you go to your obstitrician and your obstitrician says, obstitrician and your obstitrician says, obstitrician and your obstitrician says, "Ah, I don't believe in that fancy uh "Ah, I don't believe in that fancy uh "Ah, I don't believe in that fancy uh ultrasound technology. I'm going to put ultrasound technology. I'm going to put ultrasound technology. I'm going to put my hands on your belly and use manual my hands on your belly and use manual my hands on your belly and use manual palpation. And not only would you be palpation. And not only would you be palpation. And not only would you be horrified at that, but you might even horrified at that, but you might even horrified at that, but you might even report that, you know, as as report that, you know, as as report that, you know, as as malpractice. And so expectations, not malpractice. And so expectations, not malpractice. And so expectations, not only in the standard of care, but in only in the standard of care, but in only in the standard of care, but in patient demands and expectations patient demands and expectations patient demands and expectations uh consistently flip in the history of uh consistently flip in the history of uh consistently flip in the history of technology adoption medicine. And you technology adoption medicine. And you technology adoption medicine. And you know, your example is is another example know, your example is is another example know, your example is is another example of how things flip. You flip from being of how things flip. You flip from being of how things flip. You flip from being able to, you know, drive manually and able to, you know, drive manually and able to, you know, drive manually and monitor your own blood sugar levels to a monitor your own blood sugar levels to a monitor your own blood sugar levels to a feeling that of why would you do that? feeling that of why would you do that? feeling that of why would you do that? It's so primitive. It's so inexact, so It's so primitive. It's so inexact, so It's so primitive. It's so inexact, so dangerous, um, unconscionable, uh, and dangerous, um, unconscionable, uh, and dangerous, um, unconscionable, uh, and in fact irresponsible to do it that way. in fact irresponsible to do it that way. in fact irresponsible to do it that way. Right. I'm curious though that I I find Right. I'm curious though that I I find Right. I'm curious though that I I find myself feeling like old man who shakes myself feeling like old man who shakes myself feeling like old man who shakes fist at cloud because I feel like fist at cloud because I feel like fist at cloud because I feel like there's the young people are saying it's there's the young people are saying it's there's the young people are saying it's going to change the world and if you're going to change the world and if you're going to change the world and if you're you know okay boomer if you're against you know okay boomer if you're against you know okay boomer if you're against it then if you're not with us you're it then if you're not with us you're it then if you're not with us you're against us. But I I remember when like against us. But I I remember when like against us. But I I remember when like the TI83 and the TI81 calculators came the TI83 and the TI81 calculators came the TI83 and the TI81 calculators came out and the math teachers were like ah out and the math teachers were like ah out and the math teachers were like ah no one's going to be able to do math in no one's going to be able to do math in no one's going to be able to do math in their head anymore cuz you won't always their head anymore cuz you won't always their head anymore cuz you won't always have a calculator young man and here we have a calculator young man and here we have a calculator young man and here we are with a pocket supercomput. I am are with a pocket supercomput. I am are with a pocket supercomput. I am finding my myself questioning my own

finding my myself questioning my own finding my myself questioning my own opinions about tech based on my age and opinions about tech based on my age and opinions about tech based on my age and my generation and how I and I'm really my generation and how I and I'm really my generation and how I and I'm really trying to be introspective like I get trying to be introspective like I get trying to be introspective like I get that it's going to change the world but that it's going to change the world but that it's going to change the world but I also think that cognitively we don't I also think that cognitively we don't I also think that cognitively we don't know how these are going to change our know how these are going to change our know how these are going to change our brains and I don't know if having a brains and I don't know if having a brains and I don't know if having a pocket supercomputer has changed our pocket supercomputer has changed our pocket supercomputer has changed our brains for the positive. brains for the positive. brains for the positive. Yeah. You know I Yeah. You know I Yeah. You know I um I'm always a little bit more um I'm always a little bit more um I'm always a little bit more circumspect. It's easy as a as two circumspect. It's easy as a as two circumspect. It's easy as a as two people who are tech people who are tech people who are tech executives you know to look at all the executives you know to look at all the executives you know to look at all the historical examples of you know so the historical examples of you know so the historical examples of you know so the adoption of advanced technologies that adoption of advanced technologies that adoption of advanced technologies that are really enabling uh there is a are really enabling uh there is a are really enabling uh there is a consistent pattern and so you could consistent pattern and so you could consistent pattern and so you could always argue as we in tech do uh that always argue as we in tech do uh that always argue as we in tech do uh that the right side of history uh is that the right side of history uh is that the right side of history uh is that these things advance and the world gets these things advance and the world gets these things advance and the world gets better and better and I I think that's better and better and I I think that's better and better and I I think that's generally true, but I think it's also generally true, but I think it's also generally true, but I think it's also true and it's worth some reflection that true and it's worth some reflection that true and it's worth some reflection that that we lose some things along the way.

that we lose some things along the way. that we lose some things along the way. And and that isn't to say that, you And and that isn't to say that, you And and that isn't to say that, you know, the world isn't getting better and know, the world isn't getting better and know, the world isn't getting better and life isn't getting better. Uh because I life isn't getting better. Uh because I life isn't getting better. Uh because I think it it is by any measure. Um, but I think it it is by any measure. Um, but I think it it is by any measure. Um, but I think it's wrong to deny that we, you think it's wrong to deny that we, you think it's wrong to deny that we, you know, to deny that we're losing some know, to deny that we're losing some know, to deny that we're losing some things and and some skills and some things and and some skills and some things and and some skills and some abilities that maybe were important. abilities that maybe were important. abilities that maybe were important. And, you know, one thing I would say And, you know, one thing I would say And, you know, one thing I would say about especially in this new AI era is I about especially in this new AI era is I about especially in this new AI era is I I do see that the world's leading I do see that the world's leading I do see that the world's leading thinkers are actually earnestly trying thinkers are actually earnestly trying thinkers are actually earnestly trying to be thoughtful about this. You know, to be thoughtful about this. You know, to be thoughtful about this. You know, that there is a big debate. In fact, that there is a big debate. In fact, that there is a big debate. In fact, it's it's a debate of leading thinkers it's it's a debate of leading thinkers it's it's a debate of leading thinkers that I haven't seen since the human that I haven't seen since the human that I haven't seen since the human genome was mapped. You know, when when genome was mapped. You know, when when genome was mapped. You know, when when we finished mapping the human genome as we finished mapping the human genome as we finished mapping the human genome as a scientific community, it sparked a a scientific community, it sparked a a scientific community, it sparked a huge debate about what would this mean? huge debate about what would this mean? huge debate about what would this mean? Um, and you know, would genetic Um, and you know, would genetic Um, and you know, would genetic engineering be a good or a bad thing? engineering be a good or a bad thing? engineering be a good or a bad thing? And the fact that there are so many And the fact that there are so many And the fact that there are so many leading thinkers and the whole uh leading thinkers and the whole uh leading thinkers and the whole uh academic and research area of academic and research area of academic and research area of bioeththics really just mushroomed uh bioeththics really just mushroomed uh bioeththics really just mushroomed uh into hugely important kinds of uh of into hugely important kinds of uh of into hugely important kinds of uh of thought leadership and research. I think thought leadership and research. I think thought leadership and research. I think that's helped us a lot to gain as much that's helped us a lot to gain as much that's helped us a lot to gain as much of the goodness out of our understanding of the goodness out of our understanding of the goodness out of our understanding of genetics while mitigating the of genetics while mitigating the of genetics while mitigating the potential downstream harms and risks.

potential downstream harms and risks. potential downstream harms and risks. And I I see the same intensity of debate And I I see the same intensity of debate And I I see the same intensity of debate going on with AI as people are trying to going on with AI as people are trying to going on with AI as people are trying to grapple with what what this means. And grapple with what what this means. And grapple with what what this means. And so, so I think that that's a a good so, so I think that that's a a good so, so I think that that's a a good thing and it actually makes me it makes thing and it actually makes me it makes thing and it actually makes me it makes it easier for me to be optimistic about it easier for me to be optimistic about it easier for me to be optimistic about what's happening right now. Yeah. As what's happening right now. Yeah. As what's happening right now. Yeah. As someone who is and was a professor of someone who is and was a professor of someone who is and was a professor of computer science for many many years, computer science for many many years, computer science for many many years, did you plug ethics in and did you have did you plug ethics in and did you have did you plug ethics in and did you have people thinking about the tech? I mean, people thinking about the tech? I mean, people thinking about the tech? I mean, sometimes it's fun to just do techy sometimes it's fun to just do techy sometimes it's fun to just do techy stuff for techy reasons, but I've always stuff for techy reasons, but I've always stuff for techy reasons, but I've always come back to are we making someone's come back to are we making someone's come back to are we making someone's life better by doing this or are we life better by doing this or are we life better by doing this or are we making someone's my life demonstrabably making someone's my life demonstrabably making someone's my life demonstrabably worse? But a lot of people when I do worse? But a lot of people when I do worse? But a lot of people when I do these informal polls I say all right who these informal polls I say all right who these informal polls I say all right who here has ever taken a computer science here has ever taken a computer science here has ever taken a computer science ethics class the hands are not going up ethics class the hands are not going up ethics class the hands are not going up a majority or plurality uh it feels like a majority or plurality uh it feels like a majority or plurality uh it feels like we're we're not teaching computer we're we're not teaching computer we're we're not teaching computer science ethics and it feels like it's science ethics and it feels like it's science ethics and it feels like it's even more important in a post GPT4 even more important in a post GPT4 even more important in a post GPT4 world. Yeah, I I have to admit uh in world. Yeah, I I have to admit uh in world. Yeah, I I have to admit uh in earlier in my career, it really wasn't a earlier in my career, it really wasn't a earlier in my career, it really wasn't a serious thought. And in fact, if serious thought. And in fact, if serious thought. And in fact, if anything, I tried hard to adopt, you anything, I tried hard to adopt, you anything, I tried hard to adopt, you know, the sil Silicon Valley, you know, know, the sil Silicon Valley, you know, know, the sil Silicon Valley, you know, stereotype ethos of tech is good and stereotype ethos of tech is good and stereotype ethos of tech is good and more tech is better. Um, and you know, more tech is better. Um, and you know, more tech is better. Um, and you know, and just the belief that technology, you and just the belief that technology, you and just the belief that technology, you know, could be the answer to, you know, know, could be the answer to, you know, know, could be the answer to, you know, all of our problems and that we can all of our problems and that we can all of our problems and that we can solve all problems through solve all problems through solve all problems through technology. I I do think we've gotten to technology. I I do think we've gotten to technology. I I do think we've gotten to such a

such a such a more enlightened state about this. We've more enlightened state about this. We've more enlightened state about this. We've and I think a lot of that has come and I think a lot of that has come and I think a lot of that has come through lessons learned the hard way. through lessons learned the hard way. through lessons learned the hard way. you know, we we've seen so clearly uh you know, we we've seen so clearly uh you know, we we've seen so clearly uh that tech isn't different than any other that tech isn't different than any other that tech isn't different than any other technology in being dual-edged. Um, and technology in being dual-edged. Um, and technology in being dual-edged. Um, and in fact, it's more interesting and has in fact, it's more interesting and has in fact, it's more interesting and has greater positive and negative potential greater positive and negative potential greater positive and negative potential because information technology can be because information technology can be because information technology can be democratized. It can be given to democratized. It can be given to democratized. It can be given to literally everyone. You know, unlike literally everyone. You know, unlike literally everyone. You know, unlike genetic engineering where you have to genetic engineering where you have to genetic engineering where you have to have a multi-million dollar wet lab to have a multi-million dollar wet lab to have a multi-million dollar wet lab to do anything, here literally every person do anything, here literally every person do anything, here literally every person on the planet could gain access and and on the planet could gain access and and on the planet could gain access and and harness this for both good and evil. And harness this for both good and evil. And harness this for both good and evil. And so I think we it ethics has become a so I think we it ethics has become a so I think we it ethics has become a much bigger deal. much bigger deal. much bigger deal. Um whereas I think earlier in my career Um whereas I think earlier in my career Um whereas I think earlier in my career personally, I think it was an age of personally, I think it was an age of personally, I think it was an age of innocence, you know, where we only saw innocence, you know, where we only saw innocence, you know, where we only saw the goodness in things. Yeah. Yeah. I the goodness in things. Yeah. Yeah. I the goodness in things. Yeah. Yeah. I really appreciate the democratization of really appreciate the democratization of really appreciate the democratization of it. Like open source itself, like the it. Like open source itself, like the it. Like open source itself, like the the pancreas that I use is a toolkit the pancreas that I use is a toolkit the pancreas that I use is a toolkit that one could build from scratch and I that one could build from scratch and I that one could build from scratch and I did and that anyone could go and do did and that anyone could go and do did and that anyone could go and do that. And now we're seeing open weights that. And now we're seeing open weights that. And now we're seeing open weights and open models. Uh what's your thinking and open models. Uh what's your thinking and open models. Uh what's your thinking about frontier models that are closed about frontier models that are closed about frontier models that are closed versus frontier models that are more versus frontier models that are more versus frontier models that are more more open uh about where they came from, more open uh about where they came from, more open uh about where they came from, what they were trained on, and the what they were trained on, and the what they were trained on, and the corpus that uh that built them up. Yeah, corpus that uh that built them up. Yeah, corpus that uh that built them up. Yeah, I think that this is also evolving I think that this is also evolving I think that this is also evolving really quickly. Um, you know, the thing really quickly. Um, you know, the thing really quickly. Um, you know, the thing that's so interesting to me is the cost that's so interesting to me is the cost that's so interesting to me is the cost of doing something first versus the cost

of doing something first versus the cost of doing something first versus the cost of doing something 10th, you know, and of doing something 10th, you know, and of doing something 10th, you know, and so if you are Open AI or, you know, one so if you are Open AI or, you know, one so if you are Open AI or, you know, one of OpenAI's, you know, frontier of OpenAI's, you know, frontier of OpenAI's, you know, frontier uh, you know, competitors, you're trying uh, you know, competitors, you're trying uh, you know, competitors, you're trying to get to certain uh, levels of to get to certain uh, levels of to get to certain uh, levels of intelligent capability first. And and intelligent capability first. And and intelligent capability first. And and there you can see a pretty there you can see a pretty there you can see a pretty uh methodical investment strategy. You uh methodical investment strategy. You uh methodical investment strategy. You know, to get to GPT2 maybe requires, you know, to get to GPT2 maybe requires, you know, to get to GPT2 maybe requires, you know, on the order of 5 to10 million of know, on the order of 5 to10 million of know, on the order of 5 to10 million of compute cost to train a model at that compute cost to train a model at that compute cost to train a model at that level. And GPT2, you know, allows you to level. And GPT2, you know, allows you to level. And GPT2, you know, allows you to see perfect loss curves. So that gives see perfect loss curves. So that gives see perfect loss curves. So that gives you confidence that further scaling is you confidence that further scaling is you confidence that further scaling is going to get you somewhere. And then to going to get you somewhere. And then to going to get you somewhere. And then to get to GPD3 get to GPD3 get to GPD3 level and to do that first pretty level and to do that first pretty level and to do that first pretty consistently in whatever domain you're consistently in whatever domain you're consistently in whatever domain you're trying to train your A trying to train your A trying to train your A models consistently requires about 10x models consistently requires about 10x models consistently requires about 10x the compute cost. So now you're talking the compute cost. So now you're talking the compute cost. So now you're talking 50 to$100 million. And then to get to 50 to$100 million. And then to get to 50 to$100 million. And then to get to GPT4 GPT4 GPT4 level, you know, whether it's in level, you know, whether it's in level, you know, whether it's in language or molecular dynamics or, you language or molecular dynamics or, you language or molecular dynamics or, you know, weather modeling, whatever, uh, know, weather modeling, whatever, uh, know, weather modeling, whatever, uh, seems to take yet another 10x leap if seems to take yet another 10x leap if seems to take yet another 10x leap if you want to get there first. So now, you you want to get there first. So now, you you want to get there first. So now, you know, you're starting to get close to a know, you're starting to get close to a know, you're starting to get close to a billion dollars. Um, and so on. The billion dollars. Um, and so on. The billion dollars. Um, and so on. The thing that's so interesting is like in thing that's so interesting is like in thing that's so interesting is like in all things that have to do with all things that have to do with all things that have to do with technological technological technological invention, once everyone once smart

invention, once everyone once smart invention, once everyone once smart people see that something is possible, people see that something is possible, people see that something is possible, uh then it becomes so much easier for uh then it becomes so much easier for uh then it becomes so much easier for the second and third and fourth and 10th the second and third and fourth and 10th the second and third and fourth and 10th people to do it. And what we're seeing people to do it. And what we're seeing people to do it. And what we're seeing in the industry is that not being first in the industry is that not being first in the industry is that not being first means that you can also, of course, you means that you can also, of course, you means that you can also, of course, you don't benefit from being first, but you don't benefit from being first, but you don't benefit from being first, but you benefit from being able to be smarter benefit from being able to be smarter benefit from being able to be smarter and a hell of a lot cheaper to get to and a hell of a lot cheaper to get to and a hell of a lot cheaper to get to the same state. Uh, not that getting to the same state. Uh, not that getting to the same state. Uh, not that getting to a GPG4ass base model is cheap. It's a GPG4ass base model is cheap. It's a GPG4ass base model is cheap. It's still quite expensive, but it's a lot still quite expensive, but it's a lot still quite expensive, but it's a lot cheaper than it took for the very first cheaper than it took for the very first cheaper than it took for the very first innovators to get there. So, I think innovators to get there. So, I think innovators to get there. So, I think that's just a pattern that this is the that's just a pattern that this is the that's just a pattern that this is the pattern of tech that you and I have pattern of tech that you and I have pattern of tech that you and I have lived through our entire careers. Um, so lived through our entire careers. Um, so lived through our entire careers. Um, so we shouldn't be surprised by this, but we shouldn't be surprised by this, but we shouldn't be surprised by this, but just to see it playing out with just just to see it playing out with just just to see it playing out with just this stunning speed this stunning speed this stunning speed is just just incredible. And is just just incredible. And is just just incredible. And business-wise, the question is, you business-wise, the question is, you business-wise, the question is, you know, uh there's always been a big know, uh there's always been a big know, uh there's always been a big premium in our business in being premium in our business in being premium in our business in being first. Uh now the question is, you know, first. Uh now the question is, you know, first. Uh now the question is, you know, given the rapid pace of change and uh given the rapid pace of change and uh given the rapid pace of change and uh and evolution, uh is there is still the and evolution, uh is there is still the and evolution, uh is there is still the same amount of value in being first as same amount of value in being first as same amount of value in being first as there has always been?

there has always been? there has always been? The uh the the idea that we shouldn't be The uh the the idea that we shouldn't be The uh the the idea that we shouldn't be surprised, but we consistently are surprised, but we consistently are surprised, but we consistently are surprised. Uh I uh you know like I surprised. Uh I uh you know like I surprised. Uh I uh you know like I remember the first time that I saw you remember the first time that I saw you remember the first time that I saw you know a chat you know a chatbot for lack know a chat you know a chatbot for lack know a chat you know a chatbot for lack of a better words and now like I'm of a better words and now like I'm of a better words and now like I'm looking at my phone I've got an app looking at my phone I've got an app looking at my phone I've got an app called MLC chat and I've got 535 mini uh called MLC chat and I've got 535 mini uh called MLC chat and I've got 535 mini uh it's a uh a 4bit quantized uh F6 F6 F-16 it's a uh a 4bit quantized uh F6 F6 F-16 it's a uh a 4bit quantized uh F6 F6 F-16 but I'm having a conversation in but I'm having a conversation in but I'm having a conversation in airplane mode on a you know on a 15 tops airplane mode on a you know on a 15 tops airplane mode on a you know on a 15 tops iPhone. That seems insane to me. Uh, and iPhone. That seems insane to me. Uh, and iPhone. That seems insane to me. Uh, and I get it and I understand it, but now I get it and I understand it, but now I get it and I understand it, but now we're we're at a point now where the we're we're at a point now where the we're we're at a point now where the full stack, you know, you always hear full stack, you know, you always hear full stack, you know, you always hear about the full stack engineer is so deep about the full stack engineer is so deep about the full stack engineer is so deep that I find myself going back to that I find myself going back to that I find myself going back to building things from original parts just building things from original parts just building things from original parts just so I can remind myself. Like I've got a so I can remind myself. Like I've got a so I can remind myself. Like I've got a an Apple 1 that I'm building from 7400 an Apple 1 that I'm building from 7400 an Apple 1 that I'm building from 7400 series chips just to kind of get my series chips just to kind of get my series chips just to kind of get my hands back in the silicon because the hands back in the silicon because the hands back in the silicon because the the stack is so high. I'm trying to get the stack is so high. I'm trying to get the stack is so high. I'm trying to get my hands dirty so that I can emotionally my hands dirty so that I can emotionally my hands dirty so that I can emotionally like accept that there's a an SLM on my like accept that there's a an SLM on my like accept that there's a an SLM on my phone that works in airplane mode, you phone that works in airplane mode, you phone that works in airplane mode, you know. By the way, I'll take one of those know. By the way, I'll take one of those know. By the way, I'll take one of those Apple ones if you make two of them.

Apple ones if you make two of them. Apple ones if you make two of them. Yeah. Yeah, those are great. Yeah, it's Yeah. Yeah, those are great. Yeah, it's Yeah. Yeah, those are great. Yeah, it's a company called Smarty Kit. I'll send a company called Smarty Kit. I'll send a company called Smarty Kit. I'll send that to you. I um uh you know there's that to you. I um uh you know there's that to you. I um uh you know there's something that you're saying there something that you're saying there something that you're saying there though that uh is another reflection though that uh is another reflection though that uh is another reflection just looking back at my just looking back at my just looking back at my career and I think you and I uh when we career and I think you and I uh when we career and I think you and I uh when we started in all this we were able to wrap started in all this we were able to wrap started in all this we were able to wrap our heads around and know the intricate our heads around and know the intricate our heads around and know the intricate details of the entire stack end to end details of the entire stack end to end details of the entire stack end to end from the silicon or in my case the wire from the silicon or in my case the wire from the silicon or in my case the wire wraps you know all the way uh to uh all wraps you know all the way uh to uh all wraps you know all the way uh to uh all the code, every line of code in the the code, every line of code in the the code, every line of code in the applications. applications. applications. And things are just so much more And things are just so much more And things are just so much more complicated now. And when we think about complicated now. And when we think about complicated now. And when we think about what AI is going to enable, I I think AI what AI is going to enable, I I think AI what AI is going to enable, I I think AI is very quickly going to enable us to is very quickly going to enable us to is very quickly going to enable us to construct systems that are so complex construct systems that are so complex construct systems that are so complex that they will really completely defy that they will really completely defy that they will really completely defy any ability for humans to comprehend any ability for humans to comprehend any ability for humans to comprehend them fully.

them fully. them fully. um the in research where I think this um the in research where I think this um the in research where I think this might happen first uh right now u might happen first uh right now u might happen first uh right now u there's a vibrant set of researchers there's a vibrant set of researchers there's a vibrant set of researchers that are using generative AI to write that are using generative AI to write that are using generative AI to write mathematical proofs and when you ask an mathematical proofs and when you ask an mathematical proofs and when you ask an AI system to write a mathematical proof AI system to write a mathematical proof AI system to write a mathematical proof uh you generally speaking ask it to uh you generally speaking ask it to uh you generally speaking ask it to write it in a proof language a popular write it in a proof language a popular write it in a proof language a popular one is called lean uh there are others one is called lean uh there are others one is called lean uh there are others as well the interesting thing about a as well the interesting thing about a as well the interesting thing about a proof language is that there's set up so proof language is that there's set up so proof language is that there's set up so that you can use a simple type checker that you can use a simple type checker that you can use a simple type checker just like you would have in any just like you would have in any just like you would have in any programming language and these proof programming language and these proof programming language and these proof languages are set up so that if they languages are set up so that if they languages are set up so that if they type check you know for certain that the type check you know for certain that the type check you know for certain that the proof is proof is proof is valid and so I foresee in much less than valid and so I foresee in much less than valid and so I foresee in much less than 5 years that we'll have an AI system 5 years that we'll have an AI system 5 years that we'll have an AI system generate a proof of some mathematical generate a proof of some mathematical generate a proof of some mathematical theorem we'll be able to type check that theorem we'll be able to type check that theorem we'll be able to type check that proof to know that that proof is proof to know that that proof is proof to know that that proof is absolutely valid and correct But that absolutely valid and correct But that absolutely valid and correct But that proof itself might proof itself might proof itself might defy any ability for any human being, defy any ability for any human being, defy any ability for any human being, even the world's smartest human beings even the world's smartest human beings even the world's smartest human beings to understand it. And that's sort of to understand it. And that's sort of to understand it. And that's sort of like a point example of AI getting us to like a point example of AI getting us to like a point example of AI getting us to a place where it we're able to construct a place where it we're able to construct a place where it we're able to construct things, build things that work and we things, build things that work and we things, build things that work and we know, we can see them work, we can know, we can see them work, we can know, we can see them work, we can validate that they work, but we won't validate that they work, but we won't validate that they work, but we won't know how or why. And I think going to be know how or why. And I think going to be know how or why. And I think going to be when people ask me about AGI or super

when people ask me about AGI or super when people ask me about AGI or super intelligence that will be the first mark intelligence that will be the first mark intelligence that will be the first mark of it for me you know and and um and you of it for me you know and and um and you of it for me you know and and um and you know your roots and my roots. We still know your roots and my roots. We still know your roots and my roots. We still want want want to you know understand everything and we to you know understand everything and we to you know understand everything and we do things to keep that fresh uh for us do things to keep that fresh uh for us do things to keep that fresh uh for us but but I think it's going to get hard. but but I think it's going to get hard. but but I think it's going to get hard. That's a really interesting point. I That's a really interesting point. I That's a really interesting point. I think you're right that I I and perhaps think you're right that I I and perhaps think you're right that I I and perhaps people of uh you know our generation people of uh you know our generation people of uh you know our generation plus or minus a few years are still plus or minus a few years are still plus or minus a few years are still unwilling to let go of the fact that we unwilling to let go of the fact that we unwilling to let go of the fact that we took a rock, we flattened it, we infused took a rock, we flattened it, we infused took a rock, we flattened it, we infused it with lightning and now it talks to it with lightning and now it talks to it with lightning and now it talks to us. And like I want to understand the us. And like I want to understand the us. And like I want to understand the lightning and the and the squishy rock lightning and the and the squishy rock lightning and the and the squishy rock part. But my children, my 19-year-old is part. But my children, my 19-year-old is part. But my children, my 19-year-old is perfectly willing to just accept that perfectly willing to just accept that perfectly willing to just accept that the magic black box is doing a thing. the magic black box is doing a thing. the magic black box is doing a thing. And when I have to have conversations And when I have to have conversations And when I have to have conversations with him about maybe let's not with him about maybe let's not with him about maybe let's not anthropomorphize the AI, let's talk anthropomorphize the AI, let's talk anthropomorphize the AI, let's talk about what's really happening. He's about what's really happening. He's about what's really happening. He's like, nah, it helped me with my homework like, nah, it helped me with my homework like, nah, it helped me with my homework and I'm cool with that. And uh I I I was and I'm cool with that. And uh I I I was and I'm cool with that. And uh I I I was a little bit taken by your book uh the a little bit taken by your book uh the a little bit taken by your book uh the AI revolution in medicine that was kind AI revolution in medicine that was kind AI revolution in medicine that was kind of pre GPT4 general availability.

of pre GPT4 general availability. of pre GPT4 general availability. um you were kind of poking at the model um you were kind of poking at the model um you were kind of poking at the model uh in in ways where you're like you were uh in in ways where you're like you were uh in in ways where you're like you were anthropomorphizing it but you were also anthropomorphizing it but you were also anthropomorphizing it but you were also trying to understand it. Um what would trying to understand it. Um what would trying to understand it. Um what would you change about how you interacted with you change about how you interacted with you change about how you interacted with the model now that you know more since the model now that you know more since the model now that you know more since this this very good book has been this this very good book has been this this very good book has been published? Yeah, you know that that published? Yeah, you know that that published? Yeah, you know that that book, you know, uh we wrote it uh over book, you know, uh we wrote it uh over book, you know, uh we wrote it uh over the Christmas holidays in the Christmas holidays in the Christmas holidays in 2022, you know, while GPG4 was still a 2022, you know, while GPG4 was still a 2022, you know, while GPG4 was still a secret project. And um and we organized secret project. And um and we organized secret project. And um and we organized the ourselves so that the book would get the ourselves so that the book would get the ourselves so that the book would get published at the same in the same month published at the same in the same month published at the same in the same month that GP4 was released to the whole that GP4 was released to the whole that GP4 was released to the whole world. Um and so that was a time world. Um and so that was a time world. Um and so that was a time when we were just so amazed and baffled. when we were just so amazed and baffled. when we were just so amazed and baffled. Um, you know, I talk about this as the Um, you know, I talk about this as the Um, you know, I talk about this as the nine stages of grief. In fact, I think nine stages of grief. In fact, I think nine stages of grief. In fact, I think at some point in the book, I talk about at some point in the book, I talk about at some point in the book, I talk about the nine stages of AI grief. Um, you the nine stages of AI grief. Um, you the nine stages of AI grief. Um, you know, when GPD4 was first exposed to me know, when GPD4 was first exposed to me know, when GPD4 was first exposed to me by the folks at OpenAI, I was intensely by the folks at OpenAI, I was intensely by the folks at OpenAI, I was intensely skeptical, you know, because they were skeptical, you know, because they were skeptical, you know, because they were just claiming that this thing could do just claiming that this thing could do just claiming that this thing could do things that I just thought were, you things that I just thought were, you things that I just thought were, you know, not possible. Uh, and then, you know, not possible. Uh, and then, you know, not possible. Uh, and then, you know, you pass from that stage of know, you pass from that stage of know, you pass from that stage of skepticism to a stage of frustration. uh skepticism to a stage of frustration. uh skepticism to a stage of frustration. uh because I felt like I was seeing my because I felt like I was seeing my because I felt like I was seeing my colleagues in Microsoft research falling colleagues in Microsoft research falling colleagues in Microsoft research falling getting duped by this thing. Um and then getting duped by this thing. Um and then getting duped by this thing. Um and then you start to feel worried because I you start to feel worried because I you start to feel worried because I detected that wow Microsoft might

detected that wow Microsoft might detected that wow Microsoft might actually make a big bet on this thing. actually make a big bet on this thing. actually make a big bet on this thing. Uh but then you get hands-on and you Uh but then you get hands-on and you Uh but then you get hands-on and you start to en encounter things that are start to en encounter things that are start to en encounter things that are just amazing. And I remember feeling the just amazing. And I remember feeling the just amazing. And I remember feeling the joy that wow this thing is I I never joy that wow this thing is I I never joy that wow this thing is I I never thought I would live long enough to see thought I would live long enough to see thought I would live long enough to see such a technology let alone have my such a technology let alone have my such a technology let alone have my hands on it. Uh and then you get into a hands on it. Uh and then you get into a hands on it. Uh and then you get into a period of intensity. So there are these period of intensity. So there are these period of intensity. So there are these stages that that you go through. But in stages that that you go through. But in stages that that you go through. But in those early stages of joy and euphoria those early stages of joy and euphoria those early stages of joy and euphoria and you lose and you lose and you lose sleep. Um it is sleep. Um it is sleep. Um it is those aspects that feel those aspects that feel those aspects that feel empathetic that feel that draw you into empathetic that feel that draw you into empathetic that feel that draw you into anthropomorphism that end up being so anthropomorphism that end up being so anthropomorphism that end up being so interesting. And in the field of interesting. And in the field of interesting. And in the field of medicine, you know, this has been medicine, you know, this has been medicine, you know, this has been observed over and over again. In fact, observed over and over again. In fact, observed over and over again. In fact, one month after we published our book, one month after we published our book, one month after we published our book, uh UC San Diego and Stanford uh jointly uh UC San Diego and Stanford uh jointly uh UC San Diego and Stanford uh jointly published a paper uh in a medical published a paper uh in a medical published a paper uh in a medical journal uh where they used journal uh where they used journal uh where they used GPT4 to write uh to respond to emails GPT4 to write uh to respond to emails GPT4 to write uh to respond to emails from patients. Um and they compared from patients. Um and they compared from patients. Um and they compared those to the emails that doctors, human those to the emails that doctors, human those to the emails that doctors, human doctors would write. And then they had a doctors would write. And then they had a doctors would write. And then they had a blind test uh and they had patients and blind test uh and they had patients and blind test uh and they had patients and doctors grade the quality correctness uh doctors grade the quality correctness uh doctors grade the quality correctness uh of these uh responses to patient of these uh responses to patient of these uh responses to patient queries. Not only was the queries. Not only was the queries. Not only was the AI equally accurate, but by a factor of

AI equally accurate, but by a factor of AI equally accurate, but by a factor of 9 to one, the AI generated notes were 9 to one, the AI generated notes were 9 to one, the AI generated notes were judged by patients to be more judged by patients to be more judged by patients to be more empathetic. And of course, you empathetic. And of course, you empathetic. And of course, you know, you know, it seems crazy to say a know, you know, it seems crazy to say a know, you know, it seems crazy to say a machine can be empathetic. What it machine can be empathetic. What it machine can be empathetic. What it really means, I think, is that a really means, I think, is that a really means, I think, is that a frazzled doctor, you can't take the time frazzled doctor, you can't take the time frazzled doctor, you can't take the time to write more than two or three to write more than two or three to write more than two or three sentences and just be get to the point sentences and just be get to the point sentences and just be get to the point and then get on to the next email. uh and then get on to the next email. uh and then get on to the next email. uh whereas, you know, the AI can write a whereas, you know, the AI can write a whereas, you know, the AI can write a couple of paragraphs and might remember couple of paragraphs and might remember couple of paragraphs and might remember that during the encounter they were that during the encounter they were that during the encounter they were talking about uh going to a Seahawks talking about uh going to a Seahawks talking about uh going to a Seahawks game and and other stuff like that and game and and other stuff like that and game and and other stuff like that and put in those uh those nice personal put in those uh those nice personal put in those uh those nice personal touches. And so there's something there touches. And so there's something there touches. And so there's something there that is both uh interesting and that is both uh interesting and that is both uh interesting and disturbing uh but also seems to really disturbing uh but also seems to really disturbing uh but also seems to really touch people in a very meaningful and touch people in a very meaningful and touch people in a very meaningful and very practical way. And um you know I very practical way. And um you know I very practical way. And um you know I think we're still just as a society think we're still just as a society think we're still just as a society trying to come to grips with this. Yeah.

trying to come to grips with this. Yeah. trying to come to grips with this. Yeah. I I I'm struck that one could I I I'm struck that one could I I I'm struck that one could theoretically say that it can be theoretically say that it can be theoretically say that it can be infinitely empathetic and infinitely infinitely empathetic and infinitely infinitely empathetic and infinitely patient given appropriate prompting and patient given appropriate prompting and patient given appropriate prompting and in an appropriate good attitude on the in an appropriate good attitude on the in an appropriate good attitude on the part of the controlling uh controlling part of the controlling uh controlling part of the controlling uh controlling human like the I keep coming back to human like the I keep coming back to human like the I keep coming back to empathy. We need more empathy in in the empathy. We need more empathy in in the empathy. We need more empathy in in the world right now. We need more empathy in world right now. We need more empathy in world right now. We need more empathy in tech. Uh when I use co-pilot for on tech. Uh when I use co-pilot for on tech. Uh when I use co-pilot for on GitHub copilot I don't ask it how to do GitHub copilot I don't ask it how to do GitHub copilot I don't ask it how to do my homework. I basically use it as an my homework. I basically use it as an my homework. I basically use it as an enthusiastic uh pair programming partner enthusiastic uh pair programming partner enthusiastic uh pair programming partner and I find it to be infinitely patient. and I find it to be infinitely patient. and I find it to be infinitely patient. It it never judges me. It's never mean. It it never judges me. It's never mean. It it never judges me. It's never mean. It's never unkind. So then it has It's never unkind. So then it has It's never unkind. So then it has theoretically infinite empathy if I talk theoretically infinite empathy if I talk theoretically infinite empathy if I talk to it right. Um I'm struck by uh the to it right. Um I'm struck by uh the to it right. Um I'm struck by uh the epilogue in your in your book. You say epilogue in your in your book. You say epilogue in your in your book. You say it can't be that it just can't be that it can't be that it just can't be that it can't be that it just can't be that next word prediction could be next word prediction could be next word prediction could be intelligence or can it? Am I just a intelligence or can it? Am I just a intelligence or can it? Am I just a statistical model of what's the most statistical model of what's the most statistical model of what's the most likely next word from Hanselman to say? likely next word from Hanselman to say? likely next word from Hanselman to say? Is that my is that my the the animous of Is that my is that my the the animous of Is that my is that my the the animous of me? So I think me? So I think me? So I think um yeah at the time we wrote that book um yeah at the time we wrote that book um yeah at the time we wrote that book you this was I was really baffled by you this was I was really baffled by you this was I was really baffled by this but I think my understanding and this but I think my understanding and this but I think my understanding and acceptance of what's going on has acceptance of what's going on has acceptance of what's going on has evolved a lot more. Uh it's it's true evolved a lot more. Uh it's it's true evolved a lot more. Uh it's it's true that the fundamental pre-training of that the fundamental pre-training of that the fundamental pre-training of these large language models uh is to these large language models uh is to these large language models uh is to predict the next word in a predict the next word in a predict the next word in a conversation. And

conversation. And conversation. And so you know to to that extent you could so you know to to that extent you could so you know to to that extent you could say that these large language models say that these large language models say that these large language models aren't trained to do anything useful aren't trained to do anything useful aren't trained to do anything useful except except that um but uh here's an except except that um but uh here's an except except that um but uh here's an example that I'd like to give. Uh let's example that I'd like to give. Uh let's example that I'd like to give. Uh let's take the sentence and the murderer is take the sentence and the murderer is take the sentence and the murderer is blank. blank. blank. Okay. So now, uh, if you want to, you Okay. So now, uh, if you want to, you Okay. So now, uh, if you want to, you know, pick with the highest quality, you know, pick with the highest quality, you know, pick with the highest quality, you know, the the word that would fill in know, the the word that would fill in know, the the word that would fill in the blank, well, that se sentence and the blank, well, that se sentence and the blank, well, that se sentence and the murder is blank is in the context the murder is blank is in the context the murder is blank is in the context of, let's say, a whole Agatha Christie of, let's say, a whole Agatha Christie of, let's say, a whole Agatha Christie murder mystery novel. murder mystery novel. murder mystery novel. And and if you were to just approach And and if you were to just approach And and if you were to just approach this purely as a statistics question, this purely as a statistics question, this purely as a statistics question, well, there are lots of thousands of well, there are lots of thousands of well, there are lots of thousands of murder mystery novels and short stories murder mystery novels and short stories murder mystery novels and short stories and you could try to make a statistical and you could try to make a statistical and you could try to make a statistical pattern on, you know, what are the most pattern on, you know, what are the most pattern on, you know, what are the most likely names of murderers and you likely names of murderers and you likely names of murderers and you wouldn't get any good answer there. wouldn't get any good answer there. wouldn't get any good answer there. Instead, in order to really optimize the Instead, in order to really optimize the Instead, in order to really optimize the quality of the fill-in- thelank quality of the fill-in- thelank quality of the fill-in- thelank capability, you somehow have to be able capability, you somehow have to be able capability, you somehow have to be able to do some deductive reasoning. You have to do some deductive reasoning. You have to do some deductive reasoning. You have to have an understanding of the to have an understanding of the to have an understanding of the psychology of humans in different psychology of humans in different psychology of humans in different situations, what motivates them, how situations, what motivates them, how situations, what motivates them, how they react under certain kinds of they react under certain kinds of they react under certain kinds of questioning and all of those things. And questioning and all of those things. And questioning and all of those things. And so the way to understand what's going on so the way to understand what's going on so the way to understand what's going on when we try to optimize fill-in-theblank when we try to optimize fill-in-theblank when we try to optimize fill-in-theblank or next word prediction uh is as a very or next word prediction uh is as a very or next word prediction uh is as a very very large uh astronomically large

very large uh astronomically large very large uh astronomically large stochastic process that has a chance of stochastic process that has a chance of stochastic process that has a chance of accidentally discovering neural accidentally discovering neural accidentally discovering neural circuitry that implements some circuitry that implements some circuitry that implements some aspects of those reasoning aspects of those reasoning aspects of those reasoning functions. functions. functions. And the fact that that can happen at all And the fact that that can happen at all And the fact that that can happen at all even by accident even by accident even by accident um is amazing. But we're operating at um is amazing. But we're operating at um is amazing. But we're operating at scale where indeed it is actually scale where indeed it is actually scale where indeed it is actually happening. And so it it's it it's not happening. And so it it's it it's not happening. And so it it's it it's not that it's next word prediction that is that it's next word prediction that is that it's next word prediction that is you know causing us to appear to be you know causing us to appear to be you know causing us to appear to be thinking uh but the process of highly thinking uh but the process of highly thinking uh but the process of highly optimizing an ability to do very very optimizing an ability to do very very optimizing an ability to do very very good next word prediction is giving us a good next word prediction is giving us a good next word prediction is giving us a chance to really discover and chance to really discover and chance to really discover and solidify these bits of neural circuitry solidify these bits of neural circuitry solidify these bits of neural circuitry to do things. Yeah. One analogy that to do things. Yeah. One analogy that to do things. Yeah. One analogy that I've used to explain to some young I've used to explain to some young I've used to explain to some young people and I don't know if it's a good people and I don't know if it's a good people and I don't know if it's a good one is that you know we are as humans one is that you know we are as humans one is that you know we are as humans limited by the size of our stack for our limited by the size of our stack for our limited by the size of our stack for our recency and then the older you get you recency and then the older you get you recency and then the older you get you get this larger and larger and larger get this larger and larger and larger get this larger and larger and larger heap that you can pull from and then heap that you can pull from and then heap that you can pull from and then you're constantly pulling things out of you're constantly pulling things out of you're constantly pulling things out of the heap and into the into the stack and the heap and into the into the stack and the heap and into the into the stack and then there's certain presenters there's then there's certain presenters there's then there's certain presenters there's certain um thinkers uh Jamal Buouie is certain um thinkers uh Jamal Buouie is certain um thinkers uh Jamal Buouie is one I think of who's a opinion person one I think of who's a opinion person one I think of who's a opinion person for the New York Times who seems to have for the New York Times who seems to have for the New York Times who seems to have this huge corpus of information that he this huge corpus of information that he this huge corpus of information that he is pulling upon all the books that he's is pulling upon all the books that he's is pulling upon all the books that he's ever read. He's got an amazing ever read. He's got an amazing ever read. He's got an amazing vocabulary and I admire him in his vocabulary and I admire him in his vocabulary and I admire him in his ability to page in and out these pieces

ability to page in and out these pieces ability to page in and out these pieces of wisdom while I struggle to root of wisdom while I struggle to root of wisdom while I struggle to root around in in in in the the the totality around in in in in the the the totality around in in in in the the the totality of my uh of my existence. And I think of my uh of my existence. And I think of my uh of my existence. And I think that uh AI will feel like AGI when when that uh AI will feel like AGI when when that uh AI will feel like AGI when when its stack when its context window is its stack when its context window is its stack when its context window is beyond those of even the smartest beyond those of even the smartest beyond those of even the smartest person. And that's that's going to be person. And that's that's going to be person. And that's that's going to be the thing that that it has just such a the thing that that it has just such a the thing that that it has just such a large context window. It's bigger than a large context window. It's bigger than a large context window. It's bigger than a human lifetime. human lifetime. human lifetime. Yeah. You know, one thing that um you Yeah. You know, one thing that um you Yeah. You know, one thing that um you know, the up till uh now for the most know, the up till uh now for the most know, the up till uh now for the most part large language models and part large language models and part large language models and transformers transformers transformers specifically were pretty imperfect in uh specifically were pretty imperfect in uh specifically were pretty imperfect in uh memorization of things. memorization of things. memorization of things. there's a massive compression of the there's a massive compression of the there's a massive compression of the training corpus that goes on uh when you training corpus that goes on uh when you training corpus that goes on uh when you train it and into a a train it and into a a train it and into a a transformer. And so, you know, I excuse transformer. And so, you know, I excuse transformer. And so, you know, I excuse me, I was me, I was me, I was always trying to explain that to doctors always trying to explain that to doctors always trying to explain that to doctors as they're trying to come to grips with as they're trying to come to grips with as they're trying to come to grips with um generative um generative um generative AI because um you know, AI because um you know, AI because um you know, unlike a normal computer, in fact, one unlike a normal computer, in fact, one unlike a normal computer, in fact, one popular application, maybe the most popular application, maybe the most popular application, maybe the most popular application that doctors use popular application that doctors use popular application that doctors use online is something called up-to-date.

online is something called up-to-date. online is something called up-to-date. It's essentially a search engine for uh It's essentially a search engine for uh It's essentially a search engine for uh very highly curated medical very highly curated medical very highly curated medical knowledge. And so in up-to-date, you ask knowledge. And so in up-to-date, you ask knowledge. And so in up-to-date, you ask a question and you get a medically a question and you get a medically a question and you get a medically precise precise precise answer. And we've trained ourselves to answer. And we've trained ourselves to answer. And we've trained ourselves to use that just like we use web search. use that just like we use web search. use that just like we use web search. You make a query and you expect to get a You make a query and you expect to get a You make a query and you expect to get a set of answers that are pretty precise. set of answers that are pretty precise. set of answers that are pretty precise. But the transformer doesn't have that But the transformer doesn't have that But the transformer doesn't have that capability. it has very very imperfect capability. it has very very imperfect capability. it has very very imperfect memory and um of course that's now memory and um of course that's now memory and um of course that's now evolving because there's no fundamental evolving because there's no fundamental evolving because there's no fundamental reason why a computer-based system reason why a computer-based system reason why a computer-based system couldn't actually have perfect memory couldn't actually have perfect memory couldn't actually have perfect memory recall and so I I agree with you I think recall and so I I agree with you I think recall and so I I agree with you I think we're going to get to a point where we're going to get to a point where we're going to get to a point where these AI systems these AI systems these AI systems really really really are going to be benefiting are going to be benefiting are going to be benefiting from the fundamental capabilities of from the fundamental capabilities of from the fundamental capabilities of perfect memory recall that we've always perfect memory recall that we've always perfect memory recall that we've always assumed machines would have while also assumed machines would have while also assumed machines would have while also making all of these associations and making all of these associations and making all of these associations and engaging in this kind of reasoning.

engaging in this kind of reasoning. engaging in this kind of reasoning. Why is it and forgive my ignorance if Why is it and forgive my ignorance if Why is it and forgive my ignorance if this is a question I should know the this is a question I should know the this is a question I should know the answer to, but when did your deep answer to, but when did your deep answer to, but when did your deep interest in the medical aspect of interest in the medical aspect of interest in the medical aspect of technology like in everything that you technology like in everything that you technology like in everything that you do certainly recently of late uh you do certainly recently of late uh you do certainly recently of late uh you always want to make technology to make always want to make technology to make always want to make technology to make people's lives better and it seems to people's lives better and it seems to people's lives better and it seems to come back to health care. Do you have a come back to health care. Do you have a come back to health care. Do you have a background in healthcare I'm not aware background in healthcare I'm not aware background in healthcare I'm not aware of? of? of? Well, it's really been accidental. Um, Well, it's really been accidental. Um, Well, it's really been accidental. Um, and it almost didn't happen. You know, and it almost didn't happen. You know, and it almost didn't happen. You know, when I joined Microsoft in 2010, when I joined Microsoft in 2010, when I joined Microsoft in 2010, um, it was to join this great um, it was to join this great um, it was to join this great organization called Microsoft Research. organization called Microsoft Research. organization called Microsoft Research. And I was very proud to be part of And I was very proud to be part of And I was very proud to be part of Microsoft Research. And, um, I rose Microsoft Research. And, um, I rose Microsoft Research. And, um, I rose through the ranks to the point uh, where through the ranks to the point uh, where through the ranks to the point uh, where I was the leader of Microsoft Research I was the leader of Microsoft Research I was the leader of Microsoft Research worldwide. And then in 2016 worldwide. And then in 2016 worldwide. And then in 2016 uh Sacha Nadella and the CTO at that uh Sacha Nadella and the CTO at that uh Sacha Nadella and the CTO at that time Harry Sham time Harry Sham time Harry Sham uh reassigned me took me out of research uh reassigned me took me out of research uh reassigned me took me out of research and asked me to take on this 100 person and asked me to take on this 100 person and asked me to take on this 100 person team a skunk works team to rethink team a skunk works team to rethink team a skunk works team to rethink Microsoft's approach to healthcare and Microsoft's approach to healthcare and Microsoft's approach to healthcare and healthcare technology. I was devastated healthcare technology. I was devastated healthcare technology. I was devastated by that reassignment. I thought in fact by that reassignment. I thought in fact by that reassignment. I thought in fact I was being punished for some reason and I was being punished for some reason and I was being punished for some reason and actually contemplated quitting. Um you actually contemplated quitting. Um you actually contemplated quitting. Um you know not only did I not have any know not only did I not have any know not only did I not have any background in healthcare or medicine but background in healthcare or medicine but background in healthcare or medicine but you know Microsoft you know if you go to you know Microsoft you know if you go to you know Microsoft you know if you go to any healthcare organization any clinic any healthcare organization any clinic any healthcare organization any clinic around the world you will see Microsoft

around the world you will see Microsoft around the world you will see Microsoft products there. Uh we sell to literally products there. Uh we sell to literally products there. Uh we sell to literally every single healthcare organization on every single healthcare organization on every single healthcare organization on the planet. Uh I think uh one of our the planet. Uh I think uh one of our the planet. Uh I think uh one of our smallest accounts is a one nurse clinic smallest accounts is a one nurse clinic smallest accounts is a one nurse clinic in Nairobi, Kenya and then all the way in Nairobi, Kenya and then all the way in Nairobi, Kenya and then all the way to giants like United Health Group or to giants like United Health Group or to giants like United Health Group or Kaiser Permanente and everything in Kaiser Permanente and everything in Kaiser Permanente and everything in between. And so that meant there had to between. And so that meant there had to between. And so that meant there had to be maybe a dozen powerful corporate vice be maybe a dozen powerful corporate vice be maybe a dozen powerful corporate vice presidents all throughout Microsoft all presidents all throughout Microsoft all presidents all throughout Microsoft all doing their own things in healthcare. doing their own things in healthcare. doing their own things in healthcare. And so I also had to think who's going And so I also had to think who's going And so I also had to think who's going to listen to Peter Lee uh on on to listen to Peter Lee uh on on to listen to Peter Lee uh on on anything? And so that's how it started. anything? And so that's how it started. anything? And so that's how it started. Um Um Um and you know you had to kind of think and you know you had to kind of think and you know you had to kind of think you know what would we you know what would we you know what would we do? Um one question um you know Satya do? Um one question um you know Satya do? Um one question um you know Satya was worried that we weren't thinking was worried that we weren't thinking was worried that we weren't thinking enough about the cloud and AI in enough about the cloud and AI in enough about the cloud and AI in healthcare. So a first thing was to try healthcare. So a first thing was to try healthcare. So a first thing was to try to think well uh can the cloud be used to think well uh can the cloud be used to think well uh can the cloud be used to store like healthcare records and we to store like healthcare records and we to store like healthcare records and we learned early on that no uh that there learned early on that no uh that there learned early on that no uh that there were compliance falls uh there are were compliance falls uh there are were compliance falls uh there are certain data standards that we weren't certain data standards that we weren't certain data standards that we weren't supporting and not only could our cloud supporting and not only could our cloud supporting and not only could our cloud not do it but our competitors clouds not do it but our competitors clouds not do it but our competitors clouds like Google and Amazon couldn't do it like Google and Amazon couldn't do it like Google and Amazon couldn't do it either. And so that at least got us either. And so that at least got us either. And so that at least got us started with something to do, something started with something to do, something started with something to do, something to fix. Um, we made a lot of progress in to fix. Um, we made a lot of progress in to fix. Um, we made a lot of progress in that. We also started a second project that. We also started a second project that. We also started a second project in collaboration with a company called

in collaboration with a company called in collaboration with a company called Nuance and a a doctor at University of Nuance and a a doctor at University of Nuance and a a doctor at University of Pittsburgh Medical Center, Shiva. Uh and Pittsburgh Medical Center, Shiva. Uh and Pittsburgh Medical Center, Shiva. Uh and that project was called Empower MD that project was called Empower MD that project was called Empower MD because we learned that doctors were because we learned that doctors were because we learned that doctors were really suffering with having to write uh really suffering with having to write uh really suffering with having to write uh clinical notes to enter into electronic clinical notes to enter into electronic clinical notes to enter into electronic health record system uh after every health record system uh after every health record system uh after every conversation with a patient. And so we conversation with a patient. And so we conversation with a patient. And so we thought well we could use AI to listen thought well we could use AI to listen thought well we could use AI to listen to the conversation and then at least to the conversation and then at least to the conversation and then at least draft a clinical note automatically. And draft a clinical note automatically. And draft a clinical note automatically. And so that was a project called EmpowerMD. so that was a project called EmpowerMD. so that was a project called EmpowerMD. And that got serious enough that we And that got serious enough that we And that got serious enough that we actually went ahead and decided to actually went ahead and decided to actually went ahead and decided to productize that and acquire nuance in productize that and acquire nuance in productize that and acquire nuance in the process. And for Shiva, Dr. Shiva at the process. And for Shiva, Dr. Shiva at the process. And for Shiva, Dr. Shiva at UPMC, uh the venture arm of UPMC, UPMC UPMC, uh the venture arm of UPMC, UPMC UPMC, uh the venture arm of UPMC, UPMC Enterprises agreed to provide seed Enterprises agreed to provide seed Enterprises agreed to provide seed funding for Shiv to spin off a company funding for Shiv to spin off a company funding for Shiv to spin off a company to do the same thing and that's a to do the same thing and that's a to do the same thing and that's a company called a bridge. And today the company called a bridge. And today the company called a bridge. And today the top two products in that space are top two products in that space are top two products in that space are Microsoft's DAX co-pilot and a bridges a Microsoft's DAX co-pilot and a bridges a Microsoft's DAX co-pilot and a bridges a product called the bridge. So that's how product called the bridge. So that's how product called the bridge. So that's how we kind of got started. And so in that we kind of got started. And so in that we kind of got started. And so in that process of about five years of working process of about five years of working process of about five years of working on that um I got up to speed a lot. I on that um I got up to speed a lot. I on that um I got up to speed a lot. I also became a founding board member for also became a founding board member for also became a founding board member for a new medical school the Kaiser a new medical school the Kaiser a new medical school the Kaiser Permanente School of Medicine. and a new Permanente School of Medicine. and a new Permanente School of Medicine. and a new school of medicine has to come up with a school of medicine has to come up with a school of medicine has to come up with a curriculum. And so I was able to study curriculum. And so I was able to study curriculum. And so I was able to study the curriculum and at least in the curriculum and at least in the curriculum and at least in pre-clinical studies learned quite a bit

pre-clinical studies learned quite a bit pre-clinical studies learned quite a bit all the way to the point that I actually all the way to the point that I actually all the way to the point that I actually got elected to the National Academy of got elected to the National Academy of got elected to the National Academy of Medicine. And then finally Medicine. And then finally Medicine. And then finally um Kevin Scott, our CTO in 2020 hired me um Kevin Scott, our CTO in 2020 hired me um Kevin Scott, our CTO in 2020 hired me back into research. And so I thought, back into research. And so I thought, back into research. And so I thought, okay, I can separate from healthcare, okay, I can separate from healthcare, okay, I can separate from healthcare, get back to what I my true love, which get back to what I my true love, which get back to what I my true love, which is fundamental research and computer is fundamental research and computer is fundamental research and computer science. And and then the pandemic science. And and then the pandemic science. And and then the pandemic hit. And so then Microsoft decided, hit. And so then Microsoft decided, hit. And so then Microsoft decided, well, Peter, you're our healthcare well, Peter, you're our healthcare well, Peter, you're our healthcare technology guy and all of our customers technology guy and all of our customers technology guy and all of our customers and stakeholders need help from and stakeholders need help from and stakeholders need help from Microsoft to cope with the pandemic. So Microsoft to cope with the pandemic. So Microsoft to cope with the pandemic. So you need to coordinate that. you need to coordinate that. you need to coordinate that. So it kept me in the healthcare. I So it kept me in the healthcare. I So it kept me in the healthcare. I thought that that project would only thought that that project would only thought that that project would only last a summer of 2020, but of course the last a summer of 2020, but of course the last a summer of 2020, but of course the pandemic ended up being much more pandemic ended up being much more pandemic ended up being much more serious thing than that. And then when serious thing than that. And then when serious thing than that. And then when GPT 3.5 and GPD4 came GPT 3.5 and GPD4 came GPT 3.5 and GPD4 came out, there was again, you know, the out, there was again, you know, the out, there was again, you know, the question, wait a minute, you know, is question, wait a minute, you know, is question, wait a minute, you know, is this stuff good to use? Is it safe to this stuff good to use? Is it safe to this stuff good to use? Is it safe to use in healthy medicine? And I was the use in healthy medicine? And I was the use in healthy medicine? And I was the logical person to help lead that. So, I logical person to help lead that. So, I logical person to help lead that. So, I guess you could say I've been trying guess you could say I've been trying guess you could say I've been trying hard not to be in hard not to be in hard not to be in healthcare, but I keep getting pulled healthcare, but I keep getting pulled healthcare, but I keep getting pulled back in. Um, and you know, and I'm not back in. Um, and you know, and I'm not back in. Um, and you know, and I'm not unhappy about that, but it it's odd unhappy about that, but it it's odd unhappy about that, but it it's odd because it's none of it has been because it's none of it has been because it's none of it has been planned.

planned. planned. Some of the best careers are not Some of the best careers are not Some of the best careers are not planned. And I think that's a testament planned. And I think that's a testament planned. And I think that's a testament to that. to that. to that. Um, as we get ready to close, I'm glad I Um, as we get ready to close, I'm glad I Um, as we get ready to close, I'm glad I didn't leave Microsoft. That that would didn't leave Microsoft. That that would didn't leave Microsoft. That that would have been I I think we are all glad you have been I I think we are all glad you have been I I think we are all glad you didn't leave. That would have been if didn't leave. That would have been if didn't leave. That would have been if you felt you were being punished and you felt you were being punished and you felt you were being punished and quit immediately, we would have quit immediately, we would have quit immediately, we would have definitely lost out. Um, as we get ready definitely lost out. Um, as we get ready definitely lost out. Um, as we get ready to close, I know this here because Satya to close, I know this here because Satya to close, I know this here because Satya Nadella is such a remarkable leader. Uh, Nadella is such a remarkable leader. Uh, Nadella is such a remarkable leader. Uh, but I think what people on the outside but I think what people on the outside but I think what people on the outside sometimes don't know is he sometimes sometimes don't know is he sometimes sometimes don't know is he sometimes does ask people to do very very hard does ask people to do very very hard does ask people to do very very hard things and Yeah. Yeah. And it it really things and Yeah. Yeah. And it it really things and Yeah. Yeah. And it it really moves things forward. That is true. When moves things forward. That is true. When moves things forward. That is true. When you're called to serve though, you you're called to serve though, you you're called to serve though, you either step up or you don't. And it's uh either step up or you don't. And it's uh either step up or you don't. And it's uh it's good that you stepped up. Uh, I was it's good that you stepped up. Uh, I was it's good that you stepped up. Uh, I was going to say that as we get ready to going to say that as we get ready to going to say that as we get ready to close. Uh, here we are kind of in the close. Uh, here we are kind of in the close. Uh, here we are kind of in the middle of of March, what are the things middle of of March, what are the things middle of of March, what are the things that are coming out of uh, Microsoft that are coming out of uh, Microsoft that are coming out of uh, Microsoft Research that we should learn about as Research that we should learn about as Research that we should learn about as we close this podcast. What are some we close this podcast. What are some we close this podcast. What are some things that have either been announced things that have either been announced things that have either been announced or being announced here at the at the or being announced here at the at the or being announced here at the at the middle of March, middle of March, middle of March, right? Um, well, you know, we've been right? Um, well, you know, we've been right? Um, well, you know, we've been doing so much uh, particularly in uh, doing so much uh, particularly in uh, doing so much uh, particularly in uh, two areas. Uh, one is what we would call two areas. Uh, one is what we would call two areas. Uh, one is what we would call AI for science. So in the same way that AI for science. So in the same way that AI for science. So in the same way that we've discovered that generative AI we've discovered that generative AI we've discovered that generative AI architectures you know like transformers architectures you know like transformers architectures you know like transformers and diffusion uh based and diffusion uh based and diffusion uh based models seem to models seem to models seem to learn so effectively from our words and learn so effectively from our words and learn so effectively from our words and thoughts and actions. So you know you thoughts and actions. So you know you thoughts and actions. So you know you can take a big corpus of human text can take a big corpus of human text can take a big corpus of human text output or word output and it's amazing

output or word output and it's amazing output or word output and it's amazing you know what is learned there and you know what is learned there and you know what is learned there and similarly from pictures of what we do similarly from pictures of what we do similarly from pictures of what we do out in the world and out in the world and out in the world and videos those same videos those same videos those same architectures what the world is learning architectures what the world is learning architectures what the world is learning and is a subject of huge intensity in and is a subject of huge intensity in and is a subject of huge intensity in Microsoft research also work for Microsoft research also work for Microsoft research also work for observations of natural phenomena like observations of natural phenomena like observations of natural phenomena like atmospheric wind patterns or the atmospheric wind patterns or the atmospheric wind patterns or the dynamics of proteins and small molecules dynamics of proteins and small molecules dynamics of proteins and small molecules uh or uh the the uh movement of uh or uh the the uh movement of uh or uh the the uh movement of electrons electrons electrons in electrolytic uh material structures. in electrolytic uh material structures. in electrolytic uh material structures. And that is in really incredible because And that is in really incredible because And that is in really incredible because that means that if we follow the same that means that if we follow the same that means that if we follow the same path of AI scale in those areas, we path of AI scale in those areas, we path of AI scale in those areas, we might be able to do things like predict might be able to do things like predict might be able to do things like predict severe weather events weeks in advance severe weather events weeks in advance severe weather events weeks in advance or uh design new molecules, a drug or uh design new molecules, a drug or uh design new molecules, a drug molecule uh for uh known drug targets or molecule uh for uh known drug targets or molecule uh for uh known drug targets or identify new drug targets uh in identify new drug targets uh in identify new drug targets uh in pathogens.

pathogens. pathogens. um or be able to design new um or be able to design new um or be able to design new materials for everything from say solid materials for everything from say solid materials for everything from say solid state batteries to enzymes to make your state batteries to enzymes to make your state batteries to enzymes to make your vegan food taste better. Um and so that vegan food taste better. Um and so that vegan food taste better. Um and so that kind of AI for science thing I think is kind of AI for science thing I think is kind of AI for science thing I think is something that was a big something that was a big something that was a big um showcase effort uh with a whole bunch um showcase effort uh with a whole bunch um showcase effort uh with a whole bunch of new models that are now in the Azure of new models that are now in the Azure of new models that are now in the Azure foundry coming out of Microsoft research foundry coming out of Microsoft research foundry coming out of Microsoft research while being simultaneously published in while being simultaneously published in while being simultaneously published in the top scientific journals in material the top scientific journals in material the top scientific journals in material science in chemical engineering in science in chemical engineering in science in chemical engineering in physics in climate science and so on. physics in climate science and so on. physics in climate science and so on. The one big difference the thing the one The one big difference the thing the one The one big difference the thing the one thing that holds us back in all of that thing that holds us back in all of that thing that holds us back in all of that is access to training data. There's no is access to training data. There's no is access to training data. There's no internet of like molecular dynamics internet of like molecular dynamics internet of like molecular dynamics simulations. So the question is where do simulations. So the question is where do simulations. So the question is where do you get the training data? you get the training data? you get the training data? We have the compute infrastructure but We have the compute infrastructure but We have the compute infrastructure but we need the training data and on that we need the training data and on that we need the training data and on that the second thing that's emerging is the second thing that's emerging is the second thing that's emerging is quantum quantum quantum computing and I think in 2025 we are computing and I think in 2025 we are computing and I think in 2025 we are going to see the first practical going to see the first practical going to see the first practical scalable quantum machines and the very scalable quantum machines and the very scalable quantum machines and the very first first first application that at least uh I and some application that at least uh I and some application that at least uh I and some of our colleagues in Microsoft want to of our colleagues in Microsoft want to of our colleagues in Microsoft want to see is to run classical precise see is to run classical precise see is to run classical precise simulations of these natural phenomena simulations of these natural phenomena simulations of these natural phenomena to generate to generate to generate large amounts of perfectly label large amounts of perfectly label large amounts of perfectly label training data. Um and and if we can do training data. Um and and if we can do training data. Um and and if we can do that then you know we can literally have

that then you know we can literally have that then you know we can literally have the GPT4 or GPT5 the GPT4 or GPT5 the GPT4 or GPT5 of you know proteins of you know proteins of you know proteins of of materials uh of weather patterns of of materials uh of weather patterns of of materials uh of weather patterns and I think that'll be that'll be pretty and I think that'll be that'll be pretty and I think that'll be that'll be pretty stunning. Yeah. and and then the the stunning. Yeah. and and then the the stunning. Yeah. and and then the the that that that that research to that that that that research to that that that that research to practicality bridge that like it doesn't practicality bridge that like it doesn't practicality bridge that like it doesn't just become theoretical at that point just become theoretical at that point just become theoretical at that point like I love that you're putting out like I love that you're putting out like I love that you're putting out papers and simultaneously releasing a papers and simultaneously releasing a papers and simultaneously releasing a model on the foundry uh because uh I go model on the foundry uh because uh I go model on the foundry uh because uh I go back call back to the whole beginning of back call back to the whole beginning of back call back to the whole beginning of this conversation I've been told that my this conversation I've been told that my this conversation I've been told that my diabetes will be cured in 5 years every diabetes will be cured in 5 years every diabetes will be cured in 5 years every year for the last 30 years uh you know year for the last 30 years uh you know year for the last 30 years uh you know show me the money show me the practical show me the money show me the practical show me the money show me the practical thing that's going to prevent prevent thing that's going to prevent prevent thing that's going to prevent prevent someone from losing their home in a uh someone from losing their home in a uh someone from losing their home in a uh in a tornado or prevent someone from in a tornado or prevent someone from in a tornado or prevent someone from dying of glyopblastoma. You know, those dying of glyopblastoma. You know, those dying of glyopblastoma. You know, those kind of practical things. You're saying kind of practical things. You're saying kind of practical things. You're saying things are good things are coming from things are good things are coming from things are good things are coming from AI and from these models. The way I've AI and from these models. The way I've AI and from these models. The way I've tried to explain it is that across uh tried to explain it is that across uh tried to explain it is that across uh many many uh scientific many many uh scientific many many uh scientific domains we've achieved GPT2 level of domains we've achieved GPT2 level of domains we've achieved GPT2 level of capability and the only thing that's capability and the only thing that's capability and the only thing that's really preventing us to get to GPT3 is really preventing us to get to GPT3 is really preventing us to get to GPT3 is access to adequate training at Corpus.

access to adequate training at Corpus. access to adequate training at Corpus. Um and so the minute that we're able to Um and so the minute that we're able to Um and so the minute that we're able to solve those issues, you know, we'll get solve those issues, you know, we'll get solve those issues, you know, we'll get to GPD3 class capability and beyond. And to GPD3 class capability and beyond. And to GPD3 class capability and beyond. And GPD3 for us at Microsoft, for you and GPD3 for us at Microsoft, for you and GPD3 for us at Microsoft, for you and me, Scott has been important because me, Scott has been important because me, Scott has been important because GPD3 in large language models was the GPD3 in large language models was the GPD3 in large language models was the first stage where we could try to make a first stage where we could try to make a first stage where we could try to make a product out of this. And that of course product out of this. And that of course product out of this. And that of course was the first GitHub co-pilot. was the first GitHub co-pilot. was the first GitHub co-pilot. Yeah, that's the beginning of the hockey Yeah, that's the beginning of the hockey Yeah, that's the beginning of the hockey stick, right? When it starts to curve, stick, right? When it starts to curve, stick, right? When it starts to curve, then things start happening. Well, thank then things start happening. Well, thank then things start happening. Well, thank you so much Dr. Peter Lee for chatting you so much Dr. Peter Lee for chatting you so much Dr. Peter Lee for chatting with us today. We really appreciate it. with us today. We really appreciate it. with us today. We really appreciate it. Well, Scott, thanks for having me here. Well, Scott, thanks for having me here. Well, Scott, thanks for having me here. It was really fun to chat with you. We It was really fun to chat with you. We It was really fun to chat with you. We have been chatting with Dr. Peter Lee, have been chatting with Dr. Peter Lee, have been chatting with Dr. Peter Lee, the president of Microsoft Research, and the president of Microsoft Research, and the president of Microsoft Research, and this has been another episode of Hansel this has been another episode of Hansel this has been another episode of Hansel Minutes in association with the ACM Minutes in association with the ACM Minutes in association with the ACM Bitecast, and we'll see you again next Bitecast, and we'll see you again next Bitecast, and we'll see you again next week.