The next supercomputer with NVIDIA's Wen-Mei Hwu

hey friends you know here at H minutes hey friends you know here at H minutes we really appreciate our sponsors and we really appreciate our sponsors and we really appreciate our sponsors and I'm talking to JD from rayun JD why I'm talking to JD from rayun JD why I'm talking to JD from rayun JD why monitoring why do I need to monitor my monitoring why do I need to monitor my monitoring why do I need to monitor my production production production applications well first and foremost we applications well first and foremost we applications well first and foremost we obviously want to deliver a great obviously want to deliver a great obviously want to deliver a great experience to our users and as a experience to our users and as a experience to our users and as a developer you know when I'm working on developer you know when I'm working on developer you know when I'm working on my machine and something breaks the my machine and something breaks the my machine and something breaks the debugger hits I can fix it really debugger hits I can fix it really debugger hits I can fix it really quickly with all the context there so we quickly with all the context there so we quickly with all the context there so we wanted to answer the question how do we wanted to answer the question how do we wanted to answer the question how do we deliver that same experience but in deliver that same experience but in deliver that same experience but in production where everything that goes production where everything that goes production where everything that goes wrong gives me all the context to go and wrong gives me all the context to go and wrong gives me all the context to go and fix those issues really quickly for fix those issues really quickly for fix those issues really quickly for people absolutely fantastic you can go people absolutely fantastic you can go people absolutely fantastic you can go to Ray gun.com for a 14-day free trial to Ray gun.com for a 14-day free trial to Ray gun.com for a 14-day free trial and use the code Hanselman for half and use the code Hanselman for half and use the code Hanselman for half price monitoring for your first [Music] [Music] month hi I'm Scott Hanselman this is month hi I'm Scott Hanselman this is month hi I'm Scott Hanselman this is another episode of Hansel minutes in another episode of Hansel minutes in another episode of Hansel minutes in association with the ACM bcast today I association with the ACM bcast today I association with the ACM bcast today I have the distinct pleasure to speak with have the distinct pleasure to speak with have the distinct pleasure to speak with Dr wenay Who from uh Nvidia from the Dr wenay Who from uh Nvidia from the Dr wenay Who from uh Nvidia from the University of California Berkeley you University of California Berkeley you University of California Berkeley you got your PhD in the 80s and then do you got your PhD in the 80s and then do you got your PhD in the 80s and then do you also teach at the University of Illinois also teach at the University of Illinois also teach at the University of Illinois at Urbana champagne sir yes uh I taught at Urbana champagne sir yes uh I taught at Urbana champagne sir yes uh I taught there for 34 years before oh my goodness there for 34 years before oh my goodness there for 34 years before oh my goodness I graduated and uh went to Nvidia that's I graduated and uh went to Nvidia that's I graduated and uh went to Nvidia that's wonderful so um you have a uh a wonderful so um you have a uh a wonderful so um you have a uh a wonderful paper that we'll put a link to wonderful paper that we'll put a link to wonderful paper that we'll put a link to in the show notes in the show notes in the show notes called Running on parallelism and a few

called Running on parallelism and a few called Running on parallelism and a few Lessons Learned along the way and I love Lessons Learned along the way and I love Lessons Learned along the way and I love that it starts on the first slide with a that it starts on the first slide with a that it starts on the first slide with a list of years and people it's it's so list of years and people it's it's so list of years and people it's it's so refreshing and wonderful to see the refreshing and wonderful to see the refreshing and wonderful to see the acknowledgement of the history of the acknowledgement of the history of the acknowledgement of the history of the space as the first slide was that a space as the first slide was that a space as the first slide was that a conscious decision to lead with that yes conscious decision to lead with that yes conscious decision to lead with that yes uh uh this actually was the uh set of uh uh this actually was the uh set of uh uh this actually was the uh set of slides that I use uh after I received slides that I use uh after I received slides that I use uh after I received the uh ACM I e marle award and um uh so the uh ACM I e marle award and um uh so the uh ACM I e marle award and um uh so basically when the uh the committee basically when the uh the committee basically when the uh the committee chair called me um you know I told me chair called me um you know I told me chair called me um you know I told me that I would be the recipient um you that I would be the recipient um you that I would be the recipient um you know that it took me a little bit time know that it took me a little bit time know that it took me a little bit time to accept it that it if you look at the to accept it that it if you look at the to accept it that it if you look at the list of people that I had these are the list of people that I had these are the list of people that I had these are the people who literally were building the people who literally were building the people who literally were building the historic machines and um uh you the historic machines and um uh you the historic machines and um uh you the publishing extremely influential work publishing extremely influential work publishing extremely influential work along in the history many of them are along in the history many of them are along in the history many of them are the authors of the textbook that I read the authors of the textbook that I read the authors of the textbook that I read when I was gr student so the being uh so when I was gr student so the being uh so when I was gr student so the being uh so I wanted to put that slide up in the I wanted to put that slide up in the I wanted to put that slide up in the context before I talk about the any context before I talk about the any context before I talk about the any thoughts after receiving that award that thoughts after receiving that award that thoughts after receiving that award that these are the people that U that I was these are the people that U that I was these are the people that U that I was standing on their shoulder I do my work standing on their shoulder I do my work standing on their shoulder I do my work that that's so wonderful because it's that that's so wonderful because it's that that's so wonderful because it's such an amazing group and to be such an amazing group and to be such an amazing group and to be acknowledged in the same in the same acknowledged in the same in the same acknowledged in the same in the same list in the same breath going back so list in the same breath going back so list in the same breath going back so many years must feel really uh

many years must feel really uh many years must feel really uh amazing the uh the the discussion is amazing the uh the the discussion is amazing the uh the the discussion is about parallelism and where it fits into about parallelism and where it fits into about parallelism and where it fits into like the historical context of things like the historical context of things like the historical context of things and if we think about Moore's Law and if we think about Moore's Law and if we think about Moore's Law throwing transistors at the problem uh throwing transistors at the problem uh throwing transistors at the problem uh and maybe starting to flatten out we're and maybe starting to flatten out we're and maybe starting to flatten out we're seeing Computing in general start to seeing Computing in general start to seeing Computing in general start to move towards not just parallelism but move towards not just parallelism but move towards not just parallelism but massive massive parallel ism is that massive massive parallel ism is that massive massive parallel ism is that because we're starting to hit some some because we're starting to hit some some because we're starting to hit some some physical limitation some Quantum physical limitation some Quantum physical limitation some Quantum limitations we can only throw so much limitations we can only throw so much limitations we can only throw so much power at these things we have to start power at these things we have to start power at these things we have to start to go wide yeah the uh I actually kind to go wide yeah the uh I actually kind to go wide yeah the uh I actually kind of lived through the whole cycle uh you of lived through the whole cycle uh you of lived through the whole cycle uh you the uh in my professional career when I the uh in my professional career when I the uh in my professional career when I started as a grass student uh in 1983 um started as a grass student uh in 1983 um started as a grass student uh in 1983 um you know it was really the I would call you know it was really the I would call you know it was really the I would call the uh the real beginning of the mors the uh the real beginning of the mors the uh the real beginning of the mors law and um uh later on uh you know we law and um uh later on uh you know we law and um uh later on uh you know we also understood the Nar scaling and uh also understood the Nar scaling and uh also understood the Nar scaling and uh these two things U you know are kind of these two things U you know are kind of these two things U you know are kind of a uh incredibly important for the next a uh incredibly important for the next a uh incredibly important for the next um I would say 20 years so the uh mors um I would say 20 years so the uh mors um I would say 20 years so the uh mors law basically uh you drives the industry law basically uh you drives the industry law basically uh you drives the industry to reduce the uh transistor size every to reduce the uh transistor size every to reduce the uh transistor size every you know the you you can argue 18 months you know the you you can argue 18 months you know the you you can argue 18 months or two years and um so the basically um or two years and um so the basically um or two years and um so the basically um you know that allows people to to pack you know that allows people to to pack you know that allows people to to pack in uh you know twice the amount of in uh you know twice the amount of in uh you know twice the amount of transistors per chip without increasing transistors per chip without increasing transistors per chip without increasing the size of the chip right that that's

the size of the chip right that that's the size of the chip right that that's that's you know economically very very that's you know economically very very that's you know economically very very good but the but from uh for My Career good but the but from uh for My Career good but the but from uh for My Career the Nar scaling was even more important the Nar scaling was even more important the Nar scaling was even more important the Nar scaling uh you as you reduce the the Nar scaling uh you as you reduce the the Nar scaling uh you as you reduce the transition size uh you will reduce the transition size uh you will reduce the transition size uh you will reduce the voltage by um you know the square one voltage by um you know the square one voltage by um you know the square one over square root and the you uh reduce over square root and the you uh reduce over square root and the you uh reduce basically uh as a result um you can basically uh as a result um you can basically uh as a result um you can reduce the uh the power consumption uh reduce the uh the power consumption uh reduce the uh the power consumption uh you know per chip area by one/ square you know per chip area by one/ square you know per chip area by one/ square root and uh you can improve the speed of root and uh you can improve the speed of root and uh you can improve the speed of the transistor or the reduce the uh the transistor or the reduce the uh the transistor or the reduce the uh switching time and uh Time by uh you switching time and uh Time by uh you switching time and uh Time by uh you know one over square root of two and um know one over square root of two and um know one over square root of two and um you can still have more transistors you can still have more transistors you can still have more transistors right so so that is incredibly important right so so that is incredibly important right so so that is incredibly important because for the next 15 years people because for the next 15 years people because for the next 15 years people literally say that uh they can write a literally say that uh they can write a literally say that uh they can write a piece of code go to sleep for you know piece of code go to sleep for you know piece of code go to sleep for you know five years and wake up that piece of five years and wake up that piece of five years and wake up that piece of code will run much faster without doing code will run much faster without doing code will run much faster without doing anything to it and um uh this is great anything to it and um uh this is great anything to it and um uh this is great because during my grad school days uh we because during my grad school days uh we because during my grad school days uh we actually uh you know I went to un actually uh you know I went to un actually uh you know I went to un University of California Berkeley and University of California Berkeley and University of California Berkeley and that was a time when we really debated that was a time when we really debated that was a time when we really debated what the future course of what the future course of what the future course of microprocessors will be that's when uh microprocessors will be that's when uh microprocessors will be that's when uh Hennessy and Patterson were advocating

Hennessy and Patterson were advocating Hennessy and Patterson were advocating risk and that was a you know that there risk and that was a you know that there risk and that was a you know that there were huge number of discussions but the were huge number of discussions but the were huge number of discussions but the uh the research group that my advisor uh the research group that my advisor uh the research group that my advisor Yale Pat and some of my fellow grass Yale Pat and some of my fellow grass Yale Pat and some of my fellow grass students look sort of beyond the students look sort of beyond the students look sort of beyond the immediate next five years and look to immediate next five years and look to immediate next five years and look to the 10 15 years and say well um you know the 10 15 years and say well um you know the 10 15 years and say well um you know if we have so many transistors and these if we have so many transistors and these if we have so many transistors and these transistors are going to be running so transistors are going to be running so transistors are going to be running so fast how about if we use these fast how about if we use these fast how about if we use these transistors to detect the parallelism transistors to detect the parallelism transistors to detect the parallelism that the that you know nobody can that the that you know nobody can that the that you know nobody can currently detect in their processors and currently detect in their processors and currently detect in their processors and use the parallelism to run things much use the parallelism to run things much use the parallelism to run things much faster so and we got lucky we got lucky faster so and we got lucky we got lucky faster so and we got lucky we got lucky that uh by that uh by that uh by 1993 when Intel designed the first P6 1993 when Intel designed the first P6 1993 when Intel designed the first P6 the more and thear scaling got to the the more and thear scaling got to the the more and thear scaling got to the point where they can design such a point where they can design such a point where they can design such a processor based on some of the processor based on some of the processor based on some of the intellectual research we did in the intellectual research we did in the intellectual research we did in the 1980s and many many other people did you 1980s and many many other people did you 1980s and many many other people did you know after us so that they can build the know after us so that they can build the know after us so that they can build the P6 which is the pendan uh the pro P6 which is the pendan uh the pro P6 which is the pendan uh the pro processor and that processor embodied um processor and that processor embodied um processor and that processor embodied um you know all the parallelism mechanisms you know all the parallelism mechanisms you know all the parallelism mechanisms that we envisioned and um you know that that we envisioned and um you know that that we envisioned and um you know that that that was history and uh one of the that that was history and uh one of the that that was history and uh one of the interesting lessons we learn is that U interesting lessons we learn is that U interesting lessons we learn is that U you when we do you when we do you when we do research U we really need to understand research U we really need to understand research U we really need to understand where the technology is going to be 10

where the technology is going to be 10 where the technology is going to be 10 years from now because by the time years from now because by the time years from now because by the time people put any of your research ideas people put any of your research ideas people put any of your research ideas into this Hardware it will be at least into this Hardware it will be at least into this Hardware it will be at least five years if not 10 years yeah so for five years if not 10 years yeah so for five years if not 10 years yeah so for folks that are listening who may not be folks that are listening who may not be folks that are listening who may not be familiar with some of these terms I want familiar with some of these terms I want familiar with some of these terms I want to call out some things that you said to call out some things that you said to call out some things that you said that folks can go and read more about so that folks can go and read more about so that folks can go and read more about so dinard scaling also known as mosfet dinard scaling also known as mosfet dinard scaling also known as mosfet scaling that was scaling that was scaling that was 1974 when that was uh being thought 1974 when that was uh being thought 1974 when that was uh being thought about and that held as you pointed out about and that held as you pointed out about and that held as you pointed out for 25 plus years until it started to for 25 plus years until it started to for 25 plus years until it started to kind of falter in the early 2000s there kind of falter in the early 2000s there kind of falter in the early 2000s there and then with modern gpus you're looking and then with modern gpus you're looking and then with modern gpus you're looking for ways to get around it stacking chips for ways to get around it stacking chips for ways to get around it stacking chips coming up with new ways to cool uh coming up with new ways to cool uh coming up with new ways to cool uh because you're you may be in with because you're you may be in with because you're you may be in with getting higher performance without power getting higher performance without power getting higher performance without power consumption but you're still getting consumption but you're still getting consumption but you're still getting power leakage you're getting heat power leakage you're getting heat power leakage you're getting heat dissipation you're pushing up against dissipation you're pushing up against dissipation you're pushing up against the laws of physics the laws of physics the laws of physics itself you say though you're thinking 10 itself you say though you're thinking 10 itself you say though you're thinking 10 years out is there a years out is there a years out is there a limit is there a possible theoretical limit is there a possible theoretical limit is there a possible theoretical limit where you will actually say nope limit where you will actually say nope limit where you will actually say nope that is as far as this is going to go I that is as far as this is going to go I that is as far as this is going to go I think there is but we don't know exactly think there is but we don't know exactly think there is but we don't know exactly when it's going to hit um you know this when it's going to hit um you know this when it's going to hit um you know this is like a a a stock that keeps going up is like a a a stock that keeps going up is like a a a stock that keeps going up everyone says it's going to go down at everyone says it's going to go down at everyone says it's going to go down at some point but nobody knows when right some point but nobody knows when right some point but nobody knows when right so the uh you we have been saying that so the uh you we have been saying that so the uh you we have been saying that the uh the N scaling will be ending and the uh the N scaling will be ending and the uh the N scaling will be ending and which ended so the the uh the whole which ended so the the uh the whole which ended so the the uh the whole dinar scaling ended uh in early 2000 as dinar scaling ended uh in early 2000 as dinar scaling ended uh in early 2000 as you mentioned but uh uh for Mor law U

you mentioned but uh uh for Mor law U you mentioned but uh uh for Mor law U you know we can probably see at least you know we can probably see at least you know we can probably see at least another two generations and uh but we another two generations and uh but we another two generations and uh but we are already at the point where we're are already at the point where we're are already at the point where we're building computers very differently than building computers very differently than building computers very differently than scaling on the chip and Scott you scaling on the chip and Scott you scaling on the chip and Scott you already mentioned that we try to get a already mentioned that we try to get a already mentioned that we try to get a stack memory with we try to you know do stack memory with we try to you know do stack memory with we try to you know do the chip packaging but uh I think most the chip packaging but uh I think most the chip packaging but uh I think most of the Innovations today if you look at of the Innovations today if you look at of the Innovations today if you look at what the the big Innovations are for what the the big Innovations are for what the the big Innovations are for today's research we are already into today's research we are already into today's research we are already into building very very big systems uh with building very very big systems uh with building very very big systems uh with reasonable latency so the you know we're reasonable latency so the you know we're reasonable latency so the you know we're going to see big changes in networking going to see big changes in networking going to see big changes in networking and um especially you know the going and um especially you know the going and um especially you know the going from you know electrical to Optical and from you know electrical to Optical and from you know electrical to Optical and we're going to see you know the uh very we're going to see you know the uh very we're going to see you know the uh very different uh system communication different uh system communication different uh system communication Primitives that allow us to synchronize Primitives that allow us to synchronize Primitives that allow us to synchronize a huge number of machines fast and these a huge number of machines fast and these a huge number of machines fast and these are all the you know all because we are all the you know all because we are all the you know all because we already have you know we are already already have you know we are already already have you know we are already seeing that coming right that we don't seeing that coming right that we don't seeing that coming right that we don't know exactly when this will come but uh know exactly when this will come but uh know exactly when this will come but uh we need to run as fast as we can so that we need to run as fast as we can so that we need to run as fast as we can so that we're ready when they come and chances we're ready when they come and chances we're ready when they come and chances are we may not even be totally ready are we may not even be totally ready are we may not even be totally ready when it comes yeah this might be an when it comes yeah this might be an when it comes yeah this might be an interesting question or a dumb question interesting question or a dumb question interesting question or a dumb question but we we hear about people being full but we we hear about people being full but we we hear about people being full stacked engineers and you're probably stacked engineers and you're probably stacked engineers and you're probably not thinking about node or llang or not thinking about node or llang or not thinking about node or llang or JavaScript at the highest level when JavaScript at the highest level when JavaScript at the highest level when someone is a a software engineer for someone is a a software engineer for someone is a a software engineer for them low level might be going into

them low level might be going into them low level might be going into Google Chrome and hitting F12 but for Google Chrome and hitting F12 but for Google Chrome and hitting F12 but for you you're starting to bump up against you you're starting to bump up against you you're starting to bump up against like protons and electrons moving around like protons and electrons moving around like protons and electrons moving around in space but the software stack overhead in space but the software stack overhead in space but the software stack overhead has us wondering even if I as I sit here has us wondering even if I as I sit here has us wondering even if I as I sit here on a highly parallel machine why does my on a highly parallel machine why does my on a highly parallel machine why does my machine feel slow yes are you thinking machine feel slow yes are you thinking machine feel slow yes are you thinking about software paradigms and how people about software paradigms and how people about software paradigms and how people are going to program in five or 10 years are going to program in five or 10 years are going to program in five or 10 years and applying that to how you're going to and applying that to how you're going to and applying that to how you're going to create these pieces these chips con uh create these pieces these chips con uh create these pieces these chips con uh absolutely um there there are two absolutely um there there are two absolutely um there there are two important forces in these kind of things important forces in these kind of things important forces in these kind of things right one is reality um you know people right one is reality um you know people right one is reality um you know people uh there are huge number of people uh there are huge number of people uh there are huge number of people programming with python right and the so programming with python right and the so programming with python right and the so when we uh when we deal with uh when we uh when we deal with uh when we uh when we deal with uh Frameworks based on python such as torch Frameworks based on python such as torch Frameworks based on python such as torch um we see all kinds of interesting not um we see all kinds of interesting not um we see all kinds of interesting not just overhead but the uh you the just overhead but the uh you the just overhead but the uh you the bottlenecks for example uh the python bottlenecks for example uh the python bottlenecks for example uh the python interpreter lock right that that interpreter lock right that that interpreter lock right that that fundamentally restrict uh restricts the fundamentally restrict uh restricts the fundamentally restrict uh restricts the kind of parallelism we can expect at the kind of parallelism we can expect at the kind of parallelism we can expect at the source language level and uh more subtly source language level and uh more subtly source language level and uh more subtly um all the GPU um all the GPU um all the GPU activities um that are launched through activities um that are launched through activities um that are launched through the the python frame based Frameworks the the python frame based Frameworks the the python frame based Frameworks are based on apis and what that means is are based on apis and what that means is are based on apis and what that means is that these API CS are separated from that these API CS are separated from that these API CS are separated from each other okay they're not U you know each other okay they're not U you know each other okay they're not U you know they're not adjacent to each other in

they're not adjacent to each other in they're not adjacent to each other in terms of the uh the the compolation terms of the uh the the compolation terms of the uh the the compolation process or the you interpretation process or the you interpretation process or the you interpretation process we're seeing these individual process we're seeing these individual process we're seeing these individual activities so one of the important activities so one of the important activities so one of the important important things that we would uh will important things that we would uh will important things that we would uh will begin to see is how the uh you know how begin to see is how the uh you know how begin to see is how the uh you know how the lower level software and Hardware the lower level software and Hardware the lower level software and Hardware combination will be able to take this combination will be able to take this combination will be able to take this API costs and maybe doing things like API costs and maybe doing things like API costs and maybe doing things like fusing these activities on the Fly and fusing these activities on the Fly and fusing these activities on the Fly and you know optimizing away some of the you you know optimizing away some of the you you know optimizing away some of the you know inefficiencies because these are know inefficiencies because these are know inefficiencies because these are isolated events but these are the kind isolated events but these are the kind isolated events but these are the kind of things that people have been working of things that people have been working of things that people have been working on for decades and uh you know starting on for decades and uh you know starting on for decades and uh you know starting from the Java days and uh you know uh from the Java days and uh you know uh from the Java days and uh you know uh Microsoft has you know the roundtime Microsoft has you know the roundtime Microsoft has you know the roundtime compilation activities for a long long compilation activities for a long long compilation activities for a long long time you know the the the uh the RT uh time you know the the the uh the RT uh time you know the the the uh the RT uh you know all the uh WR time uh you know you know all the uh WR time uh you know you know all the uh WR time uh you know uh Linker optimizers so the but I think uh Linker optimizers so the but I think uh Linker optimizers so the but I think we were getting getting to the point we were getting getting to the point we were getting getting to the point where some of these technologies will where some of these technologies will where some of these technologies will begin to really be deployed uh in the begin to really be deployed uh in the begin to really be deployed uh in the next decade or so for the you know next decade or so for the you know next decade or so for the you know because we are running out of room for because we are running out of room for because we are running out of room for further optimiz Iz ations without further optimiz Iz ations without further optimiz Iz ations without deploying these kind of deploying these kind of deploying these kind of techniques hey friends in the interest techniques hey friends in the interest techniques hey friends in the interest of me learning more about AI for the of me learning more about AI for the of me learning more about AI for the next couple of episodes I'm going to next couple of episodes I'm going to next couple of episodes I'm going to have a little thing in the Middle where have a little thing in the Middle where have a little thing in the Middle where I'm going to bring lanver Dan VOR from I'm going to bring lanver Dan VOR from I'm going to bring lanver Dan VOR from Qualcomm in to help me understand more

Qualcomm in to help me understand more Qualcomm in to help me understand more about AI now at Qualcomm you've really about AI now at Qualcomm you've really about AI now at Qualcomm you've really taken a developer first Approach at taken a developer first Approach at taken a developer first Approach at getting folks excited about the Silicon getting folks excited about the Silicon getting folks excited about the Silicon that you're creating why why do you do that you're creating why why do you do that you're creating why why do you do that oh well in it developers basically that oh well in it developers basically that oh well in it developers basically add value to your platform add value to your platform add value to your platform right it's uh allowing them to enable or right it's uh allowing them to enable or right it's uh allowing them to enable or to use your Hardware they're going to to use your Hardware they're going to to use your Hardware they're going to come up with new applications they're come up with new applications they're come up with new applications they're going to come up with new use cases that going to come up with new use cases that going to come up with new use cases that you've never thought of and they add you've never thought of and they add you've never thought of and they add value to that and the key component of value to that and the key component of value to that and the key component of allowing developers to do this is to allowing developers to do this is to allowing developers to do this is to make sure that they've got the make sure that they've got the make sure that they've got the information available that they need and information available that they need and information available that they need and that's a big thing we refer to as that's a big thing we refer to as that's a big thing we refer to as developer first at qualcom but it's developer first at qualcom but it's developer first at qualcom but it's basically making things available so you basically making things available so you basically making things available so you can just download it you don't have to can just download it you don't have to can just download it you don't have to sign licenses the tools are there the sign licenses the tools are there the sign licenses the tools are there the open source tools are there that is in open source tools are there that is in open source tools are there that is in short what's developed first and why short what's developed first and why short what's developed first and why it's necessary yeah and engaging more it's necessary yeah and engaging more it's necessary yeah and engaging more deeply with the community I love that deeply with the community I love that deeply with the community I love that you are showing up in all of those you are showing up in all of those you are showing up in all of those spaces and folks can check out more at spaces and folks can check out more at spaces and folks can check out more at qualcom qualcom qualcom tocom developer uh to download those tocom developer uh to download those tocom developer uh to download those things and learn more things and learn more things and learn more today it seems like uh if I call if I today it seems like uh if I call if I today it seems like uh if I call if I recall when I was uh getting my first recall when I was uh getting my first recall when I was uh getting my first 386 there was the SX and the DX and then 386 there was the SX and the DX and then 386 there was the SX and the DX and then we had ones with a co-processor and we had ones with a co-processor and we had ones with a co-processor and companies would say we need to figure companies would say we need to figure companies would say we need to figure out something for that co-processor to out something for that co-processor to out something for that co-processor to do maybe people will buy it if they use do maybe people will buy it if they use do maybe people will buy it if they use Excel so then we would buy this computer Excel so then we would buy this computer Excel so then we would buy this computer because I want to use Excel when I because I want to use Excel when I because I want to use Excel when I wanted to play Quake or Doom I would buy

wanted to play Quake or Doom I would buy wanted to play Quake or Doom I would buy a 3d effect card because I wanted that a 3d effect card because I wanted that a 3d effect card because I wanted that yes uh when I want to play a great game yes uh when I want to play a great game yes uh when I want to play a great game now I'll buy an Nvidia you know now I'll buy an Nvidia you know now I'll buy an Nvidia you know processor I'm on a machine right now processor I'm on a machine right now processor I'm on a machine right now with a 4080 super and I I'm I'm thinking with a 4080 super and I I'm I'm thinking with a 4080 super and I I'm I'm thinking to myself to myself to myself what is that really good at that my what is that really good at that my what is that really good at that my Intel chip is not very good at and if I Intel chip is not very good at and if I Intel chip is not very good at and if I look to my right here I have a co-pilot look to my right here I have a co-pilot look to my right here I have a co-pilot PC with an npu from a competitor from PC with an npu from a competitor from PC with an npu from a competitor from Qualcomm so now suddenly it's not about Qualcomm so now suddenly it's not about Qualcomm so now suddenly it's not about a CPU and a co-processor I've got three a CPU and a co-processor I've got three a CPU and a co-processor I've got three very competent different kinds of very competent different kinds of very competent different kinds of processing units here do you think processing units here do you think processing units here do you think there'll only be three will I come into there'll only be three will I come into there'll only be three will I come into a world where there's five or six or 10 a world where there's five or six or 10 a world where there's five or six or 10 different fundamentally interesting different fundamentally interesting different fundamentally interesting processor units that do different tasks processor units that do different tasks processor units that do different tasks uh that's a that I think that's a uh that's a that I think that's a uh that's a that I think that's a billion dollar question so the uh I can billion dollar question so the uh I can billion dollar question so the uh I can tell you what the what that I think tell you what the what that I think tell you what the what that I think personally right um so the I think there personally right um so the I think there personally right um so the I think there will be at least more one more but um uh will be at least more one more but um uh will be at least more one more but um uh this uh this has to do with the what you this uh this has to do with the what you this uh this has to do with the what you want to do the Scott you you you hit the want to do the Scott you you you hit the want to do the Scott you you you hit the nail on the on the head um people don't nail on the on the head um people don't nail on the on the head um people don't want processors people want applications want processors people want applications want processors people want applications right people buy processors reluctantly right people buy processors reluctantly right people buy processors reluctantly because they want to run their because they want to run their because they want to run their applications well right so today U you applications well right so today U you applications well right so today U you the I I I would say that the people buy the I I I would say that the people buy the I I I would say that the people buy know know know gpus because they want to train models gpus because they want to train models gpus because they want to train models and the the these models you know that and the the these models you know that and the the these models you know that have all these you know the all the

have all these you know the all the have all these you know the all the training needs that people buy these training needs that people buy these training needs that people buy these gpus gpus gpus reluctantly so that I think you know I reluctantly so that I think you know I reluctantly so that I think you know I I'll turn your question just a little I'll turn your question just a little I'll turn your question just a little bit and and you know maybe say that the bit and and you know maybe say that the bit and and you know maybe say that the Reas uh I think there will be another Reas uh I think there will be another Reas uh I think there will be another kind of uh you know processing uh unit kind of uh you know processing uh unit kind of uh you know processing uh unit is because at some point people would is because at some point people would is because at some point people would want to be able to uh to deal with want to be able to uh to deal with want to be able to uh to deal with massive amount of data uh on the massive amount of data uh on the massive amount of data uh on the firsthand basis uh what I mean is that firsthand basis uh what I mean is that firsthand basis uh what I mean is that uh you we are going through the machine uh you we are going through the machine uh you we are going through the machine learning um you know uh era where we learning um you know uh era where we learning um you know uh era where we reduce all that data the the training reduce all that data the the training reduce all that data the the training data into these models right and uh then data into these models right and uh then data into these models right and uh then uh we converted the query process into a uh we converted the query process into a uh we converted the query process into a uh computation in U inference through uh computation in U inference through uh computation in U inference through this model and we get the information this model and we get the information this model and we get the information but uh we kind of bypass the need of but uh we kind of bypass the need of but uh we kind of bypass the need of finding data in real time Google has finding data in real time Google has finding data in real time Google has been doing search and Microsoft has been been doing search and Microsoft has been been doing search and Microsoft has been doing search and these search are doing search and these search are doing search and these search are realtime searches right these you know realtime searches right these you know realtime searches right these you know the if you use bin if you to use Google the if you use bin if you to use Google the if you use bin if you to use Google all these things are pre-index data that all these things are pre-index data that all these things are pre-index data that we hope to serve the user real time and we hope to serve the user real time and we hope to serve the user real time and um so one thing that uh that is very um so one thing that uh that is very um so one thing that uh that is very very interesting is that um you know we very interesting is that um you know we very interesting is that um you know we kind of Swing the pendulant so much in

kind of Swing the pendulant so much in kind of Swing the pendulant so much in the uh in the you know deep learning the uh in the you know deep learning the uh in the you know deep learning models and the large language models models and the large language models models and the large language models that um uh you know we should begin to that um uh you know we should begin to that um uh you know we should begin to think about what happens if people don't think about what happens if people don't think about what happens if people don't need to the you these models have their need to the you these models have their need to the you these models have their limitations they you know the and what limitations they you know the and what limitations they you know the and what if people want to be able to go straight if people want to be able to go straight if people want to be able to go straight to the data and uh but use some of the to the data and uh but use some of the to the data and uh but use some of the model uh capabilities to be able to find model uh capabilities to be able to find model uh capabilities to be able to find the data much better than the first the data much better than the first the data much better than the first previous generation search so I can previous generation search so I can previous generation search so I can imagine that the uh in order to do this imagine that the uh in order to do this imagine that the uh in order to do this we will need to have a unit that is we will need to have a unit that is we will need to have a unit that is going to be able to tolerate latency going to be able to tolerate latency going to be able to tolerate latency and and and introduce um you know very very introduce um you know very very introduce um you know very very efficient uh you know the ways to grab efficient uh you know the ways to grab efficient uh you know the ways to grab let's let's let's say one kilobyte of data out of four say one kilobyte of data out of four say one kilobyte of data out of four pedabytes of data right and um and and pedabytes of data right and um and and pedabytes of data right and um and and serve it to the user in real serve it to the user in real serve it to the user in real time and uh go through some of the time and uh go through some of the time and uh go through some of the models to give user very models to give user very models to give user very diges okay yeah solving the needle in a diges okay yeah solving the needle in a diges okay yeah solving the needle in a Hy stack problem uh at SC because that Hy stack problem uh at SC because that Hy stack problem uh at SC because that traditional you mentioned this in the traditional you mentioned this in the traditional you mentioned this in the paper in this in the presentation this paper in this in the presentation this paper in this in the presentation this this memory storage divide it feels like this memory storage divide it feels like this memory storage divide it feels like the majority of the work that my the majority of the work that my the majority of the work that my computer is doing as a user is simply computer is doing as a user is simply computer is doing as a user is simply moving data from one format to another moving data from one format to another moving data from one format to another it's just transformation and that's it's just transformation and that's it's just transformation and that's becoming quite tedious as I simply want becoming quite tedious as I simply want becoming quite tedious as I simply want to work or I want to enjoy my computer

to work or I want to enjoy my computer to work or I want to enjoy my computer absolutely so the and and uh right now absolutely so the and and uh right now absolutely so the and and uh right now we have all these file system software we have all these file system software we have all these file system software stacks and so on that you that we built stacks and so on that you that we built stacks and so on that you that we built over the years over the years over the years that are standing in the way and the that are standing in the way and the that are standing in the way and the format that you're talking about are the format that you're talking about are the format that you're talking about are the imposed partly by these kind of selfer imposed partly by these kind of selfer imposed partly by these kind of selfer Stack but you know what people really Stack but you know what people really Stack but you know what people really want is their data people don't want the want is their data people don't want the want is their data people don't want the file system people don't want the format file system people don't want the format file system people don't want the format right and how do we how do we get people right and how do we how do we get people right and how do we how do we get people what they want and I think that's going what they want and I think that's going what they want and I think that's going to be the next unit yeah and it seems to be the next unit yeah and it seems to be the next unit yeah and it seems like we're we're narrowing the space like we're we're narrowing the space like we're we're narrowing the space between our the work and the data we're between our the work and the data we're between our the work and the data we're trying to we have that storage trying to we have that storage trying to we have that storage throughput we have that latency that is throughput we have that latency that is throughput we have that latency that is being applied being applied being applied uh we're setting up little islands where uh we're setting up little islands where uh we're setting up little islands where your parallelism and your data that it's your parallelism and your data that it's your parallelism and your data that it's running on are near each other so as to running on are near each other so as to running on are near each other so as to avoid that that latency which is now avoid that that latency which is now avoid that that latency which is now down to you know what Pico seconds it's down to you know what Pico seconds it's down to you know what Pico seconds it's down to nothing it's down you know as down to nothing it's down you know as down to nothing it's down you know as little latency as possible absolutely little latency as possible absolutely little latency as possible absolutely yeah I've seen you mentioned millisecond yeah I've seen you mentioned millisecond yeah I've seen you mentioned millisecond level latency and terabytes a second level latency and terabytes a second level latency and terabytes a second throughput uh it seems like a dream but throughput uh it seems like a dream but throughput uh it seems like a dream but it's going to happen at some point I it's going to happen at some point I it's going to happen at some point I really believe that if we do the work really believe that if we do the work really believe that if we do the work right um you know when someone does this right um you know when someone does this right um you know when someone does this interview 10 years from now um you know interview 10 years from now um you know interview 10 years from now um you know the person may have a machine on the the person may have a machine on the the person may have a machine on the desktop even right I I can see how we desktop even right I I can see how we desktop even right I I can see how we can build these kind of things in the can build these kind of things in the can build these kind of things in the data center at a very high cost but the data center at a very high cost but the data center at a very high cost but the real test is that can we put this kind real test is that can we put this kind real test is that can we put this kind of capability into every person's uh you

of capability into every person's uh you of capability into every person's uh you know desktop or even um I would say even know desktop or even um I would say even know desktop or even um I would say even a small device that uh uh that the a small device that uh uh that the a small device that uh uh that the person carries that has the entire uh person carries that has the entire uh person carries that has the entire uh lifetimes memory and the information lifetimes memory and the information lifetimes memory and the information that you can retrieve real time when you that you can retrieve real time when you that you can retrieve real time when you talk to someone it's it's interesting talk to someone it's it's interesting talk to someone it's it's interesting that you mention that because uh just that you mention that because uh just that you mention that because uh just literally yesterday and I'm going to literally yesterday and I'm going to literally yesterday and I'm going to take my headphones off and grab take my headphones off and grab take my headphones off and grab something off of my my floor here just something off of my my floor here just something off of my my floor here just yesterday A friend of mine sent me a yesterday A friend of mine sent me a yesterday A friend of mine sent me a package and for those of you who are package and for those of you who are package and for those of you who are listening to this podcast you won't see listening to this podcast you won't see listening to this podcast you won't see this but they I'm holding up a a compact this but they I'm holding up a a compact this but they I'm holding up a a compact PC companion uh and some of these early PC companion uh and some of these early PC companion uh and some of these early devices the Palm Pilot we've always devices the Palm Pilot we've always devices the Palm Pilot we've always wanted our life our our family our wanted our life our our family our wanted our life our our family our friends our photos in a tiny device like friends our photos in a tiny device like friends our photos in a tiny device like this ah yes and I you know I was there this ah yes and I you know I was there this ah yes and I you know I was there my first applications were on the Palm my first applications were on the Palm my first applications were on the Palm Pilot uh you know this is a pocket Pilot uh you know this is a pocket Pilot uh you know this is a pocket supercomputer with its own GPU and now supercomputer with its own GPU and now supercomputer with its own GPU and now we're going to see AI start to move its we're going to see AI start to move its we're going to see AI start to move its way into our pockets the idea that what way into our pockets the idea that what way into our pockets the idea that what once filled a room is carried around once filled a room is carried around once filled a room is carried around with me with all day battery life and to with me with all day battery life and to with me with all day battery life and to your point the data won't be in the your point the data won't be in the your point the data won't be in the cloud it could just be in my pocket cloud it could just be in my pocket cloud it could just be in my pocket exactly and I can see how that can exactly and I can see how that can exactly and I can see how that can happen you know I work with a lot of the happen you know I work with a lot of the happen you know I work with a lot of the uh SSD storage vendors uh you know on a uh SSD storage vendors uh you know on a uh SSD storage vendors uh you know on a regular basis I work with you know the regular basis I work with you know the regular basis I work with you know the some of these you know the uh you the some of these you know the uh you the some of these you know the uh you the PCB Builders and then you know some of

PCB Builders and then you know some of PCB Builders and then you know some of these you know the uh mobile chip these you know the uh mobile chip these you know the uh mobile chip builders on uh you know I can see that builders on uh you know I can see that builders on uh you know I can see that the uh if we do the right things we in the uh if we do the right things we in the uh if we do the right things we in you know within a decade we can begin to you know within a decade we can begin to you know within a decade we can begin to have these devices that truly extend us have these devices that truly extend us have these devices that truly extend us right I I really feel that the you the right I I really feel that the you the right I I really feel that the you the real benefit for these devices is to real benefit for these devices is to real benefit for these devices is to make each of us a infinite memory cap make each of us a infinite memory cap make each of us a infinite memory cap and person and you know then but it can and person and you know then but it can and person and you know then but it can complement some of the you know some of complement some of the you know some of complement some of the you know some of some of the uh you know let's say just some of the uh you know let's say just some of the uh you know let's say just like calculators complemented our you like calculators complemented our you like calculators complemented our you know way to do arithmetic we should be know way to do arithmetic we should be know way to do arithmetic we should be able to you know remember the names of able to you know remember the names of able to you know remember the names of all our you know the associates and all our you know the associates and all our you know the associates and colleagues and so on in real time no colleagues and so on in real time no colleagues and so on in real time no problem right right now I can't I was I problem right right now I can't I was I problem right right now I can't I was I was looking at some uh some pictures of was looking at some uh some pictures of was looking at some uh some pictures of a a former US president who was being a a former US president who was being a a former US president who was being introduced to some dignitaries and you introduced to some dignitaries and you introduced to some dignitaries and you could tell that someone was whispering could tell that someone was whispering could tell that someone was whispering that's the president of France and this that's the president of France and this that's the president of France and this is his wife and their children's names is his wife and their children's names is his wife and their children's names are that's what I would like for my for are that's what I would like for my for are that's what I would like for my for my co-workers for my colleagues for my co-workers for my colleagues for my co-workers for my colleagues for people I haven't seen in 10 years is people I haven't seen in 10 years is people I haven't seen in 10 years is someone to whisper in my ear but right someone to whisper in my ear but right someone to whisper in my ear but right now to do that I have to go to the cloud now to do that I have to go to the cloud now to do that I have to go to the cloud interact with a vendor that's fast interact with a vendor that's fast interact with a vendor that's fast sparse access to massive data yes and sparse access to massive data yes and sparse access to massive data yes and who knows what that cost in power who knows what that cost in power who knows what that cost in power consumption as I go to Google photos consumption as I go to Google photos consumption as I go to Google photos today and I say show me pictures of my today and I say show me pictures of my today and I say show me pictures of my child in the snow when they were two yes child in the snow when they were two yes child in the snow when they were two yes you're saying that algorithms memory

you're saying that algorithms memory you're saying that algorithms memory capacity the theoretical limits will capacity the theoretical limits will capacity the theoretical limits will happen such that we could have that in happen such that we could have that in happen such that we could have that in our pocket and not only that you don't our pocket and not only that you don't our pocket and not only that you don't need to scroll through 100 results and need to scroll through 100 results and need to scroll through 100 results and pick the best one that's amazing that's pick the best one that's amazing that's pick the best one that's amazing that's what I want but the that's that's going what I want but the that's that's going what I want but the that's that's going to require silicon changes it's going to to require silicon changes it's going to to require silicon changes it's going to require algorithmic changes that's a a require algorithmic changes that's a a require algorithmic changes that's a a Beyond a generational shift that's going Beyond a generational shift that's going Beyond a generational shift that's going to get the entire industry to head in a to get the entire industry to head in a to get the entire industry to head in a different direction uh yes or no U you different direction uh yes or no U you different direction uh yes or no U you know we have been witnessing uh these know we have been witnessing uh these know we have been witnessing uh these kind of things if there's enough value kind of things if there's enough value kind of things if there's enough value uh you know these things move fast for uh you know these things move fast for uh you know these things move fast for example uh you know the I have seen the example uh you know the I have seen the example uh you know the I have seen the whole GPU whole GPU whole GPU uh the movement where the I still uh the movement where the I still uh the movement where the I still remember in remember in remember in 2011 uh when we proposed to build a 2011 uh when we proposed to build a 2011 uh when we proposed to build a superc computer called Blue Waters based superc computer called Blue Waters based superc computer called Blue Waters based on gpus um to msf and uh there was a on gpus um to msf and uh there was a on gpus um to msf and uh there was a review panel and uh people you know the review panel and uh people you know the review panel and uh people you know the uh this review panelists were all uh this review panelists were all uh this review panelists were all supercomputing Center directors who had supercomputing Center directors who had supercomputing Center directors who had Decades of experience building Decades of experience building Decades of experience building supercomputers and you know the uh supercomputers and you know the uh supercomputers and you know the uh serving uh communities with serving uh communities with serving uh communities with supercomputers right and you know after supercomputers right and you know after supercomputers right and you know after my presentation um you know the the my presentation um you know the the my presentation um you know the the person one of the panelists said this person one of the panelists said this person one of the panelists said this this proposal is risky to say the least this proposal is risky to say the least this proposal is risky to say the least and probably irresponsible um you know and probably irresponsible um you know and probably irresponsible um you know if you really look at it and as you know if you really look at it and as you know if you really look at it and as you know so I I I said why do you say this he so I I I said why do you say this he so I I I said why do you say this he said you know compare the number of math said you know compare the number of math said you know compare the number of math Library functions that gpus have versus

Library functions that gpus have versus Library functions that gpus have versus the uh the Intel math kernel library the uh the Intel math kernel library the uh the Intel math kernel library right and you're ways away from that and right and you're ways away from that and right and you're ways away from that and I said yes but we know that there only a I said yes but we know that there only a I said yes but we know that there only a limited number of science applications limited number of science applications limited number of science applications that people truly care about using that people truly care about using that people truly care about using supercomputers we know that this is supercomputers we know that this is supercomputers we know that this is going to start with more special purpose going to start with more special purpose going to start with more special purpose but if you look at the but if you look at the but if you look at the computational um speed over wattage this computational um speed over wattage this computational um speed over wattage this thing is a small puppy with big paws thing is a small puppy with big paws thing is a small puppy with big paws okay and let's take that step and okay and let's take that step and okay and let's take that step and somehow by miracle that panel agreed uh somehow by miracle that panel agreed uh somehow by miracle that panel agreed uh you allowed us to do it right and you you allowed us to do it right and you you allowed us to do it right and you know 12 uh 13 years later we're sitting know 12 uh 13 years later we're sitting know 12 uh 13 years later we're sitting here looking at a you know the uh all here looking at a you know the uh all here looking at a you know the uh all the math libraries that people already the math libraries that people already the math libraries that people already built and if you look at the model built and if you look at the model built and if you look at the model training process you know people put training process you know people put training process you know people put together multiple generations of these together multiple generations of these together multiple generations of these kind of Frameworks over the past five kind of Frameworks over the past five kind of Frameworks over the past five years right if it provides enough value years right if it provides enough value years right if it provides enough value if there's enough incentive it will if there's enough incentive it will if there's enough incentive it will happen that's amazing like there's happen that's amazing like there's happen that's amazing like there's there's the the people who have the there's the the people who have the there's the the people who have the audacity to do it once to propose a Peta audacity to do it once to propose a Peta audacity to do it once to propose a Peta scale supercomputer to get the National scale supercomputer to get the National scale supercomputer to get the National Science Foundation to think it's a good Science Foundation to think it's a good Science Foundation to think it's a good idea to give you $200 million to do it idea to give you $200 million to do it idea to give you $200 million to do it and now it's being done by others and and now it's being done by others and and now it's being done by others and that's that's that's unbelievable thank

that's that's that's unbelievable thank that's that's that's unbelievable thank you yeah thank you and then now that is you yeah thank you and then now that is you yeah thank you and then now that is it the delta project that is the next it the delta project that is the next it the delta project that is the next step of Blue Waters uh the Delta project step of Blue Waters uh the Delta project step of Blue Waters uh the Delta project is not meant to be uh the uh it it is is not meant to be uh the uh it it is is not meant to be uh the uh it it is the next step of Blue Waters in the the next step of Blue Waters in the the next step of Blue Waters in the sense that um um there's a real academic sense that um um there's a real academic sense that um um there's a real academic need uh for more of these gpus and need uh for more of these gpus and need uh for more of these gpus and there's a serious uh shortage of GPU there's a serious uh shortage of GPU there's a serious uh shortage of GPU available gpus available to academic available gpus available to academic available gpus available to academic researchers today and so Delta is is researchers today and so Delta is is researchers today and so Delta is is meant to uh fill that Gap um but Delta meant to uh fill that Gap um but Delta meant to uh fill that Gap um but Delta is not like Blue Waters where uh we we is not like Blue Waters where uh we we is not like Blue Waters where uh we we know there's so many known risks we know know there's so many known risks we know know there's so many known risks we know there's so many there's so many there's so many potential uh failures that can become potential uh failures that can become potential uh failures that can become scandals for the public and the and we scandals for the public and the and we scandals for the public and the and we had to you know to work with all the had to you know to work with all the had to you know to work with all the science teams and all the uh you know science teams and all the uh you know science teams and all the uh you know there are four science teams that I'm there are four science teams that I'm there are four science teams that I'm eternally grateful uh you know uh how eternally grateful uh you know uh how eternally grateful uh you know uh how how much work they put into their how much work they put into their how much work they put into their applications to you know to move them applications to you know to move them applications to you know to move them into gpus and into gpus and into gpus and prove the the benefits and c and Nvidia prove the the benefits and c and Nvidia prove the the benefits and c and Nvidia dedicated hundreds of people working on dedicated hundreds of people working on dedicated hundreds of people working on this project to to to to make sure that this project to to to to make sure that this project to to to to make sure that it does not fail right so Delta is not it does not fail right so Delta is not it does not fail right so Delta is not at that that kind of level Delta is at that that kind of level Delta is at that that kind of level Delta is really about how you know we can enable really about how you know we can enable really about how you know we can enable the poor uh you know academic the poor uh you know academic the poor uh you know academic researchers who can who really need the researchers who can who really need the researchers who can who really need the GPU time to be able to make some real

GPU time to be able to make some real GPU time to be able to make some real progress in the I see so it's a about progress in the I see so it's a about progress in the I see so it's a about access to those resources more than it access to those resources more than it access to those resources more than it is about pushing the boundaries of is about pushing the boundaries of is about pushing the boundaries of computational power exactly okay what computational power exactly okay what computational power exactly okay what projects are you excited about where projects are you excited about where projects are you excited about where folks can go and learn about folks can go and learn about folks can go and learn about supercomputing projects and you know supercomputing projects and you know supercomputing projects and you know audacious moonshot type things like that audacious moonshot type things like that audacious moonshot type things like that that are going to take us into the exf that are going to take us into the exf that are going to take us into the exf flops yeah uh so the you there are flops yeah uh so the you there are flops yeah uh so the you there are several ex uh excess skill projects uh several ex uh excess skill projects uh several ex uh excess skill projects uh you know the uh that is coming together you know the uh that is coming together you know the uh that is coming together and um uh you uh I you I I would say um and um uh you uh I you I I would say um and um uh you uh I you I I would say um most of them have you very interesting most of them have you very interesting most of them have you very interesting uh ideas you know the one but I think uh ideas you know the one but I think uh ideas you know the one but I think one of the uh the the biggest challenges one of the uh the the biggest challenges one of the uh the the biggest challenges that all these you know the superc that all these you know the superc that all these you know the superc computers are trying to figure out is computers are trying to figure out is computers are trying to figure out is this whole networking because you know this whole networking because you know this whole networking because you know the uh we we we do have the option to uh the uh we we we do have the option to uh the uh we we we do have the option to uh to build uh Optical Networks to build uh Optical Networks to build uh Optical Networks but these Optical networks are so but these Optical networks are so but these Optical networks are so expensive they're more than 10 times expensive they're more than 10 times expensive they're more than 10 times more expensive than there's uh than more expensive than there's uh than more expensive than there's uh than their copper equivalence that's why their copper equivalence that's why their copper equivalence that's why whenever the distances are short um you whenever the distances are short um you whenever the distances are short um you know the uh you know these Optical U know the uh you know these Optical U know the uh you know these Optical U connections are not used and um so you connections are not used and um so you connections are not used and um so you know I think if we look at the Next know I think if we look at the Next know I think if we look at the Next Generation um we're going to see you Generation um we're going to see you Generation um we're going to see you know uh I would say big movements in know uh I would say big movements in know uh I would say big movements in terms of the uh the network connection

terms of the uh the network connection terms of the uh the network connection and in terms of the cooling uh we're you and in terms of the cooling uh we're you and in terms of the cooling uh we're you know the we're definitely uh at the know the we're definitely uh at the know the we're definitely uh at the cooling limit you know the for all the cooling limit you know the for all the cooling limit you know the for all the machines I still remember you know that machines I still remember you know that machines I still remember you know that when we're calculating the amount of when we're calculating the amount of when we're calculating the amount of water that we need to pump through uh water that we need to pump through uh water that we need to pump through uh you know one of these data centers and I you know one of these data centers and I you know one of these data centers and I said can we really have that much water said can we really have that much water said can we really have that much water so so the you know the and uh I think so so the you know the and uh I think so so the you know the and uh I think some of the excess skill machines would some of the excess skill machines would some of the excess skill machines would uh you know will have very interesting uh you know will have very interesting uh you know will have very interesting you know the cooling uh mechanisms and you know the cooling uh mechanisms and you know the cooling uh mechanisms and these things will be publish you know these things will be publish you know these things will be publish you know the soon you know the as soon as these the soon you know the as soon as these the soon you know the as soon as these machines are operational I'm I'm trying machines are operational I'm I'm trying machines are operational I'm I'm trying to think about what I'm going to do as I to think about what I'm going to do as I to think about what I'm going to do as I uh as I retire as I think about uh as I retire as I think about uh as I retire as I think about retirement as I think about what are the retirement as I think about what are the retirement as I think about what are the next you know 50 years going to be are next you know 50 years going to be are next you know 50 years going to be are you going to to be doing this until you you going to to be doing this until you you going to to be doing this until you can't walk anymore are you going to do can't walk anymore are you going to do can't walk anymore are you going to do this until are you going to be a this until are you going to be a this until are you going to be a parallelism person and a Computing parallelism person and a Computing parallelism person and a Computing Pioneer uh until the end I might because Pioneer uh until the end I might because Pioneer uh until the end I might because you seem like you're having a lot of you seem like you're having a lot of you seem like you're having a lot of fun exactly and uh uh you you Scott let fun exactly and uh uh you you Scott let fun exactly and uh uh you you Scott let me tell you when I think I'm I'm going me tell you when I think I'm I'm going me tell you when I think I'm I'm going to retire um well I will retire when to retire um well I will retire when to retire um well I will retire when when I have that device in my hand and when I have that device in my hand and when I have that device in my hand and uh and I can go around and uh you know uh and I can go around and uh you know uh and I can go around and uh you know show people pretend to be the most show people pretend to be the most show people pretend to be the most intelligent person on Earth and um and intelligent person on Earth and um and intelligent person on Earth and um and you know spend my time you know you know spend my time you know you know spend my time you know explaining to people you know what how explaining to people you know what how explaining to people you know what how they can use this uh device that's they can use this uh device that's they can use this uh device that's wonderful I I I hope we both uh get to wonderful I I I hope we both uh get to wonderful I I I hope we both uh get to see that happen because that is a the

see that happen because that is a the see that happen because that is a the promise of a pocket supercomputer that promise of a pocket supercomputer that promise of a pocket supercomputer that really makes your life better makes your really makes your life better makes your really makes your life better makes your relationships better it is about the the relationships better it is about the the relationships better it is about the the things you can do with it it's about the things you can do with it it's about the things you can do with it it's about the people and the people and the people and the connections absolutely and I I really connections absolutely and I I really connections absolutely and I I really hope to live uh to see that day hope to live uh to see that day hope to live uh to see that day fantastic thank you so much sir for fantastic thank you so much sir for fantastic thank you so much sir for chatting with me today thank you Scott chatting with me today thank you Scott chatting with me today thank you Scott for this opportunity I have been for this opportunity I have been for this opportunity I have been chatting with Dr Win May who he is the chatting with Dr Win May who he is the chatting with Dr Win May who he is the senior director of research and senior director of research and senior director of research and distinguished research scientist at distinguished research scientist at distinguished research scientist at Nvidia Corporation and many Nvidia Corporation and many Nvidia Corporation and many congratulations as well sir on your congratulations as well sir on your congratulations as well sir on your award your ACM e award for your award your ACM e award for your award your ACM e award for your pioneering contributions in the design pioneering contributions in the design pioneering contributions in the design of these processor architectures we do of these processor architectures we do of these processor architectures we do appreciate you and all of the work that appreciate you and all of the work that appreciate you and all of the work that you and your colleagues have done to you and your colleagues have done to you and your colleagues have done to move us forward in Computing thank you move us forward in Computing thank you move us forward in Computing thank you thank you so much Scott this has been thank you so much Scott this has been thank you so much Scott this has been another episode of Hansel minutes in another episode of Hansel minutes in another episode of Hansel minutes in association with the ACM bite cast and association with the ACM bite cast and association with the ACM bite cast and we'll see you again next we'll see you again next we'll see you again next [Music] [Music] [Music] week a week a week a [Music]