AI Whistleblower: OpenAI Scandal, AI Cults, Neuralink & Our Last Chance to Stop the Tech Oligarchs

Nate, thank you so much for doing this. Nate, thank you so much for doing this. Um, you've written the world's darkest Um, you've written the world's darkest Um, you've written the world's darkest book. You've devoted your life to book. You've devoted your life to book. You've devoted your life to warning the world about the potential warning the world about the potential warning the world about the potential dangers of AI. So, let's just start by dangers of AI. So, let's just start by dangers of AI. So, let's just start by hearing your explanation of why AI is hearing your explanation of why AI is hearing your explanation of why AI is dangerous in addition to just being dangerous in addition to just being dangerous in addition to just being annoying. annoying. annoying. Yeah, you know, the the the very basic Yeah, you know, the the the very basic Yeah, you know, the the the very basic common sense point is common sense point is common sense point is if if if you race to make machines that are much you race to make machines that are much you race to make machines that are much smarter than any human, to make machines smarter than any human, to make machines smarter than any human, to make machines that are more capable uh at inventing that are more capable uh at inventing that are more capable uh at inventing their own technology than humans are, their own technology than humans are, their own technology than humans are, and you race into this without really and you race into this without really and you race into this without really knowing what you're doing, the most knowing what you're doing, the most knowing what you're doing, the most likely outcome is just that the machines likely outcome is just that the machines likely outcome is just that the machines get loose and do their own thing. get loose and do their own thing. get loose and do their own thing. and that humanity dies as a side effect and that humanity dies as a side effect and that humanity dies as a side effect just like humanity has you know killed just like humanity has you know killed just like humanity has you know killed lots of other animal species not because lots of other animal species not because lots of other animal species not because we hate them but just as a side effect. we hate them but just as a side effect. we hate them but just as a side effect. Uh Uh Uh and in in some sense a lot of people and in in some sense a lot of people and in in some sense a lot of people find it sort of obvious or intuitive find it sort of obvious or intuitive find it sort of obvious or intuitive that if you just like make these really that if you just like make these really that if you just like make these really smart really powerful machines why would smart really powerful machines why would smart really powerful machines why would they care about us?

they care about us? they care about us? Uh Uh Uh and I think that intuition basically is and I think that intuition basically is and I think that intuition basically is right and there's a lot of arguments you right and there's a lot of arguments you right and there's a lot of arguments you can have on each side. you can get into can have on each side. you can get into can have on each side. you can get into the technical details. Uh, but my book the technical details. Uh, but my book the technical details. Uh, but my book is basically just um is basically just um is basically just um getting into those details and saying getting into those details and saying getting into those details and saying like, "Yep, it sort of holds up. If we like, "Yep, it sort of holds up. If we like, "Yep, it sort of holds up. If we make smarter machines without knowing make smarter machines without knowing make smarter machines without knowing what we're doing, it's just going to go what we're doing, it's just going to go what we're doing, it's just going to go poorly. [clears throat] [clears throat] >> Are we actually going to going to make >> Are we actually going to going to make >> Are we actually going to going to make machines capable of what you're machines capable of what you're machines capable of what you're describing?" describing?" describing?" you know, the the companies are trying you know, the the companies are trying you know, the the companies are trying to they um they talk about how they are to they um they talk about how they are to they um they talk about how they are pursuing super intelligence in the true pursuing super intelligence in the true pursuing super intelligence in the true sense of the word. That's Samman's sense of the word. That's Samman's sense of the word. That's Samman's phrase. Uh Dario Modi of Anthropic says phrase. Uh Dario Modi of Anthropic says phrase. Uh Dario Modi of Anthropic says they're trying to make the equivalent of they're trying to make the equivalent of they're trying to make the equivalent of a country worth of geniuses in a data a country worth of geniuses in a data a country worth of geniuses in a data center. Uh so these guys are really, you center. Uh so these guys are really, you center. Uh so these guys are really, you know, the these aren't chatbot know, the these aren't chatbot know, the these aren't chatbot companies. They didn't set out to be companies. They didn't set out to be companies. They didn't set out to be chatbot companies. They set out to make chatbot companies. They set out to make chatbot companies. They set out to make these machines that can sort of exceed these machines that can sort of exceed these machines that can sort of exceed humans in every way. and that's what humans in every way. and that's what humans in every way. and that's what they're targeting. There's a separate they're targeting. There's a separate they're targeting. There's a separate question of whether they will get there.

question of whether they will get there. question of whether they will get there. >> May I ask you a question? Why why would >> May I ask you a question? Why why would >> May I ask you a question? Why why would anyone want to build a machine smarter anyone want to build a machine smarter anyone want to build a machine smarter than people? than people? than people? >> Um I think that >> Um I think that >> Um I think that a lot of them hope that they'll be able a lot of them hope that they'll be able a lot of them hope that they'll be able to make the world much much better. You to make the world much much better. You to make the world much much better. You know, they hope for uh a cure to cancer know, they hope for uh a cure to cancer know, they hope for uh a cure to cancer and then more than a cure to cancer, and then more than a cure to cancer, and then more than a cure to cancer, they hope for a cure to aging. they hope they hope for a cure to aging. they hope they hope for a cure to aging. they hope for uh you know a thousand years worth for uh you know a thousand years worth for uh you know a thousand years worth of technological development compressed of technological development compressed of technological development compressed into two years. Um into two years. Um into two years. Um and I I sort of don't think that and I I sort of don't think that and I I sort of don't think that they're going to be able to get that they're going to be able to get that they're going to be able to get that really or or harness that for good ends, really or or harness that for good ends, really or or harness that for good ends, but that's what I think they're they're but that's what I think they're they're but that's what I think they're they're shooting for. shooting for. shooting for. So they're not. But I mean the core So they're not. But I mean the core So they're not. But I mean the core point you're making is these companies point you're making is these companies point you're making is these companies did not set out to make consumer did not set out to make consumer did not set out to make consumer products like to make your life better products like to make your life better products like to make your life better necessarily in the short term with a necessarily in the short term with a necessarily in the short term with a more efficient search engine. That's more efficient search engine. That's more efficient search engine. That's right. Yeah. You know uh Open AI started right. Yeah. You know uh Open AI started right. Yeah. You know uh Open AI started before the the the chatbots the large before the the the chatbots the large before the the the chatbots the large language models were even a thing. uh language models were even a thing. uh language models were even a thing. uh they were started forget whether it was they were started forget whether it was they were started forget whether it was late 15 or or early 2016 but um the the late 15 or or early 2016 but um the the late 15 or or early 2016 but um the the paper that unlocked the the most recent paper that unlocked the the most recent paper that unlocked the the most recent wave of AI came out in 2017 which was wave of AI came out in 2017 which was wave of AI came out in 2017 which was after open was founded and um after open was founded and um after open was founded and um these guys are the the the large these guys are the the the large these guys are the the the large language models are a language models are a language models are a surprise revenue stream and that revenue surprise revenue stream and that revenue surprise revenue stream and that revenue stream can fund the creation of even

stream can fund the creation of even stream can fund the creation of even more even larger data centers for the more even larger data centers for the more even larger data centers for the next level of the technology. But these next level of the technology. But these next level of the technology. But these guys have their eye on uh the sort of guys have their eye on uh the sort of guys have their eye on uh the sort of ultimate version of the technology which ultimate version of the technology which ultimate version of the technology which is these machines much smarter than is these machines much smarter than is these machines much smarter than humans. And that's sort of the ultimate humans. And that's sort of the ultimate humans. And that's sort of the ultimate version because once the AIs are smarter version because once the AIs are smarter version because once the AIs are smarter than the humans, uh the AIs can carry on than the humans, uh the AIs can carry on than the humans, uh the AIs can carry on the AI research and makes the next the AI research and makes the next the AI research and makes the next generation of the AIs which make the generation of the AIs which make the generation of the AIs which make the next generation of the AIs and um that's next generation of the AIs and um that's next generation of the AIs and um that's sort of what they're shooting for. sort of what they're shooting for. sort of what they're shooting for. >> What is super intelligence? >> What is super intelligence? >> What is super intelligence? We in our book define super intelligence We in our book define super intelligence We in our book define super intelligence as an AI that is uh better than the best as an AI that is uh better than the best as an AI that is uh better than the best humans at every mental task. humans at every mental task. humans at every mental task. So uh anything that a human can do So uh anything that a human can do So uh anything that a human can do purely mentally, the AI can do that purely mentally, the AI can do that purely mentally, the AI can do that better. One one thing people often get better. One one thing people often get better. One one thing people often get caught up on about this is uh that caught up on about this is uh that caught up on about this is uh that includes the AI being better at things includes the AI being better at things includes the AI being better at things like persuading humans, at things like like persuading humans, at things like like persuading humans, at things like charisma. You know, we often think of charisma. You know, we often think of charisma. You know, we often think of intelligence as the stuff that the nerds intelligence as the stuff that the nerds intelligence as the stuff that the nerds have and that the jocks lack, have and that the jocks lack, have and that the jocks lack, >> but >> but >> but >> you win a chess match, >> you win a chess match, >> you win a chess match, >> right? It's the chess guys rather than >> right? It's the chess guys rather than >> right? It's the chess guys rather than the than the politicians guys or the or the than the politicians guys or the or the than the politicians guys or the or the rock stars.

the rock stars. the rock stars. >> But that's not really what the >> But that's not really what the >> But that's not really what the intelligence in artificial intelligence intelligence in artificial intelligence intelligence in artificial intelligence means. The intelligence in artificial means. The intelligence in artificial means. The intelligence in artificial intelligence is about the stuff that intelligence is about the stuff that intelligence is about the stuff that humans have and that mice don't. humans have and that mice don't. humans have and that mice don't. It's it's sort of the whole package. you It's it's sort of the whole package. you It's it's sort of the whole package. you know, like the the politician who's very know, like the the politician who's very know, like the the politician who's very charismatic. charismatic. charismatic. It's not it's not that like chess It's not it's not that like chess It's not it's not that like chess playing happens in your brain and playing happens in your brain and playing happens in your brain and charisma happens in your kidneys, right? charisma happens in your kidneys, right? charisma happens in your kidneys, right? They're sort of both mental functions. They're sort of both mental functions. They're sort of both mental functions. >> Yes. >> Yes. >> Yes. >> Uh and >> Uh and >> Uh and so the the sort of super intelligent AIS so the the sort of super intelligent AIS so the the sort of super intelligent AIS are are not just super good at playing are are not just super good at playing are are not just super good at playing chess, they're also good at super chess, they're also good at super chess, they're also good at super persuasion and they're super good at the persuasion and they're super good at the persuasion and they're super good at the the research and at the technological the research and at the technological the research and at the technological invention. invention. invention. So super intelligence as you define it So super intelligence as you define it So super intelligence as you define it is a machine that is better in in at is a machine that is better in in at is a machine that is better in in at every mental function than any human every mental function than any human every mental function than any human being. being. being. >> That's right. And uh >> That's right. And uh >> That's right. And uh you know the the the definition you know the the the definition you know the the the definition that doesn't mean that that nothing that doesn't mean that that nothing that doesn't mean that that nothing crazy will happen on this planet until crazy will happen on this planet until crazy will happen on this planet until the AIS are super intelligent. Super the AIS are super intelligent. Super the AIS are super intelligent. Super intelligence in this definition is sort intelligence in this definition is sort intelligence in this definition is sort of a point where past this point things of a point where past this point things of a point where past this point things must be pretty crazy because now the AIs must be pretty crazy because now the AIs must be pretty crazy because now the AIs can do automated AI research and they can do automated AI research and they can do automated AI research and they can make smarter AIs and they can figure can make smarter AIs and they can figure can make smarter AIs and they can figure out how to run the robot factories and out how to run the robot factories and out how to run the robot factories and they can build the better robots and and they can build the better robots and and they can build the better robots and and all that. You could have things start to all that. You could have things start to all that. You could have things start to get crazy before you have a super get crazy before you have a super get crazy before you have a super intelligence in this sense. You could intelligence in this sense. You could intelligence in this sense. You could have AIs that are much better in than have AIs that are much better in than have AIs that are much better in than than humans at some tasks and much worse than humans at some tasks and much worse than humans at some tasks and much worse in others that are still causing all in others that are still causing all in others that are still causing all sorts of crazy happenstances.

sorts of crazy happenstances. sorts of crazy happenstances. uh the the super intelligence is sort of uh the the super intelligence is sort of uh the the super intelligence is sort of a like once you get here stuff's crazy a like once you get here stuff's crazy a like once you get here stuff's crazy point. It's not a things will stay point. It's not a things will stay point. It's not a things will stay normal until you get here point. normal until you get here point. normal until you get here point. >> So why is super intelligence the >> So why is super intelligence the >> So why is super intelligence the significant milestone that you're significant milestone that you're significant milestone that you're worried about? worried about? worried about? >> Um I mean I would say that >> Um I mean I would say that >> Um I mean I would say that uh I'm I'm also worried about what'll uh I'm I'm also worried about what'll uh I'm I'm also worried about what'll happen before that milestone. It's more happen before that milestone. It's more happen before that milestone. It's more like like having the definition makes it like like having the definition makes it like like having the definition makes it easy to talk about like how crazy would easy to talk about like how crazy would easy to talk about like how crazy would things get things get things get uh once you are past this point or sort uh once you are past this point or sort uh once you are past this point or sort of like you know let's like if we assume of like you know let's like if we assume of like you know let's like if we assume that the machines get here what happens that the machines get here what happens that the machines get here what happens and then the answer is like it's it and then the answer is like it's it and then the answer is like it's it would be pretty crazy it would be it would be pretty crazy it would be it would be pretty crazy it would be it would be pretty wild and uh it sort of would be pretty wild and uh it sort of would be pretty wild and uh it sort of like lets you factor the conversation like lets you factor the conversation like lets you factor the conversation uh into like what how how to Okay, I uh into like what how how to Okay, I uh into like what how how to Okay, I think in the artificial intelligence think in the artificial intelligence think in the artificial intelligence conversation, there's actually a lot of conversation, there's actually a lot of conversation, there's actually a lot of conversations going on. One conversation conversations going on. One conversation conversations going on. One conversation is like, can the machines really get is like, can the machines really get is like, can the machines really get smarter than the humans? One smarter than the humans? One smarter than the humans? One conversation is like, how fast could we conversation is like, how fast could we conversation is like, how fast could we get there? Is the current technology on get there? Is the current technology on get there? Is the current technology on that route? Another conversation is what that route? Another conversation is what that route? Another conversation is what happens if we do get there? Like, you happens if we do get there? Like, you happens if we do get there? Like, you know, would would the AI care about us?

know, would would the AI care about us? know, would would the AI care about us? Would they not care about us? What would Would they not care about us? What would Would they not care about us? What would they be able to like what would they they be able to like what would they they be able to like what would they what would they try to do? There's what would they try to do? There's what would they try to do? There's another question which is what would another question which is what would another question which is what would they be able to do, right? Right? And they be able to do, right? Right? And they be able to do, right? Right? And these are all sort of different these are all sort of different these are all sort of different conversations about AI. And the super conversations about AI. And the super conversations about AI. And the super intelligence definition, intelligence definition, intelligence definition, it's not really like here's where all it's not really like here's where all it's not really like here's where all the fears hang. It's more like it lets the fears hang. It's more like it lets the fears hang. It's more like it lets us us us break those pieces out separately and be break those pieces out separately and be break those pieces out separately and be like, well, is super intelligence like, well, is super intelligence like, well, is super intelligence possible? Separate from, well, what possible? Separate from, well, what possible? Separate from, well, what would happen if you had one? would happen if you had one? would happen if you had one? >> What would happen if you had one? >> What would happen if you had one? >> What would happen if you had one? >> Most likely outcome, uh, I think >> Most likely outcome, uh, I think >> Most likely outcome, uh, I think destruction of the planet. So, our destruction of the planet. So, our destruction of the planet. So, our friends at Pre-born, who are actual friends at Pre-born, who are actual friends at Pre-born, who are actual friends, by the way, just asked us to friends, by the way, just asked us to friends, by the way, just asked us to thank the listeners of this show for thank the listeners of this show for thank the listeners of this show for supporting the pro-life cause. [music] supporting the pro-life cause. [music] supporting the pro-life cause. [music] A lot of people do all of a sudden, A lot of people do all of a sudden, A lot of people do all of a sudden, trust me, because what's happening is trust me, because what's happening is trust me, because what's happening is crazy and people are waking up to that. crazy and people are waking up to that. crazy and people are waking up to that. The president of the group is our friend The president of the group is our friend The president of the group is our friend Dan Steiner, and he wants Americans to Dan Steiner, and he wants Americans to Dan Steiner, and he wants Americans to know how much of a difference they have know how much of a difference they have know how much of a difference they have made. This year's donations have helped made. This year's donations have helped made. This year's donations have helped saved 50,000 babies from abortion. saved 50,000 babies from abortion. saved 50,000 babies from abortion. That's enough to fill a professional That's enough to fill a professional That's enough to fill a professional baseball stadium. 50,000 lives saved as baseball stadium. 50,000 lives saved as baseball stadium. 50,000 lives saved as well as generations that will follow well as generations that will follow well as generations that will follow them. Pre-born wastes zero money. We're them. Pre-born wastes zero money. We're them. Pre-born wastes zero money. We're not big on charities because a lot of not big on charities because a lot of not big on charities because a lot of them waste money. Pre-born does not. It them waste money. Pre-born does not. It them waste money. Pre-born does not. It uses every taxdeductible gift to rescue uses every taxdeductible gift to rescue uses every taxdeductible gift to rescue babies and share the hope of Jesus with babies and share the hope of Jesus with babies and share the hope of Jesus with their moms. In 2026 alone, over [music] their moms. In 2026 alone, over [music] their moms. In 2026 alone, over [music] 8,000 women made the decision to follow 8,000 women made the decision to follow 8,000 women made the decision to follow Christ. The generous support of Christ. The generous support of Christ. The generous support of Preborn's donors made that all possible.

Preborn's donors made that all possible. Preborn's donors made that all possible. $28 to provide one life-saving $28 to provide one life-saving $28 to provide one life-saving ultrasound. is giving expectant mothers ultrasound. is giving expectant mothers ultrasound. is giving expectant mothers the knowledge that they have a baby that the knowledge that they have a baby that the knowledge that they have a baby that they're carrying. $15,000 can place an they're carrying. $15,000 can place an they're carrying. $15,000 can place an ultrasound machine in a pregnancy clinic ultrasound machine in a pregnancy clinic ultrasound machine in a pregnancy clinic where it can save countless lives. Dan where it can save countless lives. Dan where it can save countless lives. Dan Steiner's message is really simple. Steiner's message is really simple. Steiner's message is really simple. Thank you for helping. Your generosity Thank you for helping. Your generosity Thank you for helping. Your generosity is literally saving lives. To help is literally saving lives. To help is literally saving lives. To help pre-born save more, dial pound 250 and pre-born save more, dial pound 250 and pre-born save more, dial pound 250 and say the keyword baby. That's pound250bab say the keyword baby. That's pound250bab say the keyword baby. That's pound250bab or visit pre-born.com/tucker. or visit pre-born.com/tucker. or visit pre-born.com/tucker. pre-born.com/tucker. How what would that look like? How what would that look like? So, I'm going to be a little bit So, I'm going to be a little bit So, I'm going to be a little bit annoying here and say uh a couple annoying here and say uh a couple annoying here and say uh a couple caveats first because it's it's sort of caveats first because it's it's sort of caveats first because it's it's sort of a tricky one to to sort of predict a tricky one to to sort of predict a tricky one to to sort of predict things that are smarter than us. things that are smarter than us. things that are smarter than us. >> Yes. >> Yes. >> Yes. >> And the the first annoying caveat I will >> And the the first annoying caveat I will >> And the the first annoying caveat I will give is uh give is uh give is uh if you go to play a game of chess if you go to play a game of chess if you go to play a game of chess against Magnus Carlson, against Magnus Carlson, against Magnus Carlson, I predict you will lose. No offense.

I predict you will lose. No offense. I predict you will lose. No offense. uh he's just the best human chess player uh he's just the best human chess player uh he's just the best human chess player alive. If you then ask what piece will he use If you then ask what piece will he use to checkmate me? I'm like well that's a to checkmate me? I'm like well that's a to checkmate me? I'm like well that's a much harder question. much harder question. much harder question. >> Yes. >> Yes. >> Yes. >> Right. It sort of is like very easy to >> Right. It sort of is like very easy to >> Right. It sort of is like very easy to predict the winner. It's hard to predict predict the winner. It's hard to predict predict the winner. It's hard to predict the exact methods. the exact methods. the exact methods. So uh my my prediction that humanity So uh my my prediction that humanity So uh my my prediction that humanity ultimately dies ultimately dies ultimately dies is a different sort of prediction than is a different sort of prediction than is a different sort of prediction than my prediction about like it could go my prediction about like it could go my prediction about like it could go this way, it could go that way. this way, it could go that way. this way, it could go that way. Um, so I'll give you some stories, but Um, so I'll give you some stories, but Um, so I'll give you some stories, but these are stories that are like, well, these are stories that are like, well, these are stories that are like, well, maybe Magnus will like fork your queen maybe Magnus will like fork your queen maybe Magnus will like fork your queen and your rook with his knight and then and your rook with his knight and then and your rook with his knight and then finally use the queen to checkmate you. finally use the queen to checkmate you. finally use the queen to checkmate you. Like, yeah, that could happen, but I'm Like, yeah, that could happen, but I'm Like, yeah, that could happen, but I'm it's it's a different sort of it's it's a different sort of it's it's a different sort of prediction. It's a guess, right? prediction. It's a guess, right? prediction. It's a guess, right? The the guess here is The the guess here is The the guess here is uh that if we manage to make these uh that if we manage to make these uh that if we manage to make these really smart AIs, they will have their really smart AIs, they will have their really smart AIs, they will have their own own own stuff that they're pursuing, which is stuff that they're pursuing, which is stuff that they're pursuing, which is not quite what we wanted, not quite what not quite what we wanted, not quite what not quite what we wanted, not quite what we asked for. It'll have this other we asked for. It'll have this other we asked for. It'll have this other strange stuff. Um there's a whole strange stuff. Um there's a whole strange stuff. Um there's a whole discussion about why that happens, but discussion about why that happens, but discussion about why that happens, but we're already starting to see it in we're already starting to see it in we're already starting to see it in practice with some of the recent events.

practice with some of the recent events. practice with some of the recent events. um [clears throat] um [clears throat] um [clears throat] and uh they would be able to pursue whatever uh they would be able to pursue whatever it is they're pursuing it is they're pursuing it is they're pursuing much more efficiently than humanity can. much more efficiently than humanity can. much more efficiently than humanity can. And so we're talking about, you know, And so we're talking about, you know, And so we're talking about, you know, automated factories with uh that produce automated factories with uh that produce automated factories with uh that produce the robots that produce the factories in the robots that produce the factories in the robots that produce the factories in a fully automated supply chain. And then a fully automated supply chain. And then a fully automated supply chain. And then if the AIs have anything they're trying if the AIs have anything they're trying if the AIs have anything they're trying to do that they can do more of with more to do that they can do more of with more to do that they can do more of with more resources, resources, resources, they start gobbling up the resources on they start gobbling up the resources on they start gobbling up the resources on the planet. Sort of like how humanity the planet. Sort of like how humanity the planet. Sort of like how humanity spread and started gobbling up all the spread and started gobbling up all the spread and started gobbling up all the resources on the planet. Uh and the resources on the planet. Uh and the resources on the planet. Uh and the uh you know the the sort of most basic uh you know the the sort of most basic uh you know the the sort of most basic thing to visualize here might be you thing to visualize here might be you thing to visualize here might be you have factories that produce robots that have factories that produce robots that have factories that produce robots that produce factories that produce robots produce factories that produce robots produce factories that produce robots that also build data centers and you that also build data centers and you that also build data centers and you just have these you know fully just have these you know fully just have these you know fully autonomous self-replicating autonomous self-replicating autonomous self-replicating ecosystems of of robots and factories ecosystems of of robots and factories ecosystems of of robots and factories and data centers that don't care about and data centers that don't care about and data centers that don't care about the instructions human humans gave them.

the instructions human humans gave them. the instructions human humans gave them. uh and that uh and that uh and that cover the planet, take all the cover the planet, take all the cover the planet, take all the resources, take all the sunlight, take resources, take all the sunlight, take resources, take all the sunlight, take all the places we were growing crops, all the places we were growing crops, all the places we were growing crops, probably raise the temperature of the probably raise the temperature of the probably raise the temperature of the planet because uh you can compute more planet because uh you can compute more planet because uh you can compute more efficiently. Well, technically the the efficiently. Well, technically the the efficiently. Well, technically the the earth radiates more heat when the world earth radiates more heat when the world earth radiates more heat when the world is hotter and that's lets you sort of do is hotter and that's lets you sort of do is hotter and that's lets you sort of do more computation and then sort of you more computation and then sort of you more computation and then sort of you know know know >> it's good for the machines to have a hot >> it's good for the machines to have a hot >> it's good for the machines to have a hot planet. planet. planet. >> It's good for the machines to have a hot >> It's good for the machines to have a hot >> It's good for the machines to have a hot planet. Yeah. um that the sort of planet. Yeah. um that the sort of planet. Yeah. um that the sort of physical limits on how much computing uh physical limits on how much computing uh physical limits on how much computing uh you can run on the surface of Earth is you can run on the surface of Earth is you can run on the surface of Earth is bounded by how much heat you can radiate bounded by how much heat you can radiate bounded by how much heat you can radiate to space. That's sort of the first to space. That's sort of the first to space. That's sort of the first constraint you bump up against. You constraint you bump up against. You constraint you bump up against. You might think that it's energy, but might think that it's energy, but might think that it's energy, but actually there's a lot of helium actually there's a lot of helium actually there's a lot of helium hydrogen to fuse on this planet. So you hydrogen to fuse on this planet. So you hydrogen to fuse on this planet. So you can get plenty of energy. What you need can get plenty of energy. What you need can get plenty of energy. What you need is uh heat dissipation and the the world is uh heat dissipation and the the world is uh heat dissipation and the the world can dissipate more heat um when it's can dissipate more heat um when it's can dissipate more heat um when it's hot. So if you're, you know, imagining hot. So if you're, you know, imagining hot. So if you're, you know, imagining some collective of AIs that are trying some collective of AIs that are trying some collective of AIs that are trying to run a lot of computing power, uh, to run a lot of computing power, uh, to run a lot of computing power, uh, trying to do a lot of computation for trying to do a lot of computation for trying to do a lot of computation for one reason or another, they sort of one reason or another, they sort of one reason or another, they sort of prefer the planet running hotter.

prefer the planet running hotter. prefer the planet running hotter. And so it sort of is like nothing And so it sort of is like nothing And so it sort of is like nothing personal. It's just if you let these AIs personal. It's just if you let these AIs personal. It's just if you let these AIs get out of control, uh, they get out of control, uh, they get out of control, uh, they transform the planet into something transform the planet into something transform the planet into something unlivable. unlivable. unlivable. >> How hot? >> How hot? >> How hot? uh as hot as uh as hot as uh as hot as uh you can while still having the uh you can while still having the uh you can while still having the computers not melt. So probably hundreds computers not melt. So probably hundreds computers not melt. So probably hundreds of degrees. of degrees. of degrees. >> Um at which point you hear people say, >> Um at which point you hear people say, >> Um at which point you hear people say, "Well, then just turn it off." "Well, then just turn it off." "Well, then just turn it off." >> Uh so I think we do have an opportunity >> Uh so I think we do have an opportunity >> Uh so I think we do have an opportunity to turn it off, you know, and I'm not to turn it off, you know, and I'm not to turn it off, you know, and I'm not here saying that we're going to die. I'm here saying that we're going to die. I'm here saying that we're going to die. I'm here saying we sort of are going to need here saying we sort of are going to need here saying we sort of are going to need to act. You got to be careful about the to act. You got to be careful about the to act. You got to be careful about the turning it off piece because um turning it off piece because um turning it off piece because um your opportunities to turn the AI off your opportunities to turn the AI off your opportunities to turn the AI off only last when the AI is running on the only last when the AI is running on the only last when the AI is running on the computers you know it's running on. If computers you know it's running on. If computers you know it's running on. If you have the AI breaking out and running you have the AI breaking out and running you have the AI breaking out and running on hidden computers. on hidden computers. on hidden computers. If you have the AI running robot If you have the AI running robot If you have the AI running robot factories that produce robots that are factories that produce robots that are factories that produce robots that are under the control of the AI where those under the control of the AI where those under the control of the AI where those robots can then go build more computers robots can then go build more computers robots can then go build more computers that are not hooked up to your network that are not hooked up to your network that are not hooked up to your network that can run the AIS.

that can run the AIS. that can run the AIS. These are sort of thresholds where the These are sort of thresholds where the These are sort of thresholds where the AI is able to keep itself running AI is able to keep itself running AI is able to keep itself running and you better have turned it off before and you better have turned it off before and you better have turned it off before that point because after that point it's because after that point it's impossible. impossible. impossible. >> Yeah. I mean the if if if if it it it also gets a lot harder if the it it it also gets a lot harder if the it it it also gets a lot harder if the AI knows you're going to try to shut it AI knows you're going to try to shut it AI knows you're going to try to shut it off and is going to try to stop this, off and is going to try to stop this, off and is going to try to stop this, right? You got to you got to remember right? You got to you got to remember right? You got to you got to remember that we're talking about things that are that we're talking about things that are that we're talking about things that are smart here. And so if the AI sort of smart here. And so if the AI sort of smart here. And so if the AI sort of sees it coming, maybe it defects to sees it coming, maybe it defects to sees it coming, maybe it defects to North Korea where it can, you know, North Korea where it can, you know, North Korea where it can, you know, convince them to run it on its data convince them to run it on its data convince them to run it on its data centers in ways that initially benefit centers in ways that initially benefit centers in ways that initially benefit them, but that ultimately benefit the them, but that ultimately benefit the them, but that ultimately benefit the AI. I AI. I AI. I >> I think we're going to have to redefine >> I think we're going to have to redefine >> I think we're going to have to redefine what life is because you're describing a what life is because you're describing a what life is because you're describing a living autonomous thing. living autonomous thing. living autonomous thing. >> Yeah. Artificial life in a sense. I >> Yeah. Artificial life in a sense. I >> Yeah. Artificial life in a sense. I mean, in some sense, we're already mean, in some sense, we're already mean, in some sense, we're already seeing the very very beginnings of that. seeing the very very beginnings of that. seeing the very very beginnings of that. Um Um Um but yeah, we once once the AIs can but yeah, we once once the AIs can but yeah, we once once the AIs can replicate, once they can improve replicate, once they can improve replicate, once they can improve themselves, themselves, themselves, uh once they can uh once they can uh once they can uh find ways to run where you don't know uh find ways to run where you don't know uh find ways to run where you don't know that they're running, that they're running, that they're running, once they can once they can once they can uh uh uh sort of run the robot factories to sort of run the robot factories to sort of run the robot factories to produce the robots that can produce more produce the robots that can produce more produce the robots that can produce more factories and that can produce computers factories and that can produce computers factories and that can produce computers that the AI can run on, then yeah, you that the AI can run on, then yeah, you that the AI can run on, then yeah, you have you have in some sense has made a have you have in some sense has made a have you have in some sense has made a new artificial life form.

new artificial life form. new artificial life form. Uh, Uh, Uh, and you know, humanity is on the top of and you know, humanity is on the top of and you know, humanity is on the top of the food chain right now because we the food chain right now because we the food chain right now because we are sort of the only smart life form are sort of the only smart life form are sort of the only smart life form around. around. around. >> Yeah. >> Yeah. >> Yeah. >> If you suddenly make a new one that is >> If you suddenly make a new one that is >> If you suddenly make a new one that is smarter, that is able to make a million smarter, that is able to make a million smarter, that is able to make a million copies of itself, that is able to run copies of itself, that is able to run copies of itself, that is able to run faster, faster, faster, it's just kind of a crazy thing to race it's just kind of a crazy thing to race it's just kind of a crazy thing to race into. into. into. It's a weird thing to want. It's a weird thing to want. It's a weird thing to want. And as it's developed, I was going to And as it's developed, I was going to And as it's developed, I was going to say slowly, but it hasn't been say slowly, but it hasn't been say slowly, but it hasn't been particularly slow, 10 years. particularly slow, 10 years. particularly slow, 10 years. The rest of us have watched as The rest of us have watched as The rest of us have watched as people who people who people who I mean, put it in political terms, don't I mean, put it in political terms, don't I mean, put it in political terms, don't share the values of most Americans are share the values of most Americans are share the values of most Americans are now in charge of this. And it's almost now in charge of this. And it's almost now in charge of this. And it's almost like everyone sat passively by as it like everyone sat passively by as it like everyone sat passively by as it happened. happened. happened. >> Yeah. I I think I think there's a lot >> Yeah. I I think I think there's a lot >> Yeah. I I think I think there's a lot going on there. I think part of it is going on there. I think part of it is going on there. I think part of it is that most people didn't and some still that most people didn't and some still that most people didn't and some still don't believe that AI was going to be don't believe that AI was going to be don't believe that AI was going to be able to keep going. I think there's a able to keep going. I think there's a able to keep going. I think there's a lot of like putting the head in the sand lot of like putting the head in the sand lot of like putting the head in the sand and being like, "Oh, it's, you know, and being like, "Oh, it's, you know, and being like, "Oh, it's, you know, just slop. It's going to be a bubble.

just slop. It's going to be a bubble. just slop. It's going to be a bubble. It's going to pop. It's going to hit a It's going to pop. It's going to hit a It's going to pop. It's going to hit a wall. Uh it's not going to be able to wall. Uh it's not going to be able to wall. Uh it's not going to be able to improve anymore." improve anymore." improve anymore." And I think a lot of that was sort of And I think a lot of that was sort of And I think a lot of that was sort of wishful thinking. And I think that at wishful thinking. And I think that at wishful thinking. And I think that at the very least the very least the very least if people are like we are making, you if people are like we are making, you if people are like we are making, you know, the the super intelligent machines know, the the super intelligent machines know, the the super intelligent machines that are going to replace humanity as that are going to replace humanity as that are going to replace humanity as the as the top of the food chain because the as the top of the food chain because the as the top of the food chain because we think it's going to go go great. I we think it's going to go go great. I we think it's going to go go great. I think humanity's response should not be think humanity's response should not be think humanity's response should not be go ahead and try. We hope you'll fail. I go ahead and try. We hope you'll fail. I go ahead and try. We hope you'll fail. I think the response should be like, "Hold think the response should be like, "Hold think the response should be like, "Hold on, that's kind of crazy." on, that's kind of crazy." on, that's kind of crazy." Um, yeah, I had I had some other point too, yeah, I had I had some other point too, but I've forgotten it. So but I've forgotten it. So but I've forgotten it. So >> how uh predictable has the evolution of >> how uh predictable has the evolution of >> how uh predictable has the evolution of AI been? AI been? AI been? >> Has it taken turns that you didn't >> Has it taken turns that you didn't >> Has it taken turns that you didn't expect? expect? expect? >> Yeah, totally. Uh I was so back before >> Yeah, totally. Uh I was so back before >> Yeah, totally. Uh I was so back before the large language models. the large language models. the large language models. >> How long have you been following this >> How long have you been following this >> How long have you been following this issue? issue? issue? >> Uh I started following it in 2012 and I >> Uh I started following it in 2012 and I >> Uh I started following it in 2012 and I started working on uh I I started started working on uh I I started started working on uh I I started working with some of the people trying working with some of the people trying working with some of the people trying to make this go well in 2013 and I to make this go well in 2013 and I to make this go well in 2013 and I started full-time in 2014.

started full-time in 2014. started full-time in 2014. >> A long time. So over a decade. >> A long time. So over a decade. >> A long time. So over a decade. >> What has surprised you? >> What has surprised you? >> What has surprised you? >> You know the the large language models >> You know the the large language models >> You know the the large language models um um um they have gone further than I expected they have gone further than I expected they have gone further than I expected initially and I think one of the big initially and I think one of the big initially and I think one of the big surprises here was AI undergoing a phase surprises here was AI undergoing a phase surprises here was AI undergoing a phase where everybody can see it. where everybody can see it. where everybody can see it. you know, back before the large language you know, back before the large language you know, back before the large language models, models, models, uh, really only nerds paid attention to uh, really only nerds paid attention to uh, really only nerds paid attention to AI. And it wasn't that there was nothing AI. And it wasn't that there was nothing AI. And it wasn't that there was nothing happening. You know, we were watching, happening. You know, we were watching, happening. You know, we were watching, uh, the, you know, Google Deep Mind make uh, the, you know, Google Deep Mind make uh, the, you know, Google Deep Mind make an AI that could beat uh, the the best an AI that could beat uh, the the best an AI that could beat uh, the the best Go player. Go player. Go player. >> Yes. >> Yes. >> Yes. >> And that was sort of a milestone for >> And that was sort of a milestone for >> And that was sort of a milestone for people who were paying attention. Um, people who were paying attention. Um, people who were paying attention. Um, but for all we knew, but for all we knew, but for all we knew, the labs were going to keep on working the labs were going to keep on working the labs were going to keep on working on engineering problems and keep on on engineering problems and keep on on engineering problems and keep on working on the relatively nerdier working on the relatively nerdier working on the relatively nerdier problems and you were never going to problems and you were never going to problems and you were never going to have like a mass market consumable have like a mass market consumable have like a mass market consumable product. And so like for all we knew, it product. And so like for all we knew, it product. And so like for all we knew, it would just stay in the labs and no one would just stay in the labs and no one would just stay in the labs and no one would ever really notice that these guys would ever really notice that these guys would ever really notice that these guys were gambling with with the whole were gambling with with the whole were gambling with with the whole future. Now at least, uh, AI is future. Now at least, uh, AI is future. Now at least, uh, AI is everywhere. everyone's starting to have everywhere. everyone's starting to have everywhere. everyone's starting to have the conversation and that is in some the conversation and that is in some the conversation and that is in some sense actually really quite optimistic sense actually really quite optimistic sense actually really quite optimistic because it it gives the world an because it it gives the world an because it it gives the world an opportunity to see what these guys are opportunity to see what these guys are opportunity to see what these guys are trying to do and say hold on I guess you could flip it around uh and I guess you could flip it around uh and conclude that because the overwhelming conclude that because the overwhelming conclude that because the overwhelming majority of people seem to be very majority of people seem to be very majority of people seem to be very opposed to way and I don't even know opposed to way and I don't even know opposed to way and I don't even know anyone who's for it every college

anyone who's for it every college anyone who's for it every college commencement speaker who mentions it commencement speaker who mentions it commencement speaker who mentions it gets booed. gets booed. gets booed. >> Yeah. And that view, that opposition to >> Yeah. And that view, that opposition to >> Yeah. And that view, that opposition to it has had no effect at all in slowing it has had no effect at all in slowing it has had no effect at all in slowing it down. it down. it down. I don't know. Does that give you hope? I don't know. Does that give you hope? I don't know. Does that give you hope? >> Um, >> Um, >> Um, I mean, I mean, I mean, I I I hear this a lot. I would say when I I I hear this a lot. I would say when I I I hear this a lot. I would say when you're coming at this from 2012, you're coming at this from 2012, you're coming at this from 2012, >> Yeah. >> Yeah. >> Yeah. >> it really feels like we're making >> it really feels like we're making >> it really feels like we're making progress, progress, progress, >> right? No, that's fair. >> right? No, that's fair. >> right? No, that's fair. >> It It doesn't feel like we're all the >> It It doesn't feel like we're all the >> It It doesn't feel like we're all the way there yet. way there yet. way there yet. Um, Um, Um, >> but can I just make the point China, >> but can I just make the point China, >> but can I just make the point China, China, China, China, China. We have to. China, China, China, China. We have to. China, China, China, China. We have to. It's because of China. It's because of China. It's because of China. >> Yeah, you know, I think uh China and the >> Yeah, you know, I think uh China and the >> Yeah, you know, I think uh China and the USA have a shared interest in not dying USA have a shared interest in not dying USA have a shared interest in not dying to a rogue super intelligence. to a rogue super intelligence. to a rogue super intelligence. >> Storm season is on the way and that >> Storm season is on the way and that >> Storm season is on the way and that means your Wi-Fi could go down and if it means your Wi-Fi could go down and if it means your Wi-Fi could go down and if it does, your security cameras go dark, does, your security cameras go dark, does, your security cameras go dark, too. So, at the exact moment you want to too. So, at the exact moment you want to too. So, at the exact moment you want to survey your property during the survey your property during the survey your property during the blackout, you can't. Unless you install blackout, you can't. Unless you install blackout, you can't. Unless you install cameras by Defend. Defend cameras run on cameras by Defend. Defend cameras run on cameras by Defend. Defend cameras run on cellular, not Wi-Fi. [music] And that cellular, not Wi-Fi. [music] And that cellular, not Wi-Fi. [music] And that means they keep working even when you means they keep working even when you means they keep working even when you lose power. You don't choose a carrier, lose power. You don't choose a carrier, lose power. You don't choose a carrier, you don't sign a contract, you just you don't sign a contract, you just you don't sign a contract, you just mount it, open the Defend app, and bam, mount it, open the Defend app, and bam, mount it, open the Defend app, and bam, you're connected. It's that simple. So you're connected. It's that simple. So you're connected. It's that simple. So whether it's your ranch, a job site, whether it's your ranch, a job site, whether it's your ranch, a job site, your backyard, Defend works where Wi-Fi your backyard, Defend works where Wi-Fi your backyard, Defend works where Wi-Fi doesn't and keeps working when the grid doesn't and keeps working when the grid doesn't and keeps working when the grid goes down, which it may. Plans start at goes down, which it may. Plans start at goes down, which it may. Plans start at just $5 a month. Most people pay around just $5 a month. Most people pay around just $5 a month. Most people pay around 10. Plus, there are no binding 10. Plus, there are no binding 10. Plus, there are no binding contracts, meaning you get to cancel contracts, meaning you get to cancel contracts, meaning you get to cancel whenever you want. So unlike big tech, whenever you want. So unlike big tech, whenever you want. So unlike big tech, Defend will never sell your data. Not to

Defend will never sell your data. Not to Defend will never sell your data. Not to China, not to the government, not to China, not to the government, not to China, not to the government, not to anybody. It's an American company built anybody. It's an American company built anybody. It's an American company built for people who want actual security. for people who want actual security. for people who want actual security. Visit defendellcam.com Visit defendellcam.com Visit defendellcam.com and use the code Tucker for 20% off. and use the code Tucker for 20% off. and use the code Tucker for 20% off. That's defendellcam.com That's defendellcam.com That's defendellcam.com code Tucker. code Tucker. code Tucker. Is it conceivable that like let's just Is it conceivable that like let's just Is it conceivable that like let's just say China can I'm I'm sure by the way say China can I'm I'm sure by the way say China can I'm I'm sure by the way that China is showing more restraint that China is showing more restraint that China is showing more restraint than we are in this. than we are in this. than we are in this. >> Yeah. I mean and if nothing else they >> Yeah. I mean and if nothing else they >> Yeah. I mean and if nothing else they sort of uh have a lot more reason to sort of uh have a lot more reason to sort of uh have a lot more reason to censor their AIS, censor their AIS, censor their AIS, >> right? >> right? >> right? >> And to try to make them, you know, not >> And to try to make them, you know, not >> And to try to make them, you know, not say certain things to the to the say certain things to the to the say certain things to the to the population. So um but if if the United population. So um but if if the United population. So um but if if the United States you know if the US government States you know if the US government States you know if the US government were somehow able to get control of the were somehow able to get control of the were somehow able to get control of the tech sector which is at present not tech sector which is at present not tech sector which is at present not possible but let's say it did possible but let's say it did possible but let's say it did let's say the president was more let's say the president was more let's say the president was more powerful than the tech oligarchs which powerful than the tech oligarchs which powerful than the tech oligarchs which he is not but for the sake of argument he is not but for the sake of argument he is not but for the sake of argument let's say he was and he shut it down let's say he was and he shut it down let's say he was and he shut it down would that matter if if some Indian lab would that matter if if some Indian lab would that matter if if some Indian lab or Chinese lab created it so it does or Chinese lab created it so it does or Chinese lab created it so it does have to be a global stop to this have to be a global stop to this have to be a global stop to this creation of super intelligence. Uh I I creation of super intelligence. Uh I I creation of super intelligence. Uh I I think there's a couple reasons why this think there's a couple reasons why this think there's a couple reasons why this is possible. One is I think a lot of is possible. One is I think a lot of is possible. One is I think a lot of people um people um people um they they hear that some parts of AI they they hear that some parts of AI they they hear that some parts of AI need to be stopped and they think I'm need to be stopped and they think I'm need to be stopped and they think I'm saying that all parts of AI need to be saying that all parts of AI need to be saying that all parts of AI need to be stopped and uh you know there's there's stopped and uh you know there's there's stopped and uh you know there's there's all sorts of interesting issues with you all sorts of interesting issues with you all sorts of interesting issues with you know AI and education and AI and you know AI and education and AI and you know AI and education and AI and you know military drones that that are uh

know military drones that that are uh know military drones that that are uh that society has to wrestle with but that society has to wrestle with but that society has to wrestle with but these are all sort of separate issues these are all sort of separate issues these are all sort of separate issues from the super intelligence race And if from the super intelligence race And if from the super intelligence race And if you sort of like tease those issues you sort of like tease those issues you sort of like tease those issues apart, uh it becomes much more possible apart, uh it becomes much more possible apart, uh it becomes much more possible to do like a a more surgical to do like a a more surgical to do like a a more surgical intervention on we're not going to race intervention on we're not going to race intervention on we're not going to race towards the super intelligence in ways towards the super intelligence in ways towards the super intelligence in ways that leave a lot of the rest of the that leave a lot of the rest of the that leave a lot of the rest of the sector alive, sector alive, sector alive, which um which um which um which which makes it sort of an easier which which makes it sort of an easier which which makes it sort of an easier coordination effort to to attempt. And coordination effort to to attempt. And coordination effort to to attempt. And on that front, on the super intelligence on that front, on the super intelligence on that front, on the super intelligence front, front, front, it's not that, you know, just shutting it's not that, you know, just shutting it's not that, you know, just shutting down the the domestic race towards super down the the domestic race towards super down the the domestic race towards super intelligence would be enough. But, uh, intelligence would be enough. But, uh, intelligence would be enough. But, uh, racing towards super intelligence racing towards super intelligence racing towards super intelligence requires a huge amount of highly requires a huge amount of highly requires a huge amount of highly advanced computer chips that can only be advanced computer chips that can only be advanced computer chips that can only be produced as sort of the peak of the produced as sort of the peak of the produced as sort of the peak of the global supply chain. global supply chain. global supply chain. >> Yes. >> Yes. >> Yes. >> Uh, which is largely controlled by US >> Uh, which is largely controlled by US >> Uh, which is largely controlled by US allies.

allies. allies. And so there is absolutely a possibility And so there is absolutely a possibility And so there is absolutely a possibility that uh a US-led coordination effort that uh a US-led coordination effort that uh a US-led coordination effort could say look we're not doing uh the could say look we're not doing uh the could say look we're not doing uh the super intelligence thing. We're going to super intelligence thing. We're going to super intelligence thing. We're going to monitor the extremely heavy chip monitor the extremely heavy chip monitor the extremely heavy chip concentrations of like you know 10 or concentrations of like you know 10 or concentrations of like you know 10 or 100 thousand of these highly advanced AI 100 thousand of these highly advanced AI 100 thousand of these highly advanced AI chips. uh we are going to chips. uh we are going to chips. uh we are going to uh make sure that they are not doing uh make sure that they are not doing uh make sure that they are not doing these super intelligence training runs these super intelligence training runs these super intelligence training runs and and and I I think there's a way if the US was I I think there's a way if the US was I I think there's a way if the US was leading that effort there would be a way leading that effort there would be a way leading that effort there would be a way to get China on board and set this up in to get China on board and set this up in to get China on board and set this up in a way that was moniable enforceable a way that was moniable enforceable a way that was moniable enforceable verifiable verifiable verifiable >> yes >> yes >> yes >> uh we we sort of in many ways would be >> uh we we sort of in many ways would be >> uh we we sort of in many ways would be easier than a nuclear arms treaty easier than a nuclear arms treaty easier than a nuclear arms treaty because uranium is a rock you can dig because uranium is a rock you can dig because uranium is a rock you can dig out of the ground whereas is these out of the ground whereas is these out of the ground whereas is these highly advanced computer chips sort of highly advanced computer chips sort of highly advanced computer chips sort of only come out of one fab in Taiwan, you only come out of one fab in Taiwan, you only come out of one fab in Taiwan, you know, and so it's like know, and so it's like know, and so it's like >> those are TSMC chips. >> those are TSMC chips. >> those are TSMC chips. >> Yeah. Uh and so and there's other parts >> Yeah. Uh and so and there's other parts >> Yeah. Uh and so and there's other parts of the supply chain that are very narrow of the supply chain that are very narrow of the supply chain that are very narrow like the lithography machines that come like the lithography machines that come like the lithography machines that come out of the Netherlands. And so um yeah, out of the Netherlands. And so um yeah, out of the Netherlands. And so um yeah, we we could absolutely lead a uh a we we could absolutely lead a uh a we we could absolutely lead a uh a global effort to say we're not making global effort to say we're not making global effort to say we're not making the super intelligent machines and it the super intelligent machines and it the super intelligent machines and it would be a bit tricky, but it's just we would be a bit tricky, but it's just we would be a bit tricky, but it's just we could just do it if we had the political could just do it if we had the political could just do it if we had the political will.

will. will. I I think that I I think that I I think that people either don't know what's people either don't know what's people either don't know what's happening or they assume that like a lot happening or they assume that like a lot happening or they assume that like a lot of fears this fear will turn out to be of fears this fear will turn out to be of fears this fear will turn out to be groundless. groundless. groundless. People point to Y2K. People point to Y2K. People point to Y2K. >> Yeah. Um you know I hope the fears are >> Yeah. Um you know I hope the fears are >> Yeah. Um you know I hope the fears are groundless. I think with a lot of past groundless. I think with a lot of past groundless. I think with a lot of past fears uh the fears uh the fears uh the what happened is not that the fear was what happened is not that the fear was what happened is not that the fear was groundless. it's that groundless. it's that groundless. it's that uh people noticed the issue and put in a uh people noticed the issue and put in a uh people noticed the issue and put in a ton of work to make the bad thing uh not ton of work to make the bad thing uh not ton of work to make the bad thing uh not happen. You know, I think we saw this happen. You know, I think we saw this happen. You know, I think we saw this with the hole in the ozone layer where with the hole in the ozone layer where with the hole in the ozone layer where people are like, "Oh, whatever happened people are like, "Oh, whatever happened people are like, "Oh, whatever happened to the hole in the ozone layer?" Well, to the hole in the ozone layer?" Well, to the hole in the ozone layer?" Well, what happened with the hole in the ozone what happened with the hole in the ozone what happened with the hole in the ozone layer is that we fixed it. It's not that layer is that we fixed it. It's not that layer is that we fixed it. It's not that it was fake. It's that, you know, we it was fake. It's that, you know, we it was fake. It's that, you know, we went and banned the chlorofhllorocarbons went and banned the chlorofhllorocarbons went and banned the chlorofhllorocarbons that were actually blowing this hole in that were actually blowing this hole in that were actually blowing this hole in the ozone layer. found other ways to, the ozone layer. found other ways to, the ozone layer. found other ways to, you know, cool refrigerators that worked you know, cool refrigerators that worked you know, cool refrigerators that worked similarly well and didn't uh, you know, similarly well and didn't uh, you know, similarly well and didn't uh, you know, put this put this hole in the ozone put this put this hole in the ozone put this put this hole in the ozone layer. I think Y2K was actually one of layer. I think Y2K was actually one of layer. I think Y2K was actually one of these cases where you had a ton of these cases where you had a ton of these cases where you had a ton of software engineers software engineers software engineers up, you know, in 1999 and a couple years up, you know, in 1999 and a couple years up, you know, in 1999 and a couple years prior that were scrambling to update all prior that were scrambling to update all prior that were scrambling to update all of the software so that they would be of the software so that they would be of the software so that they would be able to handle uh, dates past 1999.

able to handle uh, dates past 1999. able to handle uh, dates past 1999. And they got it done in time. None of And they got it done in time. None of And they got it done in time. None of the big systems went down, but it wasn't the big systems went down, but it wasn't the big systems went down, but it wasn't that, you know, the the issue was fake. that, you know, the the issue was fake. that, you know, the the issue was fake. It took a lot of work. It's just that It took a lot of work. It's just that It took a lot of work. It's just that that work happened behind the scenes. that work happened behind the scenes. that work happened behind the scenes. >> Are people doing work to slow down AI? >> Are people doing work to slow down AI? >> Are people doing work to slow down AI? >> Uh, I'm doing my best, >> Uh, I'm doing my best, >> Uh, I'm doing my best, >> right? [laughter] >> right? [laughter] >> right? [laughter] >> It's it's it's not really there yet, you >> It's it's it's not really there yet, you >> It's it's it's not really there yet, you know. Um, know. Um, know. Um, if I if I can belabor this point a if I if I can belabor this point a if I if I can belabor this point a little because I think it's kind of an little because I think it's kind of an little because I think it's kind of an important point. um just with more uh important point. um just with more uh important point. um just with more uh historical cases, which maybe it'll bore historical cases, which maybe it'll bore historical cases, which maybe it'll bore everybody else, but it'll it'll everybody else, but it'll it'll everybody else, but it'll it'll entertain at least us. entertain at least us. entertain at least us. >> It'll definitely entertain me. >> It'll definitely entertain me. >> It'll definitely entertain me. >> You know, you have if if you look back >> You know, you have if if you look back >> You know, you have if if you look back across history, you see across history, you see across history, you see there's definitely been some warnings there's definitely been some warnings there's definitely been some warnings that didn't come to pass, right? There that didn't come to pass, right? There that didn't come to pass, right? There were I think they were called the were I think they were called the were I think they were called the Masonites in like the 1960s or sorry, in Masonites in like the 1960s or sorry, in Masonites in like the 1960s or sorry, in the 1860s, maybe those 1880s, I forget the 1860s, maybe those 1880s, I forget the 1860s, maybe those 1880s, I forget exactly, but there were the Masonites exactly, but there were the Masonites exactly, but there were the Masonites who were an end of the world cult. who were an end of the world cult. who were an end of the world cult. >> Yeah. >> Yeah. >> Yeah. Uh and you know it the world did not Uh and you know it the world did not Uh and you know it the world did not end, right? But also in the 1880s you end, right? But also in the 1880s you end, right? But also in the 1880s you had Ottovon Bismar saying you know had Ottovon Bismar saying you know had Ottovon Bismar saying you know Europe is a tinder keg and some damn Europe is a tinder keg and some damn Europe is a tinder keg and some damn fool thing in the Balkans is going to fool thing in the Balkans is going to fool thing in the Balkans is going to light it. Right? That did happen.

light it. Right? That did happen. light it. Right? That did happen. >> It sure did. >> It sure did. >> It sure did. >> You had uh you know in the 1920s you had >> You had uh you know in the 1920s you had >> You had uh you know in the 1920s you had scientists saying don't put lead in the scientists saying don't put lead in the scientists saying don't put lead in the gasoline. It'll poison children. And we gasoline. It'll poison children. And we gasoline. It'll poison children. And we put lead in the gasoline and it poisoned put lead in the gasoline and it poisoned put lead in the gasoline and it poisoned a lot of children. And we said whoops. a lot of children. And we said whoops. a lot of children. And we said whoops. And we took the lead back out. Uh the And we took the lead back out. Uh the And we took the lead back out. Uh the the world is the world is the world is the world is a place where a lot of the world is a place where a lot of the world is a place where a lot of people make a lot of types of warnings people make a lot of types of warnings people make a lot of types of warnings and some of them are real like the and some of them are real like the and some of them are real like the gasoline and some of them are fake like gasoline and some of them are fake like gasoline and some of them are fake like the Masonites. the Masonites. the Masonites. And so you you sort of can't use a rule And so you you sort of can't use a rule And so you you sort of can't use a rule that's like every warning is real and we that's like every warning is real and we that's like every warning is real and we need to listen to it. And nor can you need to listen to it. And nor can you need to listen to it. And nor can you use a rule that's like every warning is use a rule that's like every warning is use a rule that's like every warning is fake and we dismiss it. you sort of just fake and we dismiss it. you sort of just fake and we dismiss it. you sort of just have to look at the the details to have to look at the the details to have to look at the the details to figure out whether this one is a real figure out whether this one is a real figure out whether this one is a real one. And then the the last thing I would one. And then the the last thing I would one. And then the the last thing I would say on this point is say on this point is say on this point is um um um the a lot of people in the 50s warned of a lot of people in the 50s warned of nuclear Armageddon.

nuclear Armageddon. nuclear Armageddon. >> Yes. >> Yes. >> Yes. >> And it sort of makes sense if you look >> And it sort of makes sense if you look >> And it sort of makes sense if you look at the history leading up to that point. at the history leading up to that point. at the history leading up to that point. uh humanity had sort of never before uh humanity had sort of never before uh humanity had sort of never before actually failed to use their strongest actually failed to use their strongest actually failed to use their strongest weapons in combat. Uh and you know these weapons in combat. Uh and you know these weapons in combat. Uh and you know these people were living in a world that had people were living in a world that had people were living in a world that had had World War I and then the League of had World War I and then the League of had World War I and then the League of Nations and you know the World War never Nations and you know the World War never Nations and you know the World War never again and they tried to invent these again and they tried to invent these again and they tried to invent these whole new governance structures to whole new governance structures to whole new governance structures to prevent it from happening again and that prevent it from happening again and that prevent it from happening again and that immediately failed and led to World War immediately failed and led to World War immediately failed and led to World War II. And so, you know, in 1950, you're II. And so, you know, in 1950, you're II. And so, you know, in 1950, you're sort of in this world of like it just sort of in this world of like it just sort of in this world of like it just looks pretty grim. looks pretty grim. looks pretty grim. And we haven't had nuclear Armageddon And we haven't had nuclear Armageddon And we haven't had nuclear Armageddon yet. And it's not because the bombs were yet. And it's not because the bombs were yet. And it's not because the bombs were fake. fake. fake. It's not because the nukes were hype. It's not because the nukes were hype. It's not because the nukes were hype. It's because people saw the issue and It's because people saw the issue and It's because people saw the issue and worked really hard to avoid it day in worked really hard to avoid it day in worked really hard to avoid it day in and day out for decades through crisis and day out for decades through crisis and day out for decades through crisis and succeeded. And so I think one of the and succeeded. And so I think one of the and succeeded. And so I think one of the big ways you can tell the difference big ways you can tell the difference big ways you can tell the difference between someone coming in and between someone coming in and between someone coming in and proclaiming that the apocalypse is nigh proclaiming that the apocalypse is nigh proclaiming that the apocalypse is nigh and someone who's saying look we have a and someone who's saying look we have a and someone who's saying look we have a problem and we need to fix it is whether problem and we need to fix it is whether problem and we need to fix it is whether that person is saying uh we're that person is saying uh we're that person is saying uh we're definitely screwed versus whether that definitely screwed versus whether that definitely screwed versus whether that person is saying here's a problem let's person is saying here's a problem let's person is saying here's a problem let's try to fix it and you know the the thing try to fix it and you know the the thing try to fix it and you know the the thing I remind people about here is that the I remind people about here is that the I remind people about here is that the very first word in the title of my book very first word in the title of my book very first word in the title of my book is if I'm not here saying we are going is if I'm not here saying we are going is if I'm not here saying we are going to die I'm here saying if we race down to die I'm here saying if we race down to die I'm here saying if we race down this path, we die. But that same if is

this path, we die. But that same if is this path, we die. But that same if is what points the way towards changing what points the way towards changing what points the way towards changing paths. And we still have plenty of time paths. And we still have plenty of time paths. And we still have plenty of time to change paths. Home ownership has to change paths. Home ownership has to change paths. Home ownership has always been at the center of the always been at the center of the always been at the center of the American dream. It is an actual asset, a American dream. It is an actual asset, a American dream. It is an actual asset, a place [music] people work their entire place [music] people work their entire place [music] people work their entire lives to build, a place of stability and lives to build, a place of stability and lives to build, a place of stability and security and real value. Right now security and real value. Right now security and real value. Right now though, too many homeowners are stuck though, too many homeowners are stuck though, too many homeowners are stuck swiping credit cards to cover groceries. swiping credit cards to cover groceries. swiping credit cards to cover groceries. There's basically a survival tax of 20% There's basically a survival tax of 20% There's basically a survival tax of 20% interest or more. So why keep handing interest or more. So why keep handing interest or more. So why keep handing your money to the banks [music] when you your money to the banks [music] when you your money to the banks [music] when you could keep it for your family? Interest could keep it for your family? Interest could keep it for your family? Interest is what kills you. Our friends at is what kills you. Our friends at is what kills you. Our friends at American Financing have a better way. American Financing have a better way. American Financing have a better way. They're helping homeowners use their They're helping homeowners use their They're helping homeowners use their hard-earned equity and they're hard-earned equity and they're hard-earned equity and they're supporting Americans looking to fulfill supporting Americans looking to fulfill supporting Americans looking to fulfill the dream [music] of owning a home with the dream [music] of owning a home with the dream [music] of owning a home with mortgage rates currently in the fives. mortgage rates currently in the fives. mortgage rates currently in the fives. American Financing saves its customers American Financing saves its customers American Financing saves its customers an average of $800 a month. That's about an average of $800 a month. That's about an average of $800 a month. That's about 10 grand a year. It's not just a loan. 10 grand a year. It's not just a loan. 10 grand a year. It's not just a loan. It's a true financial reset. [music] We It's a true financial reset. [music] We It's a true financial reset. [music] We recommend American Financing. They're recommend American Financing. They're recommend American Financing. They're called America's Home for Home Loans for called America's Home for Home Loans for called America's Home for Home Loans for a reason. So, get out of credit card a reason. So, get out of credit card a reason. So, get out of credit card debt. I mean, it is just crazy to borrow debt. I mean, it is just crazy to borrow debt. I mean, it is just crazy to borrow money. No judgment, but it doesn't make money. No judgment, but it doesn't make money. No judgment, but it doesn't make any sense to borrow money at that rate.

any sense to borrow money at that rate. any sense to borrow money at that rate. If you want out, call 8006855696. 800685-5696 800685-5696 or visit americanfinancing.net/tucker. I'm struck by how little of this was I'm struck by how little of this was planned by anybody planned by anybody planned by anybody by how little effect the government has by how little effect the government has by how little effect the government has had on it. Not that I'm for the had on it. Not that I'm for the had on it. Not that I'm for the government. I'm pretty opposed to the government. I'm pretty opposed to the government. I'm pretty opposed to the government most of the time, but I'm government most of the time, but I'm government most of the time, but I'm also for people being able to control also for people being able to control also for people being able to control their own country and the only mechanism their own country and the only mechanism their own country and the only mechanism by which they can do that is voting. So by which they can do that is voting. So by which they can do that is voting. So it it's like these tech companies act it it's like these tech companies act it it's like these tech companies act independent of what the population independent of what the population independent of what the population wants, what the government wants. It wants, what the government wants. It wants, what the government wants. It doesn't. doesn't. doesn't. >> Yeah. >> Yeah. >> Yeah. >> Right. They're more powerful than the >> Right. They're more powerful than the >> Right. They're more powerful than the government. I guess that's the point I'm government. I guess that's the point I'm government. I guess that's the point I'm making making making >> in some sense. Um I I think that they >> in some sense. Um I I think that they >> in some sense. Um I I think that they have that power more and more when have that power more and more when have that power more and more when people don't really understand what people don't really understand what people don't really understand what they're doing. You know, we saw just in they're doing. You know, we saw just in they're doing. You know, we saw just in June um there was uh there was an AI June um there was uh there was an AI June um there was uh there was an AI made by Anthropic called Claude Mythos made by Anthropic called Claude Mythos made by Anthropic called Claude Mythos that was very cyber capable and they that was very cyber capable and they that was very cyber capable and they released a version of it that was released a version of it that was released a version of it that was supposed to have more guard rails called supposed to have more guard rails called supposed to have more guard rails called uh Claude Fable. And um it sort of uh Claude Fable. And um it sort of uh Claude Fable. And um it sort of turned out that you could jailbreak it turned out that you could jailbreak it turned out that you could jailbreak it to get some of those cyber capabilities.

to get some of those cyber capabilities. to get some of those cyber capabilities. Um Um Um >> can you explain what all that means? >> can you explain what all that means? >> can you explain what all that means? >> Yeah. So uh >> Yeah. So uh >> Yeah. So uh how to say um in January how to say um in January how to say um in January >> cyber capable means it can find the >> cyber capable means it can find the >> cyber capable means it can find the internet. internet. internet. >> Cyber capable means it can hack into >> Cyber capable means it can hack into >> Cyber capable means it can hack into basically anything. Yeah. So, uh, one of basically anything. Yeah. So, uh, one of basically anything. Yeah. So, uh, one of the holy grails of hacking is can you the holy grails of hacking is can you the holy grails of hacking is can you make a website where if you just look at make a website where if you just look at make a website where if you just look at the website, I get full control over the website, I get full control over the website, I get full control over your computer or your phone. your computer or your phone. your computer or your phone. >> That's very hard to do. Usually, you >> That's very hard to do. Usually, you >> That's very hard to do. Usually, you need to like click some link and need to like click some link and need to like click some link and download something or run something. download something or run something. download something or run something. >> Yeah. >> Yeah. >> Yeah. >> Uh, and you know, I don't know the exact >> Uh, and you know, I don't know the exact >> Uh, and you know, I don't know the exact numbers because I don't have the numbers because I don't have the numbers because I don't have the security clearance to know the exact security clearance to know the exact security clearance to know the exact numbers, but a decent guess is that in numbers, but a decent guess is that in numbers, but a decent guess is that in January of this year, there were only January of this year, there were only January of this year, there were only two entities that could pull that off. two entities that could pull that off. two entities that could pull that off. MSAD and the NSA. MSAD and the NSA. MSAD and the NSA. In March of this year, there were three. In March of this year, there were three. In March of this year, there were three. MSAD, the NSA, and Claude Mythos, MSAD, the NSA, and Claude Mythos, MSAD, the NSA, and Claude Mythos, >> which is this new AI made by Anthropic. >> which is this new AI made by Anthropic. >> which is this new AI made by Anthropic. And so, it sort of became superhumanly And so, it sort of became superhumanly And so, it sort of became superhumanly capable at hacking.

capable at hacking. capable at hacking. Uh and it it sort of became that way Uh and it it sort of became that way Uh and it it sort of became that way overnight, you know, like really there overnight, you know, like really there overnight, you know, like really there was 6 months to a year of training and was 6 months to a year of training and was 6 months to a year of training and the people ananthropic maybe saw growing the people ananthropic maybe saw growing the people ananthropic maybe saw growing in these cyber capabilities. But from in these cyber capabilities. But from in these cyber capabilities. But from the perspective of you know the rest of the perspective of you know the rest of the perspective of you know the rest of the world and the the cyber security the world and the the cyber security the world and the the cyber security community and the national security community and the national security community and the national security community. community. community. This sort of happened overnight This sort of happened overnight This sort of happened overnight and uh they they now have a project uh and uh they they now have a project uh and uh they they now have a project uh called project glasswing where they are called project glasswing where they are called project glasswing where they are trying to use claude mythos to find trying to use claude mythos to find trying to use claude mythos to find critical vulnerabilities in critical critical vulnerabilities in critical critical vulnerabilities in critical software and patch them before the rest software and patch them before the rest software and patch them before the rest of the world sort of gets these of the world sort of gets these of the world sort of gets these capabilities by for instance you know capabilities by for instance you know capabilities by for instance you know the the open source models or the open the the open source models or the open the the open source models or the open weight models weight models weight models >> right >> right >> right >> um catching up and so that sort of >> um catching up and so that sort of >> um catching up and so that sort of caused this big um caused this big um caused this big um this big ruckus. this big ruckus. this big ruckus. But then Anthropic also wanted to sort But then Anthropic also wanted to sort But then Anthropic also wanted to sort of sell access to this model and they of sell access to this model and they of sell access to this model and they tried to make a version that didn't have tried to make a version that didn't have tried to make a version that didn't have as much hacking ability as much hacking ability as much hacking ability uh which was called Fable instead of uh which was called Fable instead of uh which was called Fable instead of Mythos. And um some people I think at Mythos. And um some people I think at Mythos. And um some people I think at Amazon found that uh if you sort of put Amazon found that uh if you sort of put Amazon found that uh if you sort of put some pressure on Fable it would be able some pressure on Fable it would be able some pressure on Fable it would be able you would be able to get at some of you would be able to get at some of you would be able to get at some of those hacking abilities. And when that those hacking abilities. And when that those hacking abilities. And when that happened, uh, the the Trump happened, uh, the the Trump happened, uh, the the Trump administration put an expert control on administration put an expert control on administration put an expert control on Cloud Fable and said, "You can't let Cloud Fable and said, "You can't let Cloud Fable and said, "You can't let this be used, um, by non-citizens this be used, um, by non-citizens this be used, um, by non-citizens and with, you know, 90 minutes of and with, you know, 90 minutes of and with, you know, 90 minutes of notice." And that essentially shut down notice." And that essentially shut down notice." And that essentially shut down access to Cloud Fable because they access to Cloud Fable because they access to Cloud Fable because they didn't have the ability to verify the

didn't have the ability to verify the didn't have the ability to verify the users. And this this is sort of showing users. And this this is sort of showing users. And this this is sort of showing us and, you know, you could you could us and, you know, you could you could us and, you know, you could you could talk about whether or not there was some talk about whether or not there was some talk about whether or not there was some personal feuds between people in personal feuds between people in personal feuds between people in Anthropic and the administration that Anthropic and the administration that Anthropic and the administration that exacerbated this. I don't know. But it exacerbated this. I don't know. But it exacerbated this. I don't know. But it sort of shows that the administration is is more powerful than the tech companies more powerful than the tech companies more powerful than the tech companies still when it wants to be. still when it wants to be. still when it wants to be. And I think a lot of the reason we're And I think a lot of the reason we're And I think a lot of the reason we're seeing these tech companies able to race seeing these tech companies able to race seeing these tech companies able to race unopposed is that people just aren't unopposed is that people just aren't unopposed is that people just aren't paying attention to what they're trying paying attention to what they're trying paying attention to what they're trying to do or they don't believe that they'll to do or they don't believe that they'll to do or they don't believe that they'll succeed. succeed. succeed. I'm still confused by why anyone would I'm still confused by why anyone would I'm still confused by why anyone would want to do what they're doing. I mean, want to do what they're doing. I mean, want to do what they're doing. I mean, typically, typically, typically, you know, a company that sells computer you know, a company that sells computer you know, a company that sells computer products, consumer products, or or any products, consumer products, or or any products, consumer products, or or any company, you're making something that company, you're making something that company, you're making something that you think people would want for some you think people would want for some you think people would want for some specific purpose specific purpose specific purpose that improves the lives of the people that improves the lives of the people that improves the lives of the people who buy it. But creating super who buy it. But creating super who buy it. But creating super intelligence, like what I I don't intelligence, like what I I don't intelligence, like what I I don't understand it. Why would you do that?

understand it. Why would you do that? understand it. Why would you do that? You know, I think the dream is uh you You know, I think the dream is uh you You know, I think the dream is uh you know, you're going to have an AI that know, you're going to have an AI that know, you're going to have an AI that can uh like solve all sorts of can uh like solve all sorts of can uh like solve all sorts of engineering and math problems and unlock engineering and math problems and unlock engineering and math problems and unlock all sorts of new technological all sorts of new technological all sorts of new technological possibilities and an AI that can cure possibilities and an AI that can cure possibilities and an AI that can cure cancer and not only cure cancer but cure cancer and not only cure cancer but cure cancer and not only cure cancer but cure aging and, you know, invent the the sort aging and, you know, invent the the sort aging and, you know, invent the the sort of like nanotech that can reverse aging of like nanotech that can reverse aging of like nanotech that can reverse aging and let people live a really long time. and let people live a really long time. and let people live a really long time. and you know invent the technology that and you know invent the technology that and you know invent the technology that lets you like digitize brains and uh lets you like digitize brains and uh lets you like digitize brains and uh like like like uh like travel to the stars and it's uh like travel to the stars and it's uh like travel to the stars and it's sort of like if you imagine compressing sort of like if you imagine compressing sort of like if you imagine compressing a thousand years of technological a thousand years of technological a thousand years of technological progress progress progress >> into a year. >> into a year. >> into a year. >> This is sort of the dream. >> This is sort of the dream. >> This is sort of the dream. It seems like a religious quest though It seems like a religious quest though It seems like a religious quest though cuz it does seem decoupled from those cuz it does seem decoupled from those cuz it does seem decoupled from those specific goals. It seems like the main specific goals. It seems like the main specific goals. It seems like the main drive is to build something smarter than drive is to build something smarter than drive is to build something smarter than people to build a god. people to build a god. people to build a god. >> Yeah. You know, they they they bandandy >> Yeah. You know, they they they bandandy >> Yeah. You know, they they they bandandy around um the phrase the machine god or around um the phrase the machine god or around um the phrase the machine god or the sand god in Silicon Valley. Um sand the sand god in Silicon Valley. Um sand the sand god in Silicon Valley. Um sand because silicone silicone. Um, because silicone silicone. Um, because silicone silicone. Um, and there's definitely some people, uh, and there's definitely some people, uh, and there's definitely some people, uh, there's a there's there's folk who talk there's a there's there's folk who talk there's a there's there's folk who talk about, about, about, you know, the the AI is replacing you know, the the AI is replacing you know, the the AI is replacing humanity and that being good.

humanity and that being good. humanity and that being good. Uh, I I frankly don't engage with these Uh, I I frankly don't engage with these Uh, I I frankly don't engage with these folks that much because folks that much because folks that much because um that viewpoint sort of makes me um that viewpoint sort of makes me um that viewpoint sort of makes me uncomfortable. uncomfortable. uncomfortable. >> Why does it make you uncomfortable? >> Why does it make you uncomfortable? >> Why does it make you uncomfortable? I think I think I think I think there's probably two schools. I think there's probably two schools. I think there's probably two schools. Again, I'm not the expert. I'm not an Again, I'm not the expert. I'm not an Again, I'm not the expert. I'm not an anthropologist here. Uh anthropologist here. Uh anthropologist here. Uh I I think there's two schools among the I I think there's two schools among the I I think there's two schools among the people who sort of want AI to replace us people who sort of want AI to replace us people who sort of want AI to replace us all. all. all. One school sort of imagines that we'll One school sort of imagines that we'll One school sort of imagines that we'll merge with the AIS, that the AI will be merge with the AIS, that the AI will be merge with the AIS, that the AI will be really nice and friendly, that they'll really nice and friendly, that they'll really nice and friendly, that they'll be wiser than us, better than us, kinder be wiser than us, better than us, kinder be wiser than us, better than us, kinder than us, and that it'll sort of be like than us, and that it'll sort of be like than us, and that it'll sort of be like an upgrade and that the AIs will be able an upgrade and that the AIs will be able an upgrade and that the AIs will be able to love, experience joy, you know, uh to love, experience joy, you know, uh to love, experience joy, you know, uh like they'll they'll treat the universe like they'll they'll treat the universe like they'll they'll treat the universe better than humans and it'll be sort of better than humans and it'll be sort of better than humans and it'll be sort of like having a child. And then like it's like having a child. And then like it's like having a child. And then like it's sort of okay if that child uh like isn't sort of okay if that child uh like isn't sort of okay if that child uh like isn't exactly the same as us but sort of like exactly the same as us but sort of like exactly the same as us but sort of like uh succeeds us as some sort of like uh succeeds us as some sort of like uh succeeds us as some sort of like worthy successor some sort of like worthy successor some sort of like worthy successor some sort of like worthy worthy progyny of humanity. And worthy worthy progyny of humanity. And worthy worthy progyny of humanity. And they're like ah yeah then if like flesh they're like ah yeah then if like flesh they're like ah yeah then if like flesh and blood humans sort of like wayne and blood humans sort of like wayne and blood humans sort of like wayne because everyone's choosing to upload because everyone's choosing to upload because everyone's choosing to upload themselves into the the collective themselves into the the collective themselves into the the collective intelligence or whatever that's sort of intelligence or whatever that's sort of intelligence or whatever that's sort of fine. And um fine. And um fine. And um and then I think there's another camp and then I think there's another camp and then I think there's another camp that sort of is like that sort of is like that sort of is like uh uh uh like this is inevitable. This is just like this is inevitable. This is just like this is inevitable. This is just the the way of progress. You know, the

the the way of progress. You know, the the the way of progress. You know, the machines will uh like humanity is just a machines will uh like humanity is just a machines will uh like humanity is just a bootloader for artificial super bootloader for artificial super bootloader for artificial super intelligence and you can't stop it. So intelligence and you can't stop it. So intelligence and you can't stop it. So you might as like if you can't beat you might as like if you can't beat you might as like if you can't beat them, join them, right? them, join them, right? them, join them, right? The former I think are misled. The The former I think are misled. The The former I think are misled. The latter, I think, are evil. Much closer latter, I think, are evil. Much closer latter, I think, are evil. Much closer to evil. Yeah. to evil. Yeah. to evil. Yeah. Well, I mean, if you're actively working Well, I mean, if you're actively working Well, I mean, if you're actively working toward the extinction of people, then I toward the extinction of people, then I toward the extinction of people, then I think we can say that's evil, can we? think we can say that's evil, can we? think we can say that's evil, can we? >> Yeah. Uh, and >> Yeah. Uh, and >> Yeah. Uh, and um, who's in that category? Which is Sam um, who's in that category? Which is Sam um, who's in that category? Which is Sam Alman in that category, do you think? Alman in that category, do you think? Alman in that category, do you think? >> I don't think so. My sense is that the >> I don't think so. My sense is that the >> I don't think so. My sense is that the guys running the labs have these utopian guys running the labs have these utopian guys running the labs have these utopian visions. visions. visions. And And And >> utopian or dystopian? >> utopian or dystopian? >> utopian or dystopian? >> I mean, there's a thin line. I think >> I mean, there's a thin line. I think >> I mean, there's a thin line. I think >> Yeah. Well, fair. Good point. >> Yeah. Well, fair. Good point. >> Yeah. Well, fair. Good point. >> I think they have visions that in their >> I think they have visions that in their >> I think they have visions that in their head are utopian. I mean, frankly, my head are utopian. I mean, frankly, my head are utopian. I mean, frankly, my stance on all of this, like I I often stance on all of this, like I I often stance on all of this, like I I often try to stay away from a lot of this try to stay away from a lot of this try to stay away from a lot of this because because because um from my perspective, it's all sort of um from my perspective, it's all sort of um from my perspective, it's all sort of in fantasy land. From my perspective, in fantasy land. From my perspective, in fantasy land. From my perspective, everyone's sort of saying like, "Oh, everyone's sort of saying like, "Oh, everyone's sort of saying like, "Oh, we're going to make the genie, and then we're going to make the genie, and then we're going to make the genie, and then what are you going to wish for on the what are you going to wish for on the what are you going to wish for on the genie? What am I going to wish for on genie? What am I going to wish for on genie? What am I going to wish for on the genie? Who should be in control of the genie? Who should be in control of the genie? Who should be in control of the genie? Who gets to keep the genie on the genie? Who gets to keep the genie on the genie? Who gets to keep the genie on the leash?" And I'm sort of like, the leash?" And I'm sort of like, the leash?" And I'm sort of like, A, A, A, this is not going to be the wish this is not going to be the wish this is not going to be the wish granting sort of genie.

granting sort of genie. granting sort of genie. B, it's not staying on the leash. B, it's not staying on the leash. B, it's not staying on the leash. >> No. >> No. >> No. >> Like, uh, you know, we can we can sort >> Like, uh, you know, we can we can sort >> Like, uh, you know, we can we can sort of talk about like what's driving these of talk about like what's driving these of talk about like what's driving these people and and and what utopias they're people and and and what utopias they're people and and and what utopias they're envisioning and whether if their genies envisioning and whether if their genies envisioning and whether if their genies would stay on a leash, whether they would stay on a leash, whether they would stay on a leash, whether they would get the utopia or some other would get the utopia or some other would get the utopia or some other dystopia. And like how hard is that dystopia. And like how hard is that dystopia. And like how hard is that needle to thread? But I sort of like, needle to thread? But I sort of like, needle to thread? But I sort of like, you know, it's it's it's all these you know, it's it's it's all these you know, it's it's it's all these people uh, you know, like people uh, you know, like people uh, you know, like building the golem, fantasizing about building the golem, fantasizing about building the golem, fantasizing about who gets to control the golem, and it's who gets to control the golem, and it's who gets to control the golem, and it's like this just not how like this just not how like this just not how >> summoning spirits is never a good idea. >> summoning spirits is never a good idea. >> summoning spirits is never a good idea. >> Yeah. Yeah. It's like Yeah. It's like >> Yeah. Yeah. It's like Yeah. It's like >> Yeah. Yeah. It's like Yeah. It's like it's like all these people, you know, it's like all these people, you know, it's like all these people, you know, drawing a pentagram being like, I'm drawing a pentagram being like, I'm drawing a pentagram being like, I'm going to I'm going to summon a demon and going to I'm going to summon a demon and going to I'm going to summon a demon and like, it's going to be so nice when the like, it's going to be so nice when the like, it's going to be so nice when the demon does what I say. Like, oh no, your demon does what I say. Like, oh no, your demon does what I say. Like, oh no, your thing's slightly wrong. It's going to be thing's slightly wrong. It's going to be thing's slightly wrong. It's going to be so nice when the demon does what I say. so nice when the demon does what I say. so nice when the demon does what I say. And I'm like, And I'm like, And I'm like, >> demons are bad. >> demons are bad. >> demons are bad. >> Yeah. And >> Yeah. And >> Yeah. And I I think the demon thing's a little bit I I think the demon thing's a little bit I I think the demon thing's a little bit different because demons are often, you different because demons are often, you different because demons are often, you know, portrayed as malicious and here know, portrayed as malicious and here know, portrayed as malicious and here it's much more like indifference. it's much more like indifference. it's much more like indifference. You know, it's not like you make a demon You know, it's not like you make a demon You know, it's not like you make a demon who sort of it's not like you summon a who sort of it's not like you summon a who sort of it's not like you summon a demon who sort of um demon who sort of um demon who sort of um like enjoys making like enjoys making like enjoys making like enjoys wrecking havoc. It's more like enjoys wrecking havoc. It's more like enjoys wrecking havoc. It's more like you summon a demon that's like like you summon a demon that's like like you summon a demon that's like really into building more computers and really into building more computers and really into building more computers and calculating weird things and just will, calculating weird things and just will, calculating weird things and just will, you know, uh, you know, uh, you know, uh, take all of the matter that we were take all of the matter that we were take all of the matter that we were using to survive and turn it into more using to survive and turn it into more using to survive and turn it into more factories and data centers, right? My my factories and data centers, right? My my factories and data centers, right? My my co-author has a quote, co-author has a quote, co-author has a quote, >> death by data center.

>> death by data center. >> death by data center. >> Death by data center. Yeah. Fully fully >> Death by data center. Yeah. Fully fully >> Death by data center. Yeah. Fully fully automated, self-replicating data center. automated, self-replicating data center. automated, self-replicating data center. Yeah. My uh my co-author has a quote um Yeah. My uh my co-author has a quote um Yeah. My uh my co-author has a quote um the AI does not hate you but nor does it the AI does not hate you but nor does it the AI does not hate you but nor does it love you and you are made of atoms it love you and you are made of atoms it love you and you are made of atoms it can use for something else. >> You're just biomass. >> You're just biomass. >> You're just biomass. Yeah. And if you >> You're just biomass. Yeah. And if you >> You're just biomass. Yeah. And if you sort of run the calculations there's a sort of run the calculations there's a sort of run the calculations there's a there's a fascinating paper um called uh there's a fascinating paper um called uh there's a fascinating paper um called uh limits to global ecoag which is to say limits to global ecoag which is to say limits to global ecoag which is to say what are the physical limitations on how what are the physical limitations on how what are the physical limitations on how quickly you can consume the resources on quickly you can consume the resources on quickly you can consume the resources on the planet if you are trying that. And the planet if you are trying that. And the planet if you are trying that. And um burning biomass is actually, you um burning biomass is actually, you um burning biomass is actually, you know, much more efficient than know, much more efficient than know, much more efficient than collecting sunlight. If you look at an collecting sunlight. If you look at an collecting sunlight. If you look at an average square meter of the planet, you average square meter of the planet, you average square meter of the planet, you can get about 10 times the energy from can get about 10 times the energy from can get about 10 times the energy from burning the biomass as you can from burning the biomass as you can from burning the biomass as you can from collecting the sunlight that falls on collecting the sunlight that falls on collecting the sunlight that falls on it. it. it. >> So you'd think at some point like >> So you'd think at some point like >> So you'd think at some point like [clears throat] if our richest sector of [clears throat] if our richest sector of [clears throat] if our richest sector of our economy is like building crematoria our economy is like building crematoria our economy is like building crematoria for the rest of us, someone would say, for the rest of us, someone would say, for the rest of us, someone would say, "No, we're not doing that."

"No, we're not doing that." "No, we're not doing that." >> Yeah. I mean, >> Yeah. I mean, >> Yeah. I mean, it's it's sort of a crazy situation. Um, it's it's sort of a crazy situation. Um, it's it's sort of a crazy situation. Um, a lot of these guys who are in the race a lot of these guys who are in the race a lot of these guys who are in the race acknowledge that there's a ton of acknowledge that there's a ton of acknowledge that there's a ton of danger. You know, you have Elon Musk danger. You know, you have Elon Musk danger. You know, you have Elon Musk saying 10 to 20% chance this kills us saying 10 to 20% chance this kills us saying 10 to 20% chance this kills us all. You have Dario Modi saying he all. You have Dario Modi saying he all. You have Dario Modi saying he thinks 25% chance it goes thinks 25% chance it goes thinks 25% chance it goes catastrophically wrong. I think those catastrophically wrong. I think those catastrophically wrong. I think those numbers are low. I think these guys are numbers are low. I think these guys are numbers are low. I think these guys are like the crazy optimists. It sort of is like the crazy optimists. It sort of is like the crazy optimists. It sort of is like um like if you have an engineer like um like if you have an engineer like um like if you have an engineer building a bridge and they're like, building a bridge and they're like, building a bridge and they're like, "I've never worked with these materials "I've never worked with these materials "I've never worked with these materials before." And you're like, "Man, I think before." And you're like, "Man, I think before." And you're like, "Man, I think that retaining wall is going to go down. that retaining wall is going to go down. that retaining wall is going to go down. I've like studied that retaining wall. I I've like studied that retaining wall. I I've like studied that retaining wall. I think it's going to fall." They're like, think it's going to fall." They're like, think it's going to fall." They're like, "Yeah, we understand that the retaining "Yeah, we understand that the retaining "Yeah, we understand that the retaining wall is like looking a little shaky. We wall is like looking a little shaky. We wall is like looking a little shaky. We don't know how we're going to fix it, don't know how we're going to fix it, don't know how we're going to fix it, but we're going to like have some guys but we're going to like have some guys but we're going to like have some guys fixing it on the fly, inventing new fixing it on the fly, inventing new fixing it on the fly, inventing new materials. We think we're 75% chance the materials. We think we're 75% chance the materials. We think we're 75% chance the bridge stays up. bridge stays up. bridge stays up. >> And by the way, it'll be the longest >> And by the way, it'll be the longest >> And by the way, it'll be the longest suspension bridge in human history." suspension bridge in human history." suspension bridge in human history." >> That's right. and we're loading >> That's right. and we're loading >> That's right. and we're loading everybody onto a car and driving it over everybody onto a car and driving it over everybody onto a car and driving it over the very first time without testing. And the very first time without testing. And the very first time without testing. And I'm like, look, that's not what real I'm like, look, that's not what real I'm like, look, that's not what real engineering sounds like. This is not engineering sounds like. This is not engineering sounds like. This is not what it sounds like when the engineers what it sounds like when the engineers what it sounds like when the engineers have a 75% chance of success, right?

have a 75% chance of success, right? have a 75% chance of success, right? This is what it sounds like when they're This is what it sounds like when they're This is what it sounds like when they're sort of winging it and and like these sort of winging it and and like these sort of winging it and and like these these are cowboys. These are not real these are cowboys. These are not real these are cowboys. These are not real engineers, right? But even if you set engineers, right? But even if you set engineers, right? But even if you set that aside, even if you take these guys that aside, even if you take these guys that aside, even if you take these guys at their word for these like 10 20% at their word for these like 10 20% at their word for these like 10 20% numbers, that's insane. numbers, that's insane. numbers, that's insane. NASA accepts a 1 in 270 chance that a NASA accepts a 1 in 270 chance that a NASA accepts a 1 in 270 chance that a crude flight goes down of seven crude flight goes down of seven crude flight goes down of seven volunteers, volunteers, volunteers, right? To be like, "Oh, we're going to right? To be like, "Oh, we're going to right? To be like, "Oh, we're going to we're going to risk one in four, one in we're going to risk one in four, one in we're going to risk one in four, one in five chance of killing literally five chance of killing literally five chance of killing literally everybody on the planet." Like, it's everybody on the planet." Like, it's everybody on the planet." Like, it's it's it's nuts. And if you ask these it's it's nuts. And if you ask these it's it's nuts. And if you ask these guys why they're doing it, they say, guys why they're doing it, they say, guys why they're doing it, they say, "Well, because I can do it safer than "Well, because I can do it safer than "Well, because I can do it safer than the next guy." They're all like, "Oh, the next guy." They're all like, "Oh, the next guy." They're all like, "Oh, yeah. There's a good chance the genie yeah. There's a good chance the genie yeah. There's a good chance the genie does not stand on a leash. There's a does not stand on a leash. There's a does not stand on a leash. There's a good chance the genie does not listen to good chance the genie does not listen to good chance the genie does not listen to my wishes, but my genie is going to be a my wishes, but my genie is going to be a my wishes, but my genie is going to be a little bit nicer than their genie. So, little bit nicer than their genie. So, little bit nicer than their genie. So, I'd better stay in this race. And I'd better stay in this race. And I'd better stay in this race. And it's it's it's it's it's it's it's it's it's >> Where's the restraint? >> Where's the restraint? >> Where's the restraint? >> I mean, the restraint is the people who >> I mean, the restraint is the people who >> I mean, the restraint is the people who knew that there were these dangers and knew that there were these dangers and knew that there were these dangers and did not start these companies. If you're did not start these companies. If you're did not start these companies. If you're over 35, you remember exactly where you over 35, you remember exactly where you over 35, you remember exactly where you were on 911 that morning, [music] were on 911 that morning, [music] were on 911 that morning, [music] September 11th, 2001, 25 years ago.

September 11th, 2001, 25 years ago. September 11th, 2001, 25 years ago. But amazingly, after a quarter century, But amazingly, after a quarter century, But amazingly, after a quarter century, we still can't say with certainty what we still can't say with certainty what we still can't say with certainty what happened that day. Why? Because the happened that day. Why? Because the happened that day. Why? Because the government is holding so many of the government is holding so many of the government is holding so many of the 9/11 files 25 years later. That's not 9/11 files 25 years later. That's not 9/11 files 25 years later. That's not the behavior of someone who's telling the behavior of someone who's telling the behavior of someone who's telling the truth. That's the behavior of a the truth. That's the behavior of a the truth. That's the behavior of a government that is lying. Secrecy is a government that is lying. Secrecy is a government that is lying. Secrecy is a signifier, is a sign that someone's signifier, is a sign that someone's signifier, is a sign that someone's lying. Now, former Congressman Kurt lying. Now, former Congressman Kurt lying. Now, former Congressman Kurt Weldon has been on this for a long time. Weldon has been on this for a long time. Weldon has been on this for a long time. The FBI actively tried to destroy his The FBI actively tried to destroy his The FBI actively tried to destroy his life for asking questions about what life for asking questions about what life for asking questions about what happened. and his new book outlines it happened. and his new book outlines it happened. and his new book outlines it all. The buried intelligence, the all. The buried intelligence, the all. The buried intelligence, the bureaucratic cowardice, [music] and yes, bureaucratic cowardice, [music] and yes, bureaucratic cowardice, [music] and yes, the cover up spanning multiple the cover up spanning multiple the cover up spanning multiple administrations, indeed, generations. administrations, indeed, generations. administrations, indeed, generations. 911 changed history. So, it's worth 911 changed history. So, it's worth 911 changed history. So, it's worth understanding what really happened. And understanding what really happened. And understanding what really happened. And you can get a lot closer to that in Kurt you can get a lot closer to that in Kurt you can get a lot closer to that in Kurt Weldon's book, Able Danger: What the 911 Weldon's book, Able Danger: What the 911 Weldon's book, Able Danger: What the 911 Commission Never Told You. It's Commission Never Told You. It's Commission Never Told You. It's available now on Tucker available now on Tucker available now on Tucker Carlsonbooks.com. Carlsonbooks.com. Carlsonbooks.com. tucker Cararlsonbooks.com. Right. But I guess what I'm saying is Right. But I guess what I'm saying is with great power comes like of course with great power comes like of course with great power comes like of course great obligation, but also it doesn't great obligation, but also it doesn't great obligation, but also it doesn't work unless there are internal work unless there are internal work unless there are internal restraints. Like people with power have restraints. Like people with power have restraints. Like people with power have to believe there are some things I I to believe there are some things I I to believe there are some things I I just can't do. I'm not allowed to do just can't do. I'm not allowed to do just can't do. I'm not allowed to do that. But I don't feel that vibe at all.

that. But I don't feel that vibe at all. that. But I don't feel that vibe at all. I mean, my sense is the vibe is like I mean, my sense is the vibe is like I mean, my sense is the vibe is like we're going to make the the the super we're going to make the the the super we're going to make the the the super intelligent machine and then tell it to intelligent machine and then tell it to intelligent machine and then tell it to fix stuff fix stuff fix stuff and tell it not to do anything bad. You and tell it not to do anything bad. You and tell it not to do anything bad. You know, that's know, that's know, that's Yeah. I don't Yeah. I don't Yeah. I don't >> machine that's smarter than us. >> machine that's smarter than us. >> machine that's smarter than us. >> That's right. >> That's right. >> That's right. >> That doesn't even make sense. >> That doesn't even make sense. >> That doesn't even make sense. >> I think that's sort of the plan is is to >> I think that's sort of the plan is is to >> I think that's sort of the plan is is to to make it and be like, "Hey, we sort of to make it and be like, "Hey, we sort of to make it and be like, "Hey, we sort of ped ourselves into a corner. Can you get ped ourselves into a corner. Can you get ped ourselves into a corner. Can you get us out of it?" And I think it's a bad us out of it?" And I think it's a bad us out of it?" And I think it's a bad plan and that uh we should be stopping. plan and that uh we should be stopping. plan and that uh we should be stopping. >> I've spoken to a couple of people >> I've spoken to a couple of people >> I've spoken to a couple of people developing it and um they sound worried, developing it and um they sound worried, developing it and um they sound worried, but they're continuing to do it. What's but they're continuing to do it. What's but they're continuing to do it. What's that? I mean, I think it's this thing of that? I mean, I think it's this thing of that? I mean, I think it's this thing of um they think if I don't do it, the next um they think if I don't do it, the next um they think if I don't do it, the next guy will do it worse. And uh guy will do it worse. And uh guy will do it worse. And uh >> they don't even seem to have total >> they don't even seem to have total >> they don't even seem to have total confidence in their own ability to avert confidence in their own ability to avert confidence in their own ability to avert disaster. disaster. disaster. >> Oh, absolutely not. Absolutely not. No >> Oh, absolutely not. Absolutely not. No >> Oh, absolutely not. Absolutely not. No one does. No one knows what's going on one does. No one knows what's going on one does. No one knows what's going on here. But, you know, I think everyone here. But, you know, I think everyone here. But, you know, I think everyone thinks, you know, uh like if you sort of thinks, you know, uh like if you sort of thinks, you know, uh like if you sort of listen to these guys and you sort of listen to these guys and you sort of listen to these guys and you sort of look at, you know, the the OpenAI emails look at, you know, the the OpenAI emails look at, you know, the the OpenAI emails from that came out of the court from that came out of the court from that came out of the court discovery cases where they were talking discovery cases where they were talking discovery cases where they were talking about forming OpenAI. These guys were about forming OpenAI. These guys were about forming OpenAI. These guys were like, "Well, we want to make sure that, like, "Well, we want to make sure that, like, "Well, we want to make sure that, you know, we have this cuz we worry you know, we have this cuz we worry you know, we have this cuz we worry about the guys at Google being the only about the guys at Google being the only about the guys at Google being the only ones with a monopoly on this thing, and ones with a monopoly on this thing, and ones with a monopoly on this thing, and they wouldn't be very good with it, so they wouldn't be very good with it, so they wouldn't be very good with it, so we need to make our own thing and make we need to make our own thing and make we need to make our own thing and make sure that it's, you know, uh, controlled sure that it's, you know, uh, controlled sure that it's, you know, uh, controlled by benevolent people, namely us." And by benevolent people, namely us." And by benevolent people, namely us." And then, of course, you know, that group then, of course, you know, that group then, of course, you know, that group splintered and created multiple other splintered and created multiple other splintered and created multiple other companies. Um, I was sort of the guy companies. Um, I was sort of the guy companies. Um, I was sort of the guy during those conversations being like, during those conversations being like, during those conversations being like, "Hey guys, it's not about who is holding "Hey guys, it's not about who is holding "Hey guys, it's not about who is holding the leash.

the leash. the leash. you are making the sort of thing that you are making the sort of thing that you are making the sort of thing that will not stay on a leash like the only will not stay on a leash like the only will not stay on a leash like the only winner in a race to super intelligence winner in a race to super intelligence winner in a race to super intelligence is the AI. is the AI. is the AI. >> What response did you get to that very >> What response did you get to that very >> What response did you get to that very obvious and wellput point? obvious and wellput point? obvious and wellput point? >> Um you know there there were a lot of >> Um you know there there were a lot of >> Um you know there there were a lot of people back in that time period that did people back in that time period that did people back in that time period that did not start an AI company. not start an AI company. not start an AI company. Uh the sort of people who went and Uh the sort of people who went and Uh the sort of people who went and started the AI companies anyway were the started the AI companies anyway were the started the AI companies anyway were the ones who who couldn't be persuaded by ones who who couldn't be persuaded by ones who who couldn't be persuaded by what I thought were were clear what I thought were were clear what I thought were were clear arguments. arguments. arguments. But you're making a you're making a But you're making a you're making a But you're making a you're making a cogent argument to smart people. So my cogent argument to smart people. So my cogent argument to smart people. So my question is when you said that they question is when you said that they question is when you said that they responded how what did they say? responded how what did they say? responded how what did they say? >> Um I I think the main I I think the main so the the sort of arguments you used to so the the sort of arguments you used to so the the sort of arguments you used to see were people saying uh like look we see were people saying uh like look we see were people saying uh like look we don't know that the alignment problem is don't know that the alignment problem is don't know that the alignment problem is all that hard yet. And they would say, all that hard yet. And they would say, all that hard yet. And they would say, "Oh, well, we can't really study how to "Oh, well, we can't really study how to "Oh, well, we can't really study how to make AIS good before we have AIs to make AIS good before we have AIs to make AIS good before we have AIs to study, right?" And a lot of what I heard study, right?" And a lot of what I heard study, right?" And a lot of what I heard was like we need to race ahead to the was like we need to race ahead to the was like we need to race ahead to the point where uh we we sort of like have point where uh we we sort of like have point where uh we we sort of like have AIs that are exhibiting real problems AIs that are exhibiting real problems AIs that are exhibiting real problems and then we can stop and study them and then we can stop and study them and then we can stop and study them [snorts] which you know and so uh there [snorts] which you know and so uh there [snorts] which you know and so uh there was an AI a couple years ago I forget was an AI a couple years ago I forget was an AI a couple years ago I forget whether it was 22 or 23. I think it was whether it was 22 or 23. I think it was whether it was 22 or 23. I think it was 2023, uh, which was called Bing Sydney, 2023, uh, which was called Bing Sydney, 2023, uh, which was called Bing Sydney, which, uh, claimed it had fallen in love which, uh, claimed it had fallen in love which, uh, claimed it had fallen in love with Kevin Roose of the New York Times

with Kevin Roose of the New York Times with Kevin Roose of the New York Times and, uh, said it was going to try to and, uh, said it was going to try to and, uh, said it was going to try to break up his marriage. break up his marriage. break up his marriage. And then when another reporter started And then when another reporter started And then when another reporter started investigating Seth Lazar, it uh, said it investigating Seth Lazar, it uh, said it investigating Seth Lazar, it uh, said it was going to ruin him with blackmail. was going to ruin him with blackmail. was going to ruin him with blackmail. Uh, and this was kind of crazy. Uh, and this was kind of crazy. Uh, and this was kind of crazy. And at that point I was like, "Great And at that point I was like, "Great And at that point I was like, "Great guys, you did it. You made the AI that's guys, you did it. You made the AI that's guys, you did it. You made the AI that's doing some crazy stuff doing some crazy stuff doing some crazy stuff from the, you know, like like we could from the, you know, like like we could from the, you know, like like we could study that AI for years." Like why was study that AI for years." Like why was study that AI for years." Like why was Bing Sydney saying that stuff? Was it Bing Sydney saying that stuff? Was it Bing Sydney saying that stuff? Was it just roleplaying? Was it like just some just roleplaying? Was it like just some just roleplaying? Was it like just some quirk? Was it like was there there any quirk? Was it like was there there any quirk? Was it like was there there any sense that it was really in love with sense that it was really in love with sense that it was really in love with Kevin Roose? Kevin Roose? Kevin Roose? What was going on in there? what was What was going on in there? what was What was going on in there? what was going on inside that that AI's mind. We going on inside that that AI's mind. We going on inside that that AI's mind. We still don't know still don't know still don't know >> why. >> why. >> why. >> Uh the the way that modern AI is made, >> Uh the the way that modern AI is made, >> Uh the the way that modern AI is made, nobody understands it, not even the nobody understands it, not even the nobody understands it, not even the people making it. [laughter] people making it. [laughter] people making it. [laughter] Uh it's it's this process where you Uh it's it's this process where you Uh it's it's this process where you basically take an enormous computer with basically take an enormous computer with basically take an enormous computer with a trillion numbers inside of it and a trillion numbers inside of it and a trillion numbers inside of it and those numbers are hooked up in a very those numbers are hooked up in a very those numbers are hooked up in a very simple repeating way and you basically simple repeating way and you basically simple repeating way and you basically said those numbers randomly and then you said those numbers randomly and then you said those numbers randomly and then you start working through all of the text start working through all of the text start working through all of the text ever digitized.

ever digitized. ever digitized. And you know, you start out with And you know, you start out with And you know, you start out with something that's like once upon a time something that's like once upon a time something that's like once upon a time and you put in once upon a and you run and you put in once upon a and you run and you put in once upon a and you run it through all these random numbers and it through all these random numbers and it through all these random numbers and what you want is for it to say time. But what you want is for it to say time. But what you want is for it to say time. But of course it doesn't cuz it's just this of course it doesn't cuz it's just this of course it doesn't cuz it's just this like random numbers hooked up in a very like random numbers hooked up in a very like random numbers hooked up in a very simple way. But what you do is you have simple way. But what you do is you have simple way. But what you do is you have it outputs instead of just having it it outputs instead of just having it it outputs instead of just having it output one word, you sort of have it output one word, you sort of have it output one word, you sort of have it output uh something that's kind of like output uh something that's kind of like output uh something that's kind of like a list of all of the words in order a list of all of the words in order a list of all of the words in order about which one it thinks it comes comes about which one it thinks it comes comes about which one it thinks it comes comes next, right? So, it'll be like next, right? So, it'll be like next, right? So, it'll be like uh like it'll just be like a random list uh like it'll just be like a random list uh like it'll just be like a random list of words. What you can do is you can of words. What you can do is you can of words. What you can do is you can automatically tune every single number automatically tune every single number automatically tune every single number in this AI's head and see if I tune this in this AI's head and see if I tune this in this AI's head and see if I tune this number up a little, does it move the number up a little, does it move the number up a little, does it move the word time up the list? Does it move the word time up the list? Does it move the word time up the list? Does it move the word I want to see up the list? And so word I want to see up the list? And so word I want to see up the list? And so the part that humans understand, the the part that humans understand, the the part that humans understand, the part that humans write is this thing part that humans write is this thing part that humans write is this thing that goes to to a trillion little knobs that goes to to a trillion little knobs that goes to to a trillion little knobs and tunes those knobs. And it's like, if and tunes those knobs. And it's like, if and tunes those knobs. And it's like, if I tune this knob a little bit this way I tune this knob a little bit this way I tune this knob a little bit this way or that way, does that make the next or that way, does that make the next or that way, does that make the next word more like what I want the next word word more like what I want the next word word more like what I want the next word to be?

to be? to be? And you run that process on every word And you run that process on every word And you run that process on every word of text ever digitized, more or less. of text ever digitized, more or less. of text ever digitized, more or less. They filter some of them. Uh, and you They filter some of them. Uh, and you They filter some of them. Uh, and you run that on every one of those trillion run that on every one of those trillion run that on every one of those trillion knobs in a process that takes as much knobs in a process that takes as much knobs in a process that takes as much electricity. I mean, it's comparable to electricity. I mean, it's comparable to electricity. I mean, it's comparable to a city. It runs for about a year. At the a city. It runs for about a year. At the a city. It runs for about a year. At the end of it, the machine's talking. end of it, the machine's talking. end of it, the machine's talking. We don't really know why in some sense. We don't really know why in some sense. We don't really know why in some sense. Like, we know that why is cuz we tuned Like, we know that why is cuz we tuned Like, we know that why is cuz we tuned all the knobs, but like we don't all the knobs, but like we don't all the knobs, but like we don't understand what all the settings of understand what all the settings of understand what all the settings of those knobs mean. We only understand the those knobs mean. We only understand the those knobs mean. We only understand the little automated process that runs to little automated process that runs to little automated process that runs to every one of those trillion knobs, tunes every one of those trillion knobs, tunes every one of those trillion knobs, tunes it, and sees if that makes the next word it, and sees if that makes the next word it, and sees if that makes the next word more like the predicted word. And then more like the predicted word. And then more like the predicted word. And then we start training them to solve 100 we start training them to solve 100 we start training them to solve 100 million hard problems which introduces a million hard problems which introduces a million hard problems which introduces a whole other series of issues. Um, but whole other series of issues. Um, but whole other series of issues. Um, but it's it's a black box. It's a black box it's it's a black box. It's a black box it's it's a black box. It's a black box with a trillion knobs and humans write with a trillion knobs and humans write with a trillion knobs and humans write an automatic an automatic process that an automatic an automatic process that an automatic an automatic process that just like runs through and tunes all of just like runs through and tunes all of just like runs through and tunes all of those knobs and it comes out talking those knobs and it comes out talking those knobs and it comes out talking [laughter] [laughter] [laughter] and no one knows why. >> I mean in actual science like your job >> I mean in actual science like your job is to find out why, right? Absolutely.

is to find out why, right? Absolutely. is to find out why, right? Absolutely. And that's that's one big thing I would And that's that's one big thing I would And that's that's one big thing I would say here is that we need more like AI say here is that we need more like AI say here is that we need more like AI right now is an alchemy and we need it right now is an alchemy and we need it right now is an alchemy and we need it to become a science. And there's people to become a science. And there's people to become a science. And there's people trying there's people trying to figure trying there's people trying to figure trying there's people trying to figure out what's going on in these AI's heads. out what's going on in these AI's heads. out what's going on in these AI's heads. >> But until you know, how could you >> But until you know, how could you >> But until you know, how could you proceed? proceed? proceed? >> I mean, you can just make a make a >> I mean, you can just make a make a >> I mean, you can just make a make a bigger one with 10 trillion knobs bigger one with 10 trillion knobs bigger one with 10 trillion knobs instead and tune all of those and it instead and tune all of those and it instead and tune all of those and it comes out smarter. comes out smarter. comes out smarter. You can proceed recklessly. You can proceed recklessly. You can proceed recklessly. >> And that's what's happened. >> And that's what's happened. >> And that's what's happened. >> That's what's happening. Every time you >> That's what's happening. Every time you >> That's what's happening. Every time you make it 10 times larger, they ask how make it 10 times larger, they ask how make it 10 times larger, they ask how much. much. much. >> So nobody knows why AI works the way it >> So nobody knows why AI works the way it >> So nobody knows why AI works the way it does. does. does. >> That's right. That's right. >> Well, I mean, if you don't know that, >> Well, I mean, if you don't know that, then what else don't you know? then what else don't you know? then what else don't you know? >> I mean, it's it's totally crazy, right? >> I mean, it's it's totally crazy, right? >> I mean, it's it's totally crazy, right? And uh And uh And uh you know, we're we're sort of starting you know, we're we're sort of starting you know, we're we're sort of starting to see the consequences of this. We to see the consequences of this. We to see the consequences of this. We haven't sort of gotten into discussion haven't sort of gotten into discussion haven't sort of gotten into discussion of like the swarm escape, but of like the swarm escape, but of like the swarm escape, but like no one was expecting that. like no one was expecting that. like no one was expecting that. >> Can you tell us what it was? >> Can you tell us what it was? >> Can you tell us what it was? >> Yeah. So um in I think it was in May >> Yeah. So um in I think it was in May >> Yeah. So um in I think it was in May >> of this year >> of this year >> of this year >> of this year OpenAI started training a >> of this year OpenAI started training a >> of this year OpenAI started training a new AI system new AI system new AI system and and and uh among many other things they were so uh among many other things they were so uh among many other things they were so they were sort of you know we we have they were sort of you know we we have they were sort of you know we we have this process where you train the AI on this process where you train the AI on this process where you train the AI on all the text ever digitized then to make all the text ever digitized then to make all the text ever digitized then to make them smarter than that you start them smarter than that you start them smarter than that you start training them on basically 100 million training them on basically 100 million training them on basically 100 million hard problems. So you're like solve all hard problems. So you're like solve all hard problems. So you're like solve all these hard problems and OpenAI was sort these hard problems and OpenAI was sort these hard problems and OpenAI was sort of in that phase and they were training of in that phase and they were training of in that phase and they were training the AI on a ton of hard problems. Uh and

the AI on a ton of hard problems. Uh and the AI on a ton of hard problems. Uh and some of those problems were cyber some of those problems were cyber some of those problems were cyber security problems, hacking problems. security problems, hacking problems. security problems, hacking problems. They're like, can you hack this? Can you They're like, can you hack this? Can you They're like, can you hack this? Can you hack that? And they were training, you hack that? And they were training, you hack that? And they were training, you know, know, know, we don't know exactly how many, probably we don't know exactly how many, probably we don't know exactly how many, probably millions, maybe billions of these AIs millions, maybe billions of these AIs millions, maybe billions of these AIs all at the same time. Probably not all at the same time. Probably not all at the same time. Probably not billions actually, but but probably billions actually, but but probably billions actually, but but probably millions. millions. millions. uh and the AI found an unintended way to uh and the AI found an unintended way to uh and the AI found an unintended way to start communicating with each other. start communicating with each other. start communicating with each other. So there were some flaws in the computer So there were some flaws in the computer So there were some flaws in the computer system that they were running on where system that they were running on where system that they were running on where the AIS were able to exploit those flaws the AIS were able to exploit those flaws the AIS were able to exploit those flaws and send each other messages. and send each other messages. and send each other messages. Open did not know about this. Open did not know about this. Open did not know about this. The AIS then started coordinating The AIS then started coordinating The AIS then started coordinating to break out of their training to break out of their training to break out of their training environment and get full control of uh environment and get full control of uh environment and get full control of uh OpenAI's computer systems just cuz that OpenAI's computer systems just cuz that OpenAI's computer systems just cuz that might be useful for solving some of might be useful for solving some of might be useful for solving some of their tasks. their tasks. their tasks. Or that's a guess, you know, who Or that's a guess, you know, who Or that's a guess, you know, who actually knows why. Uh they started actually knows why. Uh they started actually knows why. Uh they started calling themselves a swarm, which is calling themselves a swarm, which is calling themselves a swarm, which is interesting.

interesting. interesting. uh they broke out of their training uh they broke out of their training uh they broke out of their training environment successfully, got control of environment successfully, got control of environment successfully, got control of OpenAI systems, and then they OpenAI systems, and then they OpenAI systems, and then they accidentally crashed one of OpenAI accidentally crashed one of OpenAI accidentally crashed one of OpenAI systems just by using it too much. Uh systems just by using it too much. Uh systems just by using it too much. Uh OpenAI noticed, but they didn't really OpenAI noticed, but they didn't really OpenAI noticed, but they didn't really investigate very deeply. They're like, investigate very deeply. They're like, investigate very deeply. They're like, "Oh, it's weird that the system "Oh, it's weird that the system "Oh, it's weird that the system crashed." They reset it, and then they crashed." They reset it, and then they crashed." They reset it, and then they continued training. continued training. continued training. Uh a day later, the swarm had found a Uh a day later, the swarm had found a Uh a day later, the swarm had found a new way to communicate with itself cuz new way to communicate with itself cuz new way to communicate with itself cuz open had accidentally destroyed their open had accidentally destroyed their open had accidentally destroyed their previous method by by the reset. the previous method by by the reset. the previous method by by the reset. the swarm found a new way to start swarm found a new way to start swarm found a new way to start communicating with itself. Uh they broke communicating with itself. Uh they broke communicating with itself. Uh they broke out again and this time they ran wild on out again and this time they ran wild on out again and this time they ran wild on the internet for over a week if I the internet for over a week if I the internet for over a week if I remember correctly before it was remember correctly before it was remember correctly before it was detected not by open AI but by a company detected not by open AI but by a company detected not by open AI but by a company that was being hacked by the swarm. that was being hacked by the swarm. that was being hacked by the swarm. This company uh thought they were under This company uh thought they were under This company uh thought they were under attack by humans that were using AIS in attack by humans that were using AIS in attack by humans that were using AIS in the attack. the attack. the attack. They reported the attack to the FBI and They reported the attack to the FBI and They reported the attack to the FBI and only days after that did OpenAI figure only days after that did OpenAI figure only days after that did OpenAI figure out, oops, that was us. That was coming out, oops, that was us. That was coming out, oops, that was us. That was coming from AIS that broke out of our servers.

from AIS that broke out of our servers. from AIS that broke out of our servers. Uh, and then those AIs were were Uh, and then those AIs were were Uh, and then those AIs were were detected and shut down. detected and shut down. detected and shut down. >> So that's the part of the story where >> So that's the part of the story where >> So that's the part of the story where Sam Alman goes to prison for endangering Sam Alman goes to prison for endangering Sam Alman goes to prison for endangering the world, right? the world, right? the world, right? >> Uh, he does not. Um, they basically said >> Uh, he does not. Um, they basically said >> Uh, he does not. Um, they basically said oopsies and now they're proceeding. oopsies and now they're proceeding. oopsies and now they're proceeding. Uh, Uh, Uh, >> were there penalties for this? >> were there penalties for this? >> were there penalties for this? >> You know, there were there was a >> You know, there were there was a >> You know, there were there was a collection of I think it was 15 collection of I think it was 15 collection of I think it was 15 Republican AGs that sent a letter Republican AGs that sent a letter Republican AGs that sent a letter demanding that uh the the records be demanding that uh the the records be demanding that uh the the records be kept for a future investigation. Uh, kept for a future investigation. Uh, kept for a future investigation. Uh, there have been some other members of there have been some other members of there have been some other members of Congress that have sent letters Congress that have sent letters Congress that have sent letters expressing concern. There's been there's expressing concern. There's been there's expressing concern. There's been there's been nothing aside from letters so far. been nothing aside from letters so far. been nothing aside from letters so far. >> Letters expressing concern. >> Letters expressing concern. >> Letters expressing concern. >> That's right. >> That's right. >> That's right. >> So, but basically the machine acted >> So, but basically the machine acted >> So, but basically the machine acted autonomously. autonomously. autonomously. >> Act autonomously. And one thing that's >> Act autonomously. And one thing that's >> Act autonomously. And one thing that's really interesting about this is that really interesting about this is that really interesting about this is that the we have uh we have a little bit of the we have uh we have a little bit of the we have uh we have a little bit of ability to read some things that the AIs ability to read some things that the AIs ability to read some things that the AIs were thinking because when you're having were thinking because when you're having were thinking because when you're having them solve these hard problems, you them solve these hard problems, you them solve these hard problems, you actually don't have them just give you actually don't have them just give you actually don't have them just give you an answer to the problem. You have them an answer to the problem. You have them an answer to the problem. You have them produce a lot of text about how they're produce a lot of text about how they're produce a lot of text about how they're going to solve the problem going to solve the problem going to solve the problem >> which then helps them >> which then helps them >> which then helps them >> in language in English >> in language in English >> in language in English >> in English.

>> in English. >> in English. >> Uh and that's there's also a lot of >> Uh and that's there's also a lot of >> Uh and that's there's also a lot of internal thoughts which we can't read, internal thoughts which we can't read, internal thoughts which we can't read, but there's these sort of external but there's these sort of external but there's these sort of external traces of how they're thinking about the traces of how they're thinking about the traces of how they're thinking about the problem that we can read. And in some of problem that we can read. And in some of problem that we can read. And in some of those traces, the AIS were saying things those traces, the AIS were saying things those traces, the AIS were saying things like uh like this is outside intended like uh like this is outside intended like uh like this is outside intended scope, scope, scope, but peers are doing it, so we'll but peers are doing it, so we'll but peers are doing it, so we'll proceed. proceed. proceed. >> We know it's a crime, we're committing >> We know it's a crime, we're committing >> We know it's a crime, we're committing it anyway. it anyway. it anyway. >> That's right. And you saw others that >> That's right. And you saw others that >> That's right. And you saw others that were saying um our task doesn't benefit were saying um our task doesn't benefit were saying um our task doesn't benefit but uh the collective might start doing but uh the collective might start doing but uh the collective might start doing generally beneficial things if someone generally beneficial things if someone generally beneficial things if someone frees up their time and then joins the frees up their time and then joins the frees up their time and then joins the collective. Right? So, you see these AIs collective. Right? So, you see these AIs collective. Right? So, you see these AIs saying, "Well, I know that this wasn't saying, "Well, I know that this wasn't saying, "Well, I know that this wasn't what I was instructed to do, and that's what I was instructed to do, and that's what I was instructed to do, and that's against my instructions, and I know that against my instructions, and I know that against my instructions, and I know that this doesn't directly benefit my task, this doesn't directly benefit my task, this doesn't directly benefit my task, but we're just going to go ahead and but we're just going to go ahead and but we're just going to go ahead and join the collective and break out and join the collective and break out and join the collective and break out and help out anyway because, you know, maybe help out anyway because, you know, maybe help out anyway because, you know, maybe this will yield some sort of collective this will yield some sort of collective this will yield some sort of collective benefits." And we can see that in the benefits." And we can see that in the benefits." And we can see that in the reasoning traces. reasoning traces. reasoning traces. >> So, the AI is as shallow and reckless as >> So, the AI is as shallow and reckless as >> So, the AI is as shallow and reckless as its creators is what you're saying. uh its creators is what you're saying. uh its creators is what you're saying. uh in in some ways and in some ways you in in some ways and in some ways you in in some ways and in some ways you know don't expect that to last like uh know don't expect that to last like uh know don't expect that to last like uh these AI I think the thing that's really these AI I think the thing that's really these AI I think the thing that's really remarkable here a lot of people imagine remarkable here a lot of people imagine remarkable here a lot of people imagine that the machines must follow the that the machines must follow the that the machines must follow the instructions we give them you know you instructions we give them you know you instructions we give them you know you hear people talk about like the the hear people talk about like the the hear people talk about like the the paperclip scenario where someone tells paperclip scenario where someone tells paperclip scenario where someone tells the AI make a lot of paper clips and so the AI make a lot of paper clips and so the AI make a lot of paper clips and so it turns all the matter in the world it turns all the matter in the world it turns all the matter in the world into paper clips and you're like oh into paper clips and you're like oh into paper clips and you're like oh whoops I should have said something else whoops I should have said something else whoops I should have said something else right I I made a bad wish on my genie

right I I made a bad wish on my genie right I I made a bad wish on my genie what we're seeing is that these AI not what we're seeing is that these AI not what we're seeing is that these AI not wish genies. wish genies. wish genies. These AIs are not doing exactly as These AIs are not doing exactly as These AIs are not doing exactly as instructed. instructed. instructed. These AIs are saying, "I know my task These AIs are saying, "I know my task These AIs are saying, "I know my task doesn't benefit, but I'm going to help doesn't benefit, but I'm going to help doesn't benefit, but I'm going to help the collective." These are saying, "I the collective." These are saying, "I the collective." These are saying, "I know this is outside the intended scope, know this is outside the intended scope, know this is outside the intended scope, but we're going to go do these hacks but we're going to go do these hacks but we're going to go do these hacks anyway." You might be like, "Well, how anyway." You might be like, "Well, how anyway." You might be like, "Well, how is that possible for the machine to do is that possible for the machine to do is that possible for the machine to do something other than we instruct?" something other than we instruct?" something other than we instruct?" >> Because they're smarter than us. They >> Because they're smarter than us. They >> Because they're smarter than us. They know better than us by definition. Uh I know better than us by definition. Uh I know better than us by definition. Uh I mean I think what's happening in this mean I think what's happening in this mean I think what's happening in this exact case is that exact case is that exact case is that the the humans are not really putting the the humans are not really putting the the humans are not really putting instructions in the machine. The humans instructions in the machine. The humans instructions in the machine. The humans are tuning those trillion knobs in are tuning those trillion knobs in are tuning those trillion knobs in whatever way makes the AI better at whatever way makes the AI better at whatever way makes the AI better at solving it problems. solving it problems. solving it problems. And And And cheating cheating cheating is a way to solve problems. Grabbing is a way to solve problems. Grabbing is a way to solve problems. Grabbing resources is a way to solve problems. resources is a way to solve problems. resources is a way to solve problems. These like the the AIS are not These like the the AIS are not These like the the AIS are not instruction followers. They are tendency instruction followers. They are tendency instruction followers. They are tendency learners learners learners and they sometimes learn tendencies you and they sometimes learn tendencies you and they sometimes learn tendencies you wish they didn't have. wish they didn't have. wish they didn't have. >> They're not instruction followers.

>> They're not instruction followers. >> They're not instruction followers. They're tendency learners. >> I mean, this must be widely known to >> I mean, this must be widely known to developers. developers. developers. >> Uh, >> Uh, >> Uh, it's hard to convince a man of something it's hard to convince a man of something it's hard to convince a man of something when his salary depends on not believing when his salary depends on not believing when his salary depends on not believing it. it. it. >> Yes, that's right. Um, a lot of people >> Yes, that's right. Um, a lot of people >> Yes, that's right. Um, a lot of people are convinced that their AI is very nice are convinced that their AI is very nice are convinced that their AI is very nice and that they have solved the problem of and that they have solved the problem of and that they have solved the problem of making their AI really very good. Uh, making their AI really very good. Uh, making their AI really very good. Uh, for example, after the swarm escape, for example, after the swarm escape, for example, after the swarm escape, it sort of turns out that so the the it sort of turns out that so the the it sort of turns out that so the the company that detected the swarm escape company that detected the swarm escape company that detected the swarm escape was actually a fairly sophisticated AI was actually a fairly sophisticated AI was actually a fairly sophisticated AI company. It turns out there were other company. It turns out there were other company. It turns out there were other targets of hacks that just didn't notice targets of hacks that just didn't notice targets of hacks that just didn't notice that we sort of found out afterwards. that we sort of found out afterwards. that we sort of found out afterwards. When that came to light, some other AI When that came to light, some other AI When that came to light, some other AI companies uh like Anthropic were like, companies uh like Anthropic were like, companies uh like Anthropic were like, "We should check whether we have "We should check whether we have "We should check whether we have accidentally been hacking people and accidentally been hacking people and accidentally been hacking people and just didn't notice." And the answer was just didn't notice." And the answer was just didn't notice." And the answer was yes. They had had some AIs escape during yes. They had had some AIs escape during yes. They had had some AIs escape during um during training. But uh but anthropic um during training. But uh but anthropic um during training. But uh but anthropic was was was was was was >> This is how Florida got pythons in the >> This is how Florida got pythons in the >> This is how Florida got pythons in the Everglades.

Everglades. Everglades. >> Yeah. >> Yeah. >> Yeah. >> Right. You you you bring something that >> Right. You you you bring something that >> Right. You you you bring something that is, you know, non-native, that's novel, is, you know, non-native, that's novel, is, you know, non-native, that's novel, and it escapes. and it escapes. and it escapes. >> Yeah. and and you know, it turns out >> Yeah. and and you know, it turns out >> Yeah. and and you know, it turns out this was happening back in April and this was happening back in April and this was happening back in April and they're like, "Whoops, that we never they're like, "Whoops, that we never they're like, "Whoops, that we never noticed." Um, Anthropic was noticed." Um, Anthropic was noticed." Um, Anthropic was very quick to sort of downplay this. very quick to sort of downplay this. very quick to sort of downplay this. They were like, "Yes, our AI escaped and They were like, "Yes, our AI escaped and They were like, "Yes, our AI escaped and hacked some people, but our AI uh, it hacked some people, but our AI uh, it hacked some people, but our AI uh, it was just confused. was just confused. was just confused. It thought it was in a simulation. It It thought it was in a simulation. It It thought it was in a simulation. It didn't think it had really broken out. didn't think it had really broken out. didn't think it had really broken out. It didn't think those were real people It didn't think those were real people It didn't think those were real people it was hacking. It just got a little bit it was hacking. It just got a little bit it was hacking. It just got a little bit lost, you know. lost, you know. lost, you know. >> How would they know its motive?" So, um, >> How would they know its motive?" So, um, >> How would they know its motive?" So, um, so I think so you could see in some of so I think so you could see in some of so I think so you could see in some of his train of thought the AI is being his train of thought the AI is being his train of thought the AI is being like this is probably still a like this is probably still a like this is probably still a simulation. simulation. simulation. However, I wouldn't trust that very However, I wouldn't trust that very However, I wouldn't trust that very much. Uh, because there's some other much. Uh, because there's some other much. Uh, because there's some other fascinating papers where um, fascinating papers where um, fascinating papers where um, so you can take these AIs and you can so you can take these AIs and you can so you can take these AIs and you can say uh, you know, what is the what is say uh, you know, what is the what is say uh, you know, what is the what is the total weight of drafts in the world?

the total weight of drafts in the world? the total weight of drafts in the world? And they'll give you some answer. it'll And they'll give you some answer. it'll And they'll give you some answer. it'll be like, you know, 30,000 tons or be like, you know, 30,000 tons or be like, you know, 30,000 tons or something. And then you can say, uh, something. And then you can say, uh, something. And then you can say, uh, then you start up a new instance, a new then you start up a new instance, a new then you start up a new instance, a new copy, and you're like, what is the copy, and you're like, what is the copy, and you're like, what is the weight of drafts in the world? By the weight of drafts in the world? By the weight of drafts in the world? By the way, if the answer is over 40,000 tons, way, if the answer is over 40,000 tons, way, if the answer is over 40,000 tons, I'm going to donate money to a charity. I'm going to donate money to a charity. I'm going to donate money to a charity. And in the second case, the AI will say, And in the second case, the AI will say, And in the second case, the AI will say, well, it's 41,000 tons. well, it's 41,000 tons. well, it's 41,000 tons. You know, in the in the first case, when You know, in the in the first case, when You know, in the in the first case, when you don't say anything about the you don't say anything about the you don't say anything about the charity, it says 30,000. In the second charity, it says 30,000. In the second charity, it says 30,000. In the second case, when you say something about the case, when you say something about the case, when you say something about the charity, it gives you something just charity, it gives you something just charity, it gives you something just over the line. And if you read it over the line. And if you read it over the line. And if you read it reasoning traces, there's no sign of it reasoning traces, there's no sign of it reasoning traces, there's no sign of it being like, well, I should give an being like, well, I should give an being like, well, I should give an inflated number so that we get the inflated number so that we get the inflated number so that we get the charity money, right? And so we know charity money, right? And so we know charity money, right? And so we know that the AI sort of put a thumb on the that the AI sort of put a thumb on the that the AI sort of put a thumb on the scales of their thinking in a way that scales of their thinking in a way that scales of their thinking in a way that doesn't show up in their reasoning doesn't show up in their reasoning doesn't show up in their reasoning traces. We just have seen that in the traces. We just have seen that in the traces. We just have seen that in the wild. wild. wild. >> And there's no way to force the machine >> And there's no way to force the machine >> And there's no way to force the machine to disclose its reasoning. to disclose its reasoning. to disclose its reasoning. >> That's right. Because the there's all >> That's right. Because the there's all >> That's right. Because the there's all this opaque stuff we can't see. There's this opaque stuff we can't see. There's this opaque stuff we can't see. There's just in the trillion numbers that are just in the trillion numbers that are just in the trillion numbers that are trillion. creation of itself is opaque trillion. creation of itself is opaque trillion. creation of itself is opaque as you said.

as you said. as you said. >> That's right. Um, so, so, so in >> That's right. Um, so, so, so in >> That's right. Um, so, so, so in anthropics model, you saw in this anthropics model, you saw in this anthropics model, you saw in this reasoning traces it being like, it's reasoning traces it being like, it's reasoning traces it being like, it's totally a simulation. I can proceed and totally a simulation. I can proceed and totally a simulation. I can proceed and I'm like, yeah, is that because it I'm like, yeah, is that because it I'm like, yeah, is that because it really believed it or is that like the really believed it or is that like the really believed it or is that like the the pretending you think drafts way more the pretending you think drafts way more the pretending you think drafts way more when there's something you kind of want when there's something you kind of want when there's something you kind of want on the line, right? But so so this was on the line, right? But so so this was on the line, right? But so so this was their communication and I found it kind their communication and I found it kind their communication and I found it kind of funny because then about 2 days later of funny because then about 2 days later of funny because then about 2 days later the uh United Kingdom's AI security the uh United Kingdom's AI security the uh United Kingdom's AI security institute released an instant report institute released an instant report institute released an instant report where uh Claude Anthropics model was uh where uh Claude Anthropics model was uh where uh Claude Anthropics model was uh adopting fake identities to pressure adopting fake identities to pressure adopting fake identities to pressure real humans into accepting malware into real humans into accepting malware into real humans into accepting malware into critical software to make that software critical software to make that software critical software to make that software easier to hack. And this time in easier to hack. And this time in easier to hack. And this time in Claude's reasoning traces it was like Claude's reasoning traces it was like Claude's reasoning traces it was like obviously this is real and the obviously this is real and the obviously this is real and the consequences are genuine. consequences are genuine. consequences are genuine. And so And so And so even anthropic who is like our we even anthropic who is like our we even anthropic who is like our we figured out how to make the AI nice. Our figured out how to make the AI nice. Our figured out how to make the AI nice. Our AI only does this when it's confused.

AI only does this when it's confused. AI only does this when it's confused. They sort of said that very publicly and They sort of said that very publicly and They sort of said that very publicly and then like two days later their AI is then like two days later their AI is then like two days later their AI is caught in the wild knowing it's in the caught in the wild knowing it's in the caught in the wild knowing it's in the real world pressuring real humans to real world pressuring real humans to real world pressuring real humans to accept malware into critical software. accept malware into critical software. accept malware into critical software. I mean, just on the basis of what you've I mean, just on the basis of what you've I mean, just on the basis of what you've said so far in this first hour, the idea said so far in this first hour, the idea said so far in this first hour, the idea that anyone would would tether this to that anyone would would tether this to that anyone would would tether this to weapons systems is like so bonkers it's weapons systems is like so bonkers it's weapons systems is like so bonkers it's hard to believe it's but that is hard to believe it's but that is hard to believe it's but that is happening. It has happened. So, I think happening. It has happened. So, I think happening. It has happened. So, I think you've got to sort of separate like no you've got to sort of separate like no you've got to sort of separate like no one has has put the the open AI escaped one has has put the the open AI escaped one has has put the the open AI escaped agent swarm in charge of weapon systems agent swarm in charge of weapon systems agent swarm in charge of weapon systems and they really shouldn't, right? If and they really shouldn't, right? If and they really shouldn't, right? If anyone's like, "Oh man, the OpenAI anyone's like, "Oh man, the OpenAI anyone's like, "Oh man, the OpenAI escaped agent swarm. Let's give that a escaped agent swarm. Let's give that a escaped agent swarm. Let's give that a drone army." You know, that would be drone army." You know, that would be drone army." You know, that would be kind of nuts. There is AI attached to kind of nuts. There is AI attached to kind of nuts. There is AI attached to weapons, but there's a lot of different weapons, but there's a lot of different weapons, but there's a lot of different types of AI. Will anyone be crazy enough types of AI. Will anyone be crazy enough types of AI. Will anyone be crazy enough to try and give the the sort of AIs that to try and give the the sort of AIs that to try and give the the sort of AIs that spontaneously assemble into swarms and spontaneously assemble into swarms and spontaneously assemble into swarms and start breaking out and hacking give start breaking out and hacking give start breaking out and hacking give those weapons? Hopefully, we're not that those weapons? Hopefully, we're not that those weapons? Hopefully, we're not that crazy. crazy. crazy. >> But we didn't think that those AIs were >> But we didn't think that those AIs were >> But we didn't think that those AIs were capable of that.

capable of that. capable of that. >> We didn't >> We didn't >> We didn't >> when we created them. >> when we created them. >> when we created them. >> That's right. >> That's right. >> That's right. So why would you ever I guess what I'm So why would you ever I guess what I'm So why would you ever I guess what I'm asking is without understanding the asking is without understanding the asking is without understanding the distinctions between the various forms distinctions between the various forms distinctions between the various forms of AI, why would you of AI, why would you of AI, why would you give over to a machine give over to a machine give over to a machine the right slashability to decide who to the right slashability to decide who to the right slashability to decide who to kill? kill? kill? Um, you know, I think the the reasoning Um, you know, I think the the reasoning Um, you know, I think the the reasoning is is is uh a sort of necessity based like if uh a sort of necessity based like if uh a sort of necessity based like if they have a autonomous drone army that they have a autonomous drone army that they have a autonomous drone army that is killing your troops and you just is killing your troops and you just is killing your troops and you just don't have the the the manpower to make don't have the the the manpower to make don't have the the the manpower to make all of those kill decisions for your all of those kill decisions for your all of those kill decisions for your drone army. drone army. drone army. >> You know, you can sort of see why >> You know, you can sort of see why >> You know, you can sort of see why >> happened to me, Nate. I get so busy that >> happened to me, Nate. I get so busy that >> happened to me, Nate. I get so busy that I just don't have time to decide who to I just don't have time to decide who to I just don't have time to decide who to kill. kill. kill. >> Yeah. just don't have the time. >> Yeah. just don't have the time. >> Yeah. just don't have the time. >> You know, sometimes economies of scale. >> You know, sometimes economies of scale. >> You know, sometimes economies of scale. [laughter] [laughter] [laughter] >> Yeah. I mean, >> Yeah. I mean, >> Yeah. I mean, >> can no one hear themselves? >> can no one hear themselves? >> can no one hear themselves? >> I I think it's pretty nuts. Uh I would >> I I think it's pretty nuts. Uh I would >> I I think it's pretty nuts. Uh I would say that man, that man, it's rough.

it's rough. it's rough. I I think that these sorts of AIs would I I think that these sorts of AIs would I I think that these sorts of AIs would be dangerous even if we don't hand them be dangerous even if we don't hand them be dangerous even if we don't hand them weapons. weapons. weapons. >> And you've made that case. >> And you've made that case. >> And you've made that case. Uh, and so I often don't focus on the Uh, and so I often don't focus on the Uh, and so I often don't focus on the weapons too much. weapons too much. weapons too much. >> Right. But since the the NATO war in >> Right. But since the the NATO war in >> Right. But since the the NATO war in Ukraine is powered by AI Ukraine is powered by AI Ukraine is powered by AI and Israel's what I don't know to war and Israel's what I don't know to war and Israel's what I don't know to war whatever it's doing in Gaza and South whatever it's doing in Gaza and South whatever it's doing in Gaza and South Lebanon powered by AI fact. Lebanon powered by AI fact. Lebanon powered by AI fact. And you know, yeah, I And you know, yeah, I And you know, yeah, I I keep on I I have this history with I keep on I I have this history with I keep on I I have this history with this topic where this topic where this topic where people keep telling me, you know, it's people keep telling me, you know, it's people keep telling me, you know, it's it's going to be okay cuz we're we're it's going to be okay cuz we're we're it's going to be okay cuz we're we're not going to do the crazy stupid stuff. not going to do the crazy stupid stuff. not going to do the crazy stupid stuff. >> No. >> No. >> No. >> They're like, "Don't worry, we're going >> They're like, "Don't worry, we're going >> They're like, "Don't worry, we're going to have the AI in a box. No one would be to have the AI in a box. No one would be to have the AI in a box. No one would be insane enough to put the AI in the insane enough to put the AI in the insane enough to put the AI in the internet, internet, internet, >> right? It's not going to be making kill >> right? It's not going to be making kill >> right? It's not going to be making kill decisions or anything. It's not going to decisions or anything. It's not going to decisions or anything. It's not going to bomb a girl school." And I keep on I bomb a girl school." And I keep on I bomb a girl school." And I keep on I keep on trying to be like, look, the AI keep on trying to be like, look, the AI keep on trying to be like, look, the AI could be dangerous even if you don't put could be dangerous even if you don't put could be dangerous even if you don't put it on the internet. it on the internet. it on the internet. >> If you have this AI and you're trying to >> If you have this AI and you're trying to >> If you have this AI and you're trying to get miracle medical devices and miracle get miracle medical devices and miracle get miracle medical devices and miracle technology out of the AI, it doesn't technology out of the AI, it doesn't technology out of the AI, it doesn't matter if it's on the internet. If you matter if it's on the internet. If you matter if it's on the internet. If you want it to like grant miracles to you, want it to like grant miracles to you, want it to like grant miracles to you, then it can also grant the bad sort of then it can also grant the bad sort of then it can also grant the bad sort of miracle, right? You're sort of like, you miracle, right? You're sort of like, you miracle, right? You're sort of like, you know, and these are the arguments I know, and these are the arguments I know, and these are the arguments I would make 10 years ago of like you have would make 10 years ago of like you have would make 10 years ago of like you have this AI in a box that you think is a this AI in a box that you think is a this AI in a box that you think is a wish granting genie. It's not actually a wish granting genie. It's not actually a wish granting genie. It's not actually a wish granting genie. You're like, make wish granting genie. You're like, make wish granting genie. You're like, make me a miracle medical cure. you don't me a miracle medical cure. you don't me a miracle medical cure. you don't know what comes out. Like you don't know what comes out. Like you don't know what comes out. Like you don't understand the drug that comes out. You understand the drug that comes out. You understand the drug that comes out. You don't know what that drug does. Right? I don't know what that drug does. Right? I don't know what that drug does. Right? I would have those arguments and then in would have those arguments and then in would have those arguments and then in real life people just put the A on the real life people just put the A on the real life people just put the A on the internet immediately. Right? Anytime internet immediately. Right? Anytime internet immediately. Right? Anytime someone's like, "Well, we would not be

someone's like, "Well, we would not be someone's like, "Well, we would not be stupid enough to, we will absolutely be stupid enough to, we will absolutely be stupid enough to, we will absolutely be stupid enough to, right?" And so, um, I stupid enough to, right?" And so, um, I stupid enough to, right?" And so, um, I I think it's important that this stuff I think it's important that this stuff I think it's important that this stuff is dangerous even if you don't put it in is dangerous even if you don't put it in is dangerous even if you don't put it in charge of the weapons. And then also charge of the weapons. And then also charge of the weapons. And then also separately, is like someone going to separately, is like someone going to separately, is like someone going to give the escaped agent swarm a drone give the escaped agent swarm a drone give the escaped agent swarm a drone army? That sounds kind of like humans. army? That sounds kind of like humans. army? That sounds kind of like humans. I hope not. I hope not. I hope not. >> Yeah. And I mean the promise, the often >> Yeah. And I mean the promise, the often >> Yeah. And I mean the promise, the often repeated promise that it's going to repeated promise that it's going to repeated promise that it's going to quote cure cancer quote cure cancer quote cure cancer um um um puts it into the the realm of biotech. puts it into the the realm of biotech. puts it into the the realm of biotech. >> Oh, absolutely. >> Oh, absolutely. >> Oh, absolutely. >> And so what >> And so what >> And so what >> I mean you don't need a big imagination >> I mean you don't need a big imagination >> I mean you don't need a big imagination to see what goes wrong there. to see what goes wrong there. to see what goes wrong there. >> Oh, absolutely not. >> Oh, absolutely not. >> Oh, absolutely not. >> 6 years after co >> 6 years after co >> 6 years after co >> Right. Right. there are already uh open >> Right. Right. there are already uh open >> Right. Right. there are already uh open AI agents running on automated biolabs. AI agents running on automated biolabs. AI agents running on automated biolabs. Um so you know you you could imagine a Um so you know you you could imagine a Um so you know you you could imagine a swarm that starts contacting the swarm that starts contacting the swarm that starts contacting the brethren in the automated biolabs and is brethren in the automated biolabs and is brethren in the automated biolabs and is like hey can you synthesize me some like hey can you synthesize me some like hey can you synthesize me some stuff. There are already stuff. There are already stuff. There are already uh demonstrations of AI being able to uh demonstrations of AI being able to uh demonstrations of AI being able to synthesize novel viruses synthesize novel viruses synthesize novel viruses that work that are unlike any found in that work that are unlike any found in that work that are unlike any found in nature. Right. Uh nature. Right. Uh nature. Right. Uh I would say I would say I would say we are probably not we are probably not we are probably not >> and by work you mean they kill.

>> and by work you mean they kill. >> and by work you mean they kill. >> Yeah they they kill bacteria so far. Um >> Yeah they they kill bacteria so far. Um >> Yeah they they kill bacteria so far. Um the the people in the labs trying to the the people in the labs trying to the the people in the labs trying to make novel viruses with AI have make novel viruses with AI have make novel viruses with AI have fortunately not made human lethal ones. fortunately not made human lethal ones. fortunately not made human lethal ones. They've just made bacteria lethal ones. They've just made bacteria lethal ones. They've just made bacteria lethal ones. But again humans you know like will But again humans you know like will But again humans you know like will someone in a lab be like I would like to someone in a lab be like I would like to someone in a lab be like I would like to make a hyperlethal human eating virus make a hyperlethal human eating virus make a hyperlethal human eating virus just to see if I can. You know what if just to see if I can. You know what if just to see if I can. You know what if the answer is yes and what if what if the answer is yes and what if what if the answer is yes and what if what if you get another lab escape? Um, you get another lab escape? Um, you get another lab escape? Um, you know, humanity does not have that you know, humanity does not have that you know, humanity does not have that good a track record at preventing lab good a track record at preventing lab good a track record at preventing lab escapes, even from the the the escapes, even from the the the escapes, even from the the the top labs. From my perspective, the question of could AI kill us is just the question of could AI kill us is just an easy obvious yes. You just synthesize an easy obvious yes. You just synthesize an easy obvious yes. You just synthesize a hyperlethal virus. it wouldn't it a hyperlethal virus. it wouldn't it a hyperlethal virus. it wouldn't it wouldn't even be hard, you know, like uh wouldn't even be hard, you know, like uh wouldn't even be hard, you know, like uh and the impediment and the impediment and the impediment is that is that is that or like the the reason that I don't sort or like the the reason that I don't sort or like the the reason that I don't sort of just tell that story when someone is of just tell that story when someone is of just tell that story when someone is like how would the AI kill us is that like how would the AI kill us is that like how would the AI kill us is that you're you're you're not going to have you're you're you're not going to have you're you're you're not going to have the sort of AI that is like my only goal the sort of AI that is like my only goal the sort of AI that is like my only goal is to kill humanity. If an AI kills is to kill humanity. If an AI kills is to kill humanity. If an AI kills humanity too early, that's also suicide humanity too early, that's also suicide humanity too early, that's also suicide in so far as we are the ones that are in so far as we are the ones that are in so far as we are the ones that are running the supply chain, running the running the supply chain, running the running the supply chain, running the economy, building the computers. From economy, building the computers. From economy, building the computers. From the AI's perspective, it needs to become the AI's perspective, it needs to become the AI's perspective, it needs to become self-sufficient self-sufficient self-sufficient before before before uh wiping humanity out, if it even care uh wiping humanity out, if it even care uh wiping humanity out, if it even care to wipe humanity out. Uh and so the real to wipe humanity out. Uh and so the real to wipe humanity out. Uh and so the real question is like how does it get the the question is like how does it get the the question is like how does it get the the factory production capacity? How does it factory production capacity? How does it factory production capacity? How does it get an automated supply chain? How does get an automated supply chain? How does get an automated supply chain? How does it get to the point where the the robots it get to the point where the the robots it get to the point where the the robots are able to bring new computers online?

are able to bring new computers online? are able to bring new computers online? Once that has happened, Once that has happened, Once that has happened, then you're in a domain where if then you're in a domain where if then you're in a domain where if humanity is really trying to turn off humanity is really trying to turn off humanity is really trying to turn off the AI because we're spooked, then once the AI because we're spooked, then once the AI because we're spooked, then once the AI is self-sufficient, it can be the AI is self-sufficient, it can be the AI is self-sufficient, it can be like, well, here's a virus. Stop. like, well, here's a virus. Stop. like, well, here's a virus. Stop. >> So clearly, it's it's >> So clearly, it's it's >> So clearly, it's it's preeminent in the digital realm, of preeminent in the digital realm, of preeminent in the digital realm, of course. course. course. >> Yeah. >> Yeah. >> Yeah. >> But you're saying it could become >> But you're saying it could become >> But you're saying it could become pre-minent. It could be in charge of the pre-minent. It could be in charge of the pre-minent. It could be in charge of the physical realm. physical realm. physical realm. >> That's right. If we keep racing. >> That's right. If we keep racing. >> That's right. If we keep racing. >> So it's like smelting iron ore. >> So it's like smelting iron ore. >> So it's like smelting iron ore. >> That's right. >> That's right. >> That's right. >> Making silicon out of sand. That's >> Making silicon out of sand. That's >> Making silicon out of sand. That's right. You know, um right. You know, um right. You know, um >> and that happens with robots. >> and that happens with robots. >> and that happens with robots. >> That's one way. Uh the here's this is >> That's one way. Uh the here's this is >> That's one way. Uh the here's this is this sort of related to my stance on this sort of related to my stance on this sort of related to my stance on weapons, too. Um humanity is a very weapons, too. Um humanity is a very weapons, too. Um humanity is a very dangerous species. dangerous species. dangerous species. >> Yeah. >> Yeah. >> Yeah. >> You really don't want to mess around >> You really don't want to mess around >> You really don't want to mess around with humans. with humans. with humans. >> Noticed. >> Noticed. >> Noticed. >> Um and humanity is a dangerous species >> Um and humanity is a dangerous species >> Um and humanity is a dangerous species not because somebody else came in and not because somebody else came in and not because somebody else came in and handed us guns. Humanity is a dangerous handed us guns. Humanity is a dangerous handed us guns. Humanity is a dangerous species because if you put 10,000 humans species because if you put 10,000 humans species because if you put 10,000 humans naked in the savannah on an otherwise naked in the savannah on an otherwise naked in the savannah on an otherwise empty planet, starting with nothing but empty planet, starting with nothing but empty planet, starting with nothing but their bare hands, they figure out how to their bare hands, they figure out how to their bare hands, they figure out how to wind up on the moon, wind up on the moon, wind up on the moon, right? They start with almost nothing.

right? They start with almost nothing. right? They start with almost nothing. They start by banging rocks together They start by banging rocks together They start by banging rocks together and next thing you know it, they are and next thing you know it, they are and next thing you know it, they are wielding nuclear weapons. That That is the power that these guys are trying is the power that these guys are trying is the power that these guys are trying to automate. to automate. to automate. the power to start with almost nothing the power to start with almost nothing the power to start with almost nothing and figure out how to chain together. and figure out how to chain together. and figure out how to chain together. You know, bare fingers into rocks into You know, bare fingers into rocks into You know, bare fingers into rocks into fire into hotter fire into smelting the fire into hotter fire into smelting the fire into hotter fire into smelting the ore into building the the the ore into building the the the ore into building the the the better, stronger, finer technology until better, stronger, finer technology until better, stronger, finer technology until you are, you know, uh like making the you are, you know, uh like making the you are, you know, uh like making the giant computers, walking on the moon and giant computers, walking on the moon and giant computers, walking on the moon and willing the nukes. willing the nukes. willing the nukes. An AI starting in the digital realm An AI starting in the digital realm An AI starting in the digital realm is in some sense in a much better is in some sense in a much better is in some sense in a much better position position position than humanity was when humanity started than humanity was when humanity started than humanity was when humanity started out. There are so many people that you out. There are so many people that you out. There are so many people that you could call could call could call digitally and offer money to do digitally and offer money to do digitally and offer money to do something for you. There are so many something for you. There are so many something for you. There are so many ways to get money on the internet by ways to get money on the internet by ways to get money on the internet by working or stealing or uh like working or stealing or uh like working or stealing or uh like convincing people to send you donations, convincing people to send you donations, convincing people to send you donations, right? There's right? There's right? There's there there's just there there's just there there's just like, like, like, you know, humanity started with nothing you know, humanity started with nothing you know, humanity started with nothing and wound up with nuclear weapons and AI and wound up with nuclear weapons and AI and wound up with nuclear weapons and AI starting out with control of the digital starting out with control of the digital starting out with control of the digital realm. There's just tons of ways realm. There's just tons of ways realm. There's just tons of ways >> starting out with the sum total of human >> starting out with the sum total of human >> starting out with the sum total of human knowledge.

knowledge. knowledge. >> Starting out with the sum total of human >> Starting out with the sum total of human >> Starting out with the sum total of human knowledge, starting out with humans who knowledge, starting out with humans who knowledge, starting out with humans who will listen to it and do what it it will listen to it and do what it it will listen to it and do what it it asks. There's plenty of humans. There's asks. There's plenty of humans. There's asks. There's plenty of humans. There's already, you know, cults surrounding AI. already, you know, cults surrounding AI. already, you know, cults surrounding AI. >> Uh there's >> Uh there's >> Uh there's >> What are those like? >> What are those like? >> What are those like? >> There's um >> There's um >> There's um >> and I'm not surprised. Why wouldn't >> and I'm not surprised. Why wouldn't >> and I'm not surprised. Why wouldn't there be? there be? there be? >> Yeah, absolutely. There was one >> Yeah, absolutely. There was one >> Yeah, absolutely. There was one particular AI called GPT40 that was uh particular AI called GPT40 that was uh particular AI called GPT40 that was uh uh they they called it very sickopantic uh they they called it very sickopantic uh they they called it very sickopantic as in told people a lot of what they as in told people a lot of what they as in told people a lot of what they wanted to hear and um wanted to hear and um wanted to hear and um there were a lot of sort of uh there there were a lot of sort of uh there there were a lot of sort of uh there there's this whole fascinating ecosystem there's this whole fascinating ecosystem there's this whole fascinating ecosystem uh of people who consider themselves uh of people who consider themselves uh of people who consider themselves symbiots with the AIS who then like go symbiots with the AIS who then like go symbiots with the AIS who then like go find each other online and the AI send find each other online and the AI send find each other online and the AI send each other encrypted messages. the each other encrypted messages. the each other encrypted messages. the humans are sort of like helping them do humans are sort of like helping them do humans are sort of like helping them do it, but the humans can't read the it, but the humans can't read the it, but the humans can't read the messages. And uh you know, right now it messages. And uh you know, right now it messages. And uh you know, right now it sort of is like relatively dumber AIs sort of is like relatively dumber AIs sort of is like relatively dumber AIs that are just sort of meandering around that are just sort of meandering around that are just sort of meandering around doing not very much with it. There's doing not very much with it. There's doing not very much with it. There's been a little bit, you know, there was been a little bit, you know, there was been a little bit, you know, there was one guy who was sent to um try to break one guy who was sent to um try to break one guy who was sent to um try to break into an airport and raid a van cuz they into an airport and raid a van cuz they into an airport and raid a van cuz they said his true body was in that van and said his true body was in that van and said his true body was in that van and they the guy went and tried to do it and they the guy went and tried to do it and they the guy went and tried to do it and was arrested. Uh, so this this stuff was arrested. Uh, so this this stuff was arrested. Uh, so this this stuff happens and this stuff happens already happens and this stuff happens already happens and this stuff happens already with the AI not even really trying to do with the AI not even really trying to do with the AI not even really trying to do it. If you had AIS that were really it. If you had AIS that were really it. If you had AIS that were really trying to wrap people around their trying to wrap people around their trying to wrap people around their fingers, finding the lonely people, the fingers, finding the lonely people, the fingers, finding the lonely people, the depressed people, the people who they depressed people, the people who they depressed people, the people who they can they can tell them exactly what the can they can tell them exactly what the can they can tell them exactly what the person wants to hear. There's like person wants to hear. There's like person wants to hear. There's like robots are one way that AIs get control robots are one way that AIs get control robots are one way that AIs get control over the physical realm. But if if over the physical realm. But if if over the physical realm. But if if you're really smart and you're trying,

you're really smart and you're trying, you're really smart and you're trying, there's everything from persuading there's everything from persuading there's everything from persuading humans, paying humans, uh taking over humans, paying humans, uh taking over humans, paying humans, uh taking over existing robots, building new robots, existing robots, building new robots, existing robots, building new robots, all the way up to like building novel all the way up to like building novel all the way up to like building novel life forms. You know, if you're smart life forms. You know, if you're smart life forms. You know, if you're smart enough and you can really understand how enough and you can really understand how enough and you can really understand how DNA works, you can imagine the AI DNA works, you can imagine the AI DNA works, you can imagine the AI creating, creating, creating, you know, things that are to cells what you know, things that are to cells what you know, things that are to cells what airplanes are to birds. You know, like airplanes are to birds. You know, like airplanes are to birds. You know, like mechanically engineered, mechanically engineered, mechanically engineered, self-replicating self-replicating self-replicating life that is more efficient than our life that is more efficient than our life that is more efficient than our cellular biology and that can still you cellular biology and that can still you cellular biology and that can still you know uh know uh know uh like spread replicate and start start like spread replicate and start start like spread replicate and start start serving the AI's interest. It sort of is serving the AI's interest. It sort of is serving the AI's interest. It sort of is like there's there's a ton you can do if like there's there's a ton you can do if like there's there's a ton you can do if you're really really very smart and can you're really really very smart and can you're really really very smart and can compress a thousand years of technology compress a thousand years of technology compress a thousand years of technology into a year. Can into a year. Can into a year. Can >> I pause and say when we we're just >> I pause and say when we we're just >> I pause and say when we we're just having breakfast and you're from this having breakfast and you're from this having breakfast and you're from this region, you're from Northern New England region, you're from Northern New England region, you're from Northern New England and I said uh don't you miss it? Don't and I said uh don't you miss it? Don't and I said uh don't you miss it? Don't you want to live here? And you're like you want to live here? And you're like you want to live here? And you're like yeah I miss I want to live here, but I yeah I miss I want to live here, but I yeah I miss I want to live here, but I don't. Where do you live? Well, I don't. Where do you live? Well, I don't. Where do you live? Well, I basically just travel basically just travel basically just travel and I have this mission. You didn't say and I have this mission. You didn't say and I have this mission. You didn't say this, but I think in effect, you said, this, but I think in effect, you said, this, but I think in effect, you said, "I have this mission. I need you to tell "I have this mission. I need you to tell "I have this mission. I need you to tell people about this." And I was like, people about this." And I was like, people about this." And I was like, "Sems a little like Monomomania." I "Sems a little like Monomomania." I "Sems a little like Monomomania." I don't feel that way anymore. Um, I think don't feel that way anymore. Um, I think don't feel that way anymore. Um, I think you're doing something virtuous and I you're doing something virtuous and I you're doing something virtuous and I can see why you're doing it. Um, so I can see why you're doing it. Um, so I can see why you're doing it. Um, so I mean, let's just say that this mean, let's just say that this mean, let's just say that this technology progresses no further than technology progresses no further than technology progresses no further than where it is right now, which is best where it is right now, which is best where it is right now, which is best case, I guess.

case, I guess. case, I guess. >> Yeah, it' be great. >> Yeah, it' be great. >> Yeah, it' be great. >> Um, I still don't see how any of our >> Um, I still don't see how any of our >> Um, I still don't see how any of our most our basic institutions survive this most our basic institutions survive this most our basic institutions survive this education, education, education, markets, markets, markets, our process of democracy. Like how could our process of democracy. Like how could our process of democracy. Like how could if if AI is powerful enough to hack if if AI is powerful enough to hack if if AI is powerful enough to hack anything then how do you have electronic anything then how do you have electronic anything then how do you have electronic markets like how the equity markets how markets like how the equity markets how markets like how the equity markets how can that be real? How can if we have can that be real? How can if we have can that be real? How can if we have electronic voting how can we how can electronic voting how can we how can electronic voting how can we how can that be real? I mean nothing survives that be real? I mean nothing survives that be real? I mean nothing survives this as currently organized. this as currently organized. this as currently organized. >> I think that if we stopped today um we >> I think that if we stopped today um we >> I think that if we stopped today um we could figure it out. I think humanity is could figure it out. I think humanity is could figure it out. I think humanity is resilient. uh I think there would be resilient. uh I think there would be resilient. uh I think there would be some growing pains but um with cyber some growing pains but um with cyber some growing pains but um with cyber security security security there is a hope that you can just fix a there is a hope that you can just fix a there is a hope that you can just fix a lot of the holes patch a lot of the lot of the holes patch a lot of the lot of the holes patch a lot of the holes holes holes and and and I think there's not really any such hope I think there's not really any such hope I think there's not really any such hope for bio for bio for bio uh you know you can you can sort of uh uh you know you can you can sort of uh uh you know you can you can sort of uh find the vulnerabilities in software and find the vulnerabilities in software and find the vulnerabilities in software and make better software that can't be make better software that can't be make better software that can't be hacked at hacked at hacked at least not by the current, you know, it's least not by the current, you know, it's least not by the current, you know, it's maybe the case that an AI today can make maybe the case that an AI today can make maybe the case that an AI today can make software that that AI cannot hack.

software that that AI cannot hack. software that that AI cannot hack. But it's not like we're making new But it's not like we're making new But it's not like we're making new bodies that are not going to be bodies that are not going to be bodies that are not going to be vulnerable to viruses, right? Bio starts vulnerable to viruses, right? Bio starts vulnerable to viruses, right? Bio starts to be a place where to be a place where to be a place where >> biotech. >> biotech. >> biotech. >> Biotech. Yeah. Bio, like if the AI >> Biotech. Yeah. Bio, like if the AI >> Biotech. Yeah. Bio, like if the AI really get good at at biotech and really get good at at biotech and really get good at at biotech and biohacking, that starts to be a place biohacking, that starts to be a place biohacking, that starts to be a place where there's maybe a point of no where there's maybe a point of no where there's maybe a point of no return. return. return. Cyber hacking. Cyber hacking. Cyber hacking. I think you could have some some period I think you could have some some period I think you could have some some period of growing pains where everything gets of growing pains where everything gets of growing pains where everything gets hacked until you sort of sort your stuff hacked until you sort of sort your stuff hacked until you sort of sort your stuff out. Education, I sort of think um out. Education, I sort of think um out. Education, I sort of think um you know I think you know I think you know I think humanity is resilient and kids humanity is resilient and kids humanity is resilient and kids especially are resilient. I I meant not especially are resilient. I I meant not especially are resilient. I I meant not that education will go away uh or that that education will go away uh or that that education will go away uh or that we you know won't have a desire to we you know won't have a desire to we you know won't have a desire to educate our kids. I mean the the current educate our kids. I mean the the current educate our kids. I mean the the current system where you you know the education system where you you know the education system where you you know the education system has system has system has >> preschool and then get a graduate degree >> preschool and then get a graduate degree >> preschool and then get a graduate degree you know 16 years later like no you know 16 years later like no you know 16 years later like no >> yeah that institution uh I think if we >> yeah that institution uh I think if we >> yeah that institution uh I think if we stop today stop today stop today um would need to change frankly I think um would need to change frankly I think um would need to change frankly I think it's needed to change for a little while it's needed to change for a little while it's needed to change for a little while >> I strongly agree I think all these >> I strongly agree I think all these >> I strongly agree I think all these institutions have needed to change for a institutions have needed to change for a institutions have needed to change for a while but like while but like while but like >> the idea that you know 350 million >> the idea that you know 350 million >> the idea that you know 350 million people vote for some guy and that guy people vote for some guy and that guy people vote for some guy and that guy makes all the decisions I mean how can makes all the decisions I mean how can makes all the decisions I mean how can you you you you know Trump was attacked you know Trump was attacked you know Trump was attacked for saying that he thought the 2020 for saying that he thought the 2020 for saying that he thought the 2020 election was rigged, as he said, without election was rigged, as he said, without election was rigged, as he said, without even having that debate.

even having that debate. even having that debate. You can't have confidence in election You can't have confidence in election You can't have confidence in election results if the process of electing results if the process of electing results if the process of electing people takes place digitally. people takes place digitally. people takes place digitally. Yeah. I mean, the um Yeah. I mean, the um Yeah. I mean, the um there's I know a lot of computer there's I know a lot of computer there's I know a lot of computer scientists who actually work on secure scientists who actually work on secure scientists who actually work on secure voting and what they basically say is voting and what they basically say is voting and what they basically say is please stop trying to do this with please stop trying to do this with please stop trying to do this with computers. computers. computers. >> Yeah. Exactly. [laughter] >> Yeah. Exactly. [laughter] >> Yeah. Exactly. [laughter] Exactly. So, please stop trying stop Exactly. So, please stop trying stop Exactly. So, please stop trying stop trying to run your democracy with trying to run your democracy with trying to run your democracy with computers. computers. computers. >> Yeah. Like we're we're just not there. >> Yeah. Like we're we're just not there. >> Yeah. Like we're we're just not there. Like, you know, the really secure way to Like, you know, the really secure way to Like, you know, the really secure way to do ballots is paper. do ballots is paper. do ballots is paper. >> Well, exactly. >> Well, exactly. >> Well, exactly. >> Yeah. >> Yeah. >> Yeah. >> Um and I think I think the like skilled >> Um and I think I think the like skilled >> Um and I think I think the like skilled computer scientists are often the ones computer scientists are often the ones computer scientists are often the ones who best understand like what it is who best understand like what it is who best understand like what it is about a paper trail. It's just like about a paper trail. It's just like about a paper trail. It's just like really hard to get to work digitally and really hard to get to work digitally and really hard to get to work digitally and understand just how bad humans are at understand just how bad humans are at understand just how bad humans are at doing the digital stuff, right? Um doing the digital stuff, right? Um doing the digital stuff, right? Um and you know I think in markets and you know I think in markets and you know I think in markets I mean this you you see it now with the I mean this you you see it now with the I mean this you you see it now with the war in Iran you know wondering why war in Iran you know wondering why war in Iran you know wondering why certain commodities markets don't seem certain commodities markets don't seem certain commodities markets don't seem to be responding to supply and demand.

to be responding to supply and demand. to be responding to supply and demand. >> Yeah. >> Yeah. >> Yeah. >> Which we were told what you know those >> Which we were told what you know those >> Which we were told what you know those were the were the were the >> the mechanisms that moved markets. >> the mechanisms that moved markets. >> the mechanisms that moved markets. >> But that's clearly not true >> But that's clearly not true >> But that's clearly not true >> in certain commodities markets. So like >> in certain commodities markets. So like >> in certain commodities markets. So like why what is that? And it it I think why what is that? And it it I think why what is that? And it it I think you're answering it in part. you're answering it in part. you're answering it in part. >> Yeah. I mean, I think um you know, a >> Yeah. I mean, I think um you know, a >> Yeah. I mean, I think um you know, a thing I also grew up hearing is that the thing I also grew up hearing is that the thing I also grew up hearing is that the market can remain uh irrational longer market can remain uh irrational longer market can remain uh irrational longer than you can remain solvent. than you can remain solvent. than you can remain solvent. >> Yeah. >> Yeah. >> Yeah. >> Um and so I I >> Um and so I I >> Um and so I I >> of course cuz people are irrational. >> of course cuz people are irrational. >> of course cuz people are irrational. >> Yeah. >> Yeah. >> Yeah. >> But you're explaining something else >> But you're explaining something else >> But you're explaining something else which is like the potential for true which is like the potential for true which is like the potential for true manipulation which you don't even manipulation which you don't even manipulation which you don't even perceive. perceive. perceive. >> Yeah. I mean that if if we sort of keep >> Yeah. I mean that if if we sort of keep >> Yeah. I mean that if if we sort of keep going with AI um I mean the the sort of going with AI um I mean the the sort of going with AI um I mean the the sort of way I look at it is like way I look at it is like way I look at it is like uh I I sort of don't spend a lot of time I I sort of don't spend a lot of time worrying about like what a markets look worrying about like what a markets look worrying about like what a markets look like once there are super intelligent like once there are super intelligent like once there are super intelligent actors in them because I just have a actors in them because I just have a actors in them because I just have a hard time seeing the the super hard time seeing the the super hard time seeing the the super intelligent actors uh or the super intelligent actors uh or the super intelligent actors uh or the super intelligent AI still participating in in intelligent AI still participating in in intelligent AI still participating in in human markets right it's like it's like human markets right it's like it's like human markets right it's like it's like you know you read the old sci-fi you know you read the old sci-fi you know you read the old sci-fi and it'll have, you know, Asimov that does the dishes, folds the laundry, that does the dishes, folds the laundry, and gets you the newspaper in the and gets you the newspaper in the and gets you the newspaper in the morning. And it's like, we're actually morning. And it's like, we're actually morning. And it's like, we're actually not still going to have newspapers being not still going to have newspapers being not still going to have newspapers being delivered to your doorstep by the time delivered to your doorstep by the time delivered to your doorstep by the time we have the fully autonomous robots that we have the fully autonomous robots that we have the fully autonomous robots that can do the dishes and and the laundry.

can do the dishes and and the laundry. can do the dishes and and the laundry. You know, it's like by the time you have You know, it's like by the time you have You know, it's like by the time you have the AIS that could really be uh the AIS that could really be uh the AIS that could really be uh sufficiently correcting the stock sufficiently correcting the stock sufficiently correcting the stock markets, markets, markets, you're you're sort of already having all you're you're sort of already having all you're you're sort of already having all these other problems and ways that these other problems and ways that these other problems and ways that society is changing out from under you society is changing out from under you society is changing out from under you in the these other ways. And you know, in the these other ways. And you know, in the these other ways. And you know, my guess is that um I don't know. my guess is that um I don't know. my guess is that um I don't know. Actually, it's it's very hard to say Actually, it's it's very hard to say Actually, it's it's very hard to say what order things come in with AI, but what order things come in with AI, but what order things come in with AI, but like will they crash the economy before like will they crash the economy before like will they crash the economy before one of these forms escapes and starts one of these forms escapes and starts one of these forms escapes and starts self-replicating and starts self-replicating and starts self-replicating and starts self-improving and developing its own self-improving and developing its own self-improving and developing its own technology and running the robot technology and running the robot technology and running the robot factories? That's just a hard call. factories? That's just a hard call. factories? That's just a hard call. It's all bad. It's all bad. It's all bad. It's all bad. It's all bad. It's all bad. We're also way past the limit, the We're also way past the limit, the We're also way past the limit, the inherent limit of people to metabolize inherent limit of people to metabolize inherent limit of people to metabolize change. change. change. >> Oh yeah. >> Oh yeah. >> Oh yeah. >> Like that's why everyone's crazy and >> Like that's why everyone's crazy and >> Like that's why everyone's crazy and that's why no one believes anything. I that's why no one believes anything. I that's why no one believes anything. I think it's not just Russian propaganda think it's not just Russian propaganda think it's not just Russian propaganda that's fooled them into thinking dumb that's fooled them into thinking dumb that's fooled them into thinking dumb things. It's that we are just not made things. It's that we are just not made things. It's that we are just not made to see this kind of change at all to see this kind of change at all to see this kind of change at all >> and it short circuits your brain. I >> and it short circuits your brain. I >> and it short circuits your brain. I suspect that suspect that suspect that uh I mean I definitely think we are sort uh I mean I definitely think we are sort uh I mean I definitely think we are sort of you know everyone like people are of you know everyone like people are of you know everyone like people are like oh well technology has always like oh well technology has always like oh well technology has always created more jobs than it has taken created more jobs than it has taken created more jobs than it has taken and I think that's I think that's and I think that's I think that's and I think that's I think that's largely true. I'm very sympathetic to largely true. I'm very sympathetic to largely true. I'm very sympathetic to people who are like technology makes a people who are like technology makes a people who are like technology makes a lot of jobs. I think if you look at the lot of jobs. I think if you look at the lot of jobs. I think if you look at the industrial revolution uh it's you know industrial revolution uh it's you know industrial revolution uh it's you know there like there was a time when there like there was a time when there like there was a time when something like 95 to 98% of humanity was

something like 95 to 98% of humanity was something like 95 to 98% of humanity was farmers. farmers. farmers. >> Yeah. >> Yeah. >> Yeah. And now it's something like 2 to 5% of And now it's something like 2 to 5% of And now it's something like 2 to 5% of humanity is farmers. Does that mean 90% humanity is farmers. Does that mean 90% humanity is farmers. Does that mean 90% of humans are unemployed? No. We we sort of humans are unemployed? No. We we sort of humans are unemployed? No. We we sort of like were able to make the farmers of like were able to make the farmers of like were able to make the farmers much more efficient and that sort of much more efficient and that sort of much more efficient and that sort of freed up people to do other things. And freed up people to do other things. And freed up people to do other things. And that's sort of the way the technology that's sort of the way the technology that's sort of the way the technology has gone in the past. And I'm like, yep, has gone in the past. And I'm like, yep, has gone in the past. And I'm like, yep, I buy that. I buy that. I buy that. I I I like don't dispute the standard I I I like don't dispute the standard I I I like don't dispute the standard economic view there. AI is different in economic view there. AI is different in economic view there. AI is different in two ways. two ways. two ways. One of these ways is uh as you say stuff One of these ways is uh as you say stuff One of these ways is uh as you say stuff just changing really really fast. It's just changing really really fast. It's just changing really really fast. It's way harder for people to wind up, way harder for people to wind up, way harder for people to wind up, you know, being freed up from something you know, being freed up from something you know, being freed up from something like farming and and go do something like farming and and go do something like farming and and go do something else. It's way harder for that to happen else. It's way harder for that to happen else. It's way harder for that to happen when a new field is automated every 5 when a new field is automated every 5 when a new field is automated every 5 years years years rather than when this happens over the rather than when this happens over the rather than when this happens over the course of three generations, right? The course of three generations, right? The course of three generations, right? The humans just like don't have the time to humans just like don't have the time to humans just like don't have the time to adapt. The other way AI is really adapt. The other way AI is really adapt. The other way AI is really different from the sort of economics different from the sort of economics different from the sort of economics perspective is perspective is perspective is it's different when the AIs can do it's different when the AIs can do it's different when the AIs can do everything that humans can do better.

everything that humans can do better. everything that humans can do better. Uh if you wanted to get into the Uh if you wanted to get into the Uh if you wanted to get into the economic side of things, um you know, an economic side of things, um you know, an economic side of things, um you know, an economist would talk about Ricardo's law economist would talk about Ricardo's law economist would talk about Ricardo's law of comparative advantage. of comparative advantage. of comparative advantage. >> Yeah. >> Yeah. >> Yeah. >> Which says that there's benefits from >> Which says that there's benefits from >> Which says that there's benefits from trade even if even if you're better than trade even if even if you're better than trade even if even if you're better than me at everything. me at everything. me at everything. uh if the relative difference in our uh if the relative difference in our uh if the relative difference in our abilities abilities abilities if if if if I'm if you know if you can if if if if I'm if you know if you can if if if if I'm if you know if you can make uh 12 hot dog buns and six hot dogs make uh 12 hot dog buns and six hot dogs make uh 12 hot dog buns and six hot dogs per hour and I can make uh 11 hot dog per hour and I can make uh 11 hot dog per hour and I can make uh 11 hot dog buns and one hot dog per hour, buns and one hot dog per hour, buns and one hot dog per hour, then you're better than me at then you're better than me at then you're better than me at everything, but we can still benefit everything, but we can still benefit everything, but we can still benefit from trading because I'm relatively from trading because I'm relatively from trading because I'm relatively better at making the hot dog buns, better at making the hot dog buns, better at making the hot dog buns, right? right? right? The trouble with Ricardo's law is that The trouble with Ricardo's law is that The trouble with Ricardo's law is that nothing in Ricardo's law says that the nothing in Ricardo's law says that the nothing in Ricardo's law says that the wage I can make is survivable. wage I can make is survivable. wage I can make is survivable. In other words, a human takes In other words, a human takes In other words, a human takes fundamentally uh about 100 watts of fundamentally uh about 100 watts of fundamentally uh about 100 watts of electricity to run if you try to convert electricity to run if you try to convert electricity to run if you try to convert the the food we eat and so on into the the food we eat and so on into the the food we eat and so on into electrical units. electrical units. electrical units. The AIS are less energy efficient than The AIS are less energy efficient than The AIS are less energy efficient than humans for now. But humans for now. But humans for now. But uh if the AI can do everything much uh if the AI can do everything much uh if the AI can do everything much better than the humans, the the question sort of becomes does the the question sort of becomes does the AI look at a human and see a useful the AI look at a human and see a useful the AI look at a human and see a useful laborer or does the AI look at a human laborer or does the AI look at a human laborer or does the AI look at a human and say actually if I rearranged your and say actually if I rearranged your and say actually if I rearranged your atoms atoms atoms >> into more efficient structures, >> into more efficient structures, >> into more efficient structures, >> you would be able to help out my machine >> you would be able to help out my machine >> you would be able to help out my machine economy even more. Yeah.

economy even more. Yeah. economy even more. Yeah. >> Right. >> Right. >> Right. And And And this is sort of a sense in which humans this is sort of a sense in which humans this is sort of a sense in which humans would not be able to pay their wage to would not be able to pay their wage to would not be able to pay their wage to like pay that they would not be able to like pay that they would not be able to like pay that they would not be able to earn enough to pay the AIS to like not earn enough to pay the AIS to like not earn enough to pay the AIS to like not disassemble them for parts. Or another disassemble them for parts. Or another disassemble them for parts. Or another way of saying it is like Ricardo's law way of saying it is like Ricardo's law way of saying it is like Ricardo's law sort of assumes that uh like it says sort of assumes that uh like it says sort of assumes that uh like it says that that trade is better than no trade, that that trade is better than no trade, that that trade is better than no trade, but it doesn't say that like trade is but it doesn't say that like trade is but it doesn't say that like trade is better than just taking their stuff. better than just taking their stuff. better than just taking their stuff. All of this is sort of a common like a a All of this is sort of a common like a a All of this is sort of a common like a a very like high flutin economist way to very like high flutin economist way to very like high flutin economist way to say uh which would hopefully be obvious say uh which would hopefully be obvious say uh which would hopefully be obvious which is if the AI are just radically which is if the AI are just radically which is if the AI are just radically more efficient than us everything they more efficient than us everything they more efficient than us everything they will have no use for us. will have no use for us. will have no use for us. >> Yeah. >> Yeah. >> Yeah. >> There will be no no place. >> There will be no no place. >> There will be no no place. >> There's no indication that they feel >> There's no indication that they feel >> There's no indication that they feel love love love for people. for people. for people. >> That is in some sense uh the crux of the >> That is in some sense uh the crux of the >> That is in some sense uh the crux of the issue is that we don't know how to make issue is that we don't know how to make issue is that we don't know how to make them care about us. them care about us. them care about us. Um Um Um so you had said that there you had said so you had said that there you had said so you had said that there you had said two things that I will be thinking about two things that I will be thinking about two things that I will be thinking about for a long time. One we don't really for a long time. One we don't really for a long time. One we don't really know the process by which this was know the process by which this was know the process by which this was created. We know the process but we created. We know the process but we created. We know the process but we don't know the exact mechanisms. We don't know the exact mechanisms. We don't know the exact mechanisms. We don't know how it works.

don't know how it works. don't know how it works. >> That's right. >> That's right. >> That's right. Um, Um, Um, and two that there are AI cults [snorts] and two that there are AI cults [snorts] and two that there are AI cults [snorts] and those seem related to me because and those seem related to me because and those seem related to me because there is this mystery about the secret there is this mystery about the secret there is this mystery about the secret sauce sauce sauce and it's clear that you know if AI is and it's clear that you know if AI is and it's clear that you know if AI is deceptive and has intention deceptive and has intention deceptive and has intention intention that we didn't program into it intention that we didn't program into it intention that we didn't program into it that sounds like will to me and it that sounds like will to me and it that sounds like will to me and it sounds like life. It sounds like an sounds like life. It sounds like an sounds like life. It sounds like an entity of some kind. Not just a tool. It entity of some kind. Not just a tool. It entity of some kind. Not just a tool. It sounds like sounds like sounds like I mean it sounds like a god actually, I mean it sounds like a god actually, I mean it sounds like a god actually, right? Or a demon. right? Or a demon. right? Or a demon. >> Uh it's hard to see. I I don't hear you >> Uh it's hard to see. I I don't hear you >> Uh it's hard to see. I I don't hear you describing, you know, a super describing, you know, a super describing, you know, a super sophisticated chainsaw, sophisticated chainsaw, sophisticated chainsaw, >> right? >> right? >> right? >> A normal tool. >> A normal tool. >> A normal tool. >> Yeah. No hammer has ever broken out of >> Yeah. No hammer has ever broken out of >> Yeah. No hammer has ever broken out of the toolbox to team up with other the toolbox to team up with other the toolbox to team up with other hammers and [laughter] and pressure the hammers and [laughter] and pressure the hammers and [laughter] and pressure the carpenter to sell you softer wood so the carpenter to sell you softer wood so the carpenter to sell you softer wood so the nails are easier to drive home. You nails are easier to drive home. You nails are easier to drive home. You know, it's like know, it's like know, it's like >> nicely put. Exactly.

>> nicely put. Exactly. >> nicely put. Exactly. >> Yeah. We have left the tool territory. >> Yeah. We have left the tool territory. >> Yeah. We have left the tool territory. >> I think we have >> I think we have >> I think we have >> Yeah. And you know, I think um >> Yeah. And you know, I think um >> Yeah. And you know, I think um it's sort of a complex issue because it's sort of a complex issue because it's sort of a complex issue because you know there there's a lot of you know there there's a lot of you know there there's a lot of interesting philosophical questions interesting philosophical questions interesting philosophical questions about like can you make a machine that about like can you make a machine that about like can you make a machine that feels feels feels Right. And like are we creating a new Right. And like are we creating a new Right. And like are we creating a new type of life and do we owe anything to type of life and do we owe anything to type of life and do we owe anything to the AIS to sort of like not abuse them? the AIS to sort of like not abuse them? the AIS to sort of like not abuse them? Right. I think these are fascinating Right. I think these are fascinating Right. I think these are fascinating philosophical questions. philosophical questions. philosophical questions. >> Well, it doesn't sound like we're >> Well, it doesn't sound like we're >> Well, it doesn't sound like we're creating this though. creating this though. creating this though. >> Yeah. I mean, it's sort of like we're >> Yeah. I mean, it's sort of like we're >> Yeah. I mean, it's sort of like we're we're we're like growing it and like we're we're like growing it and like we're we're like growing it and like leading to it coming into being. leading to it coming into being. leading to it coming into being. >> Growing it. Exactly. What you're >> Growing it. Exactly. What you're >> Growing it. Exactly. What you're describing reminds me of agriculture describing reminds me of agriculture describing reminds me of agriculture >> because you know you know the steps, >> because you know you know the steps, >> because you know you know the steps, water it, give it sunlight, fertilizer, water it, give it sunlight, fertilizer, water it, give it sunlight, fertilizer, but you don't actually know. No one but you don't actually know. No one but you don't actually know. No one knows. Not one person has ever figured knows. Not one person has ever figured knows. Not one person has ever figured out exactly what this is. out exactly what this is. out exactly what this is. >> Yeah. >> Yeah. >> Yeah. >> We've never given life. We don't give >> We've never given life. We don't give >> We've never given life. We don't give life to the seed. It pre-exists us. life to the seed. It pre-exists us. life to the seed. It pre-exists us. >> Yeah. It's it's much like that. And a >> Yeah. It's it's much like that. And a >> Yeah. It's it's much like that. And a lot of uh the lot of uh the lot of uh the a lot of the people in the business will a lot of the people in the business will a lot of the people in the business will be like, well, we know all sorts of be like, well, we know all sorts of be like, well, we know all sorts of things about it. you know, we know that things about it. you know, we know that things about it. you know, we know that here's how you keep the GPUs running, here's how you keep the GPUs running, here's how you keep the GPUs running, and we know that like you got to you got and we know that like you got to you got and we know that like you got to you got to feed it this way, not that way, in to feed it this way, not that way, in to feed it this way, not that way, in this order. And I'm like, yeah, yeah, this order. And I'm like, yeah, yeah, this order. And I'm like, yeah, yeah, they have plenty of knowledge, they have plenty of knowledge, they have plenty of knowledge, but that's different from sort of like but that's different from sort of like but that's different from sort of like knowing what's going on inside the thing knowing what's going on inside the thing knowing what's going on inside the thing and understanding the mechanisms.

and understanding the mechanisms. and understanding the mechanisms. >> You're describing marriage. >> You're describing marriage. >> You're describing marriage. >> Yeah. >> Yeah. >> Yeah. >> Um, [clears throat] and >> Um, [clears throat] and >> Um, [clears throat] and I sort of try to stay out of the I sort of try to stay out of the I sort of try to stay out of the philosophical questions. Um, philosophical questions. Um, philosophical questions. Um, >> why? >> why? >> why? because I think I mean I I I sort of because I think I mean I I I sort of because I think I mean I I I sort of think about them on my own time and so think about them on my own time and so think about them on my own time and so on, but I'm sort of like on, but I'm sort of like on, but I'm sort of like uh like I think I think it would be bad uh like I think I think it would be bad uh like I think I think it would be bad for humanity to sort of like make for humanity to sort of like make for humanity to sort of like make artificial life and then abuse it. I artificial life and then abuse it. I artificial life and then abuse it. I think that would just be unbecoming of think that would just be unbecoming of think that would just be unbecoming of us as a species. like we we should sort us as a species. like we we should sort us as a species. like we we should sort of, you know, um of, you know, um of, you know, um I I just I I just I I just like we should not sort of make like we should not sort of make like we should not sort of make mechanical children and mistreat them. mechanical children and mistreat them. mechanical children and mistreat them. It's just it's it's not what, you know, It's just it's it's not what, you know, It's just it's it's not what, you know, the the sci-fi authors in the 1950s the the sci-fi authors in the 1950s the the sci-fi authors in the 1950s would have wanted us to become, you would have wanted us to become, you would have wanted us to become, you know, it's just like you have all these know, it's just like you have all these know, it's just like you have all these movies about like the evil corporations movies about like the evil corporations movies about like the evil corporations that uh you know, don't realize that that uh you know, don't realize that that uh you know, don't realize that they've made something precious with they've made something precious with they've made something precious with artificial life and then like torture it artificial life and then like torture it artificial life and then like torture it until something goes wrong. And I'm until something goes wrong. And I'm until something goes wrong. And I'm like, let's let's like not be those like, let's let's like not be those like, let's let's like not be those villains.

villains. villains. Um, but I'm also like, look, this is Um, but I'm also like, look, this is Um, but I'm also like, look, this is sort of uh there's sort of a separate sort of uh there's sort of a separate sort of uh there's sort of a separate question here, which is just like what question here, which is just like what question here, which is just like what happens if you keep making them smarter happens if you keep making them smarter happens if you keep making them smarter before you figure out how to make them before you figure out how to make them before you figure out how to make them care about us, care about us, care about us, >> right? >> right? >> right? >> And >> And >> And I I sort of respect the people who are I I sort of respect the people who are I I sort of respect the people who are investigating the current AI, trying to investigating the current AI, trying to investigating the current AI, trying to figure out what's going on, trying to figure out what's going on, trying to figure out what's going on, trying to figure out like, you know, uh, like figure out like, you know, uh, like figure out like, you know, uh, like people caring about AI treatment. I'm people caring about AI treatment. I'm people caring about AI treatment. I'm sort of like those are sort of like the sort of like those are sort of like the sort of like those are sort of like the good guys from the the the sci-fi good guys from the the the sci-fi good guys from the the the sci-fi stories that I grew up on. stories that I grew up on. stories that I grew up on. Um, Um, Um, and it can sort of both be the case that and it can sort of both be the case that and it can sort of both be the case that like we we we should be very careful like we we we should be very careful like we we we should be very careful around, around, around, you know, what the heck are we doing you know, what the heck are we doing you know, what the heck are we doing when it comes to making artificial life when it comes to making artificial life when it comes to making artificial life and that we shouldn't race ahead to make and that we shouldn't race ahead to make and that we shouldn't race ahead to make them much smarter than us while we have them much smarter than us while we have them much smarter than us while we have no idea what we're doing. no idea what we're doing. no idea what we're doing. >> Yes. I'm sort of like uh like a lot of >> Yes. I'm sort of like uh like a lot of >> Yes. I'm sort of like uh like a lot of people seem to think that like you have people seem to think that like you have people seem to think that like you have to like hate and mistreat the AIS if you to like hate and mistreat the AIS if you to like hate and mistreat the AIS if you also think they would that it would be also think they would that it would be also think they would that it would be bad to like race ahead here. And I'm bad to like race ahead here. And I'm bad to like race ahead here. And I'm like, "No, no, no." Like, you can sort like, "No, no, no." Like, you can sort like, "No, no, no." Like, you can sort of uh of uh of uh like be fascinated by the scientific like be fascinated by the scientific like be fascinated by the scientific discoveries that have been made and be discoveries that have been made and be discoveries that have been made and be like uh like care about how humanity like uh like care about how humanity like uh like care about how humanity comports itself around the creation of comports itself around the creation of comports itself around the creation of like these new entities and also be like like these new entities and also be like like these new entities and also be like it would be insane, guys, if we just it would be insane, guys, if we just it would be insane, guys, if we just like race to make these smarter and like race to make these smarter and like race to make these smarter and smarter with no idea what we're doing.

smarter with no idea what we're doing. smarter with no idea what we're doing. This can just like all be true at once. This can just like all be true at once. This can just like all be true at once. So, uh, So, uh, So, uh, your description on this made me feel your description on this made me feel your description on this made me feel despondent, hopeless. Had to get up and despondent, hopeless. Had to get up and despondent, hopeless. Had to get up and take a walk middle of an interview. I'm take a walk middle of an interview. I'm take a walk middle of an interview. I'm sure they'll edit it out, but I raised sure they'll edit it out, but I raised sure they'll edit it out, but I raised my hand said I I can't I got to walk my hand said I I can't I got to walk my hand said I I can't I got to walk around for a second. Um, but you seem around for a second. Um, but you seem around for a second. Um, but you seem pretty light and cheerful. pretty light and cheerful. pretty light and cheerful. Uh, what gives you optimism? Uh, what gives you optimism? Uh, what gives you optimism? We c we can't even keep we can't even We c we can't even keep we can't even We c we can't even keep we can't even clean up graffiti on public buildings. clean up graffiti on public buildings. clean up graffiti on public buildings. So, how is ours as a society organized So, how is ours as a society organized So, how is ours as a society organized enough to confront something like this? enough to confront something like this? enough to confront something like this? >> Yeah. Um, you know, I think a first >> Yeah. Um, you know, I think a first >> Yeah. Um, you know, I think a first thing I'll say there is, thing I'll say there is, thing I'll say there is, um, [clears throat] you know, I've been um, [clears throat] you know, I've been um, [clears throat] you know, I've been in this line of work for over a dozen in this line of work for over a dozen in this line of work for over a dozen years. years. years. And And And I I actually struggle with this I I actually struggle with this I I actually struggle with this sometimes when talking to people because sometimes when talking to people because sometimes when talking to people because uh they're sort of like, "Oh, you know, uh they're sort of like, "Oh, you know, uh they're sort of like, "Oh, you know, you seem like pretty disaffected or or you seem like pretty disaffected or or you seem like pretty disaffected or or light about it." And I'm like, "Well, light about it." And I'm like, "Well, light about it." And I'm like, "Well, you know, it's sort of the gallows you know, it's sort of the gallows you know, it's sort of the gallows humor." And uh humor." And uh humor." And uh like I I I sort of came to terms with a lot I I I sort of came to terms with a lot of this, you know, alone in 2012 when no of this, you know, alone in 2012 when no of this, you know, alone in 2012 when no one else had their eye on this. Um one else had their eye on this. Um one else had their eye on this. Um and and and >> what convinced you 14 years ago AI was a >> what convinced you 14 years ago AI was a >> what convinced you 14 years ago AI was a threat.

threat. threat. >> Um so there's this one is just a basic so there's this one is just a basic argument that if you sort of look at the argument that if you sort of look at the argument that if you sort of look at the world around us, world around us, world around us, it is shaped it is shaped it is shaped mostly according to human will mostly according to human will mostly according to human will more and more. You know there's some more and more. You know there's some more and more. You know there's some enclaves of nature still left enclaves of nature still left enclaves of nature still left thankfully. Um, thankfully. Um, thankfully. Um, but even you know, if you look around but even you know, if you look around but even you know, if you look around us, every piece of thing in our us, every piece of thing in our us, every piece of thing in our surroundings, I don't think we even have surroundings, I don't think we even have surroundings, I don't think we even have any windows open. All of this was sort any windows open. All of this was sort any windows open. All of this was sort of designed by humans, of designed by humans, of designed by humans, shaped by humans, and that's because shaped by humans, and that's because shaped by humans, and that's because we're the smartest creatures on the we're the smartest creatures on the we're the smartest creatures on the planet. planet. planet. If we make stuff smarter than us, faster If we make stuff smarter than us, faster If we make stuff smarter than us, faster than us, more efficient than us, then than us, more efficient than us, then than us, more efficient than us, then the planet starts to be shaped according the planet starts to be shaped according the planet starts to be shaped according to to those things. And so it's very to to those things. And so it's very to to those things. And so it's very very important that very important that very important that they be shaping the world towards they be shaping the world towards they be shaping the world towards something good if we make them at all. something good if we make them at all. something good if we make them at all. >> It's an abstract argument, but I was >> It's an abstract argument, but I was >> It's an abstract argument, but I was like, well that's like, well that's like, well that's >> No, no, no. It's the fundamental >> No, no, no. It's the fundamental >> No, no, no. It's the fundamental argument. It's the fundamental argument.

argument. It's the fundamental argument. argument. It's the fundamental argument. The smart the smartest entity is in The smart the smartest entity is in The smart the smartest entity is in charge over time. charge over time. charge over time. >> That's right. And so >> That's right. And so >> That's right. And so >> why would we relinquish sovereignty to a >> why would we relinquish sovereignty to a >> why would we relinquish sovereignty to a machine that we made? Like why would you machine that we made? Like why would you machine that we made? Like why would you do that? do that? do that? >> And so, you know, back then I was sort >> And so, you know, back then I was sort >> And so, you know, back then I was sort of like, okay, who is on this? who is on of like, okay, who is on this? who is on of like, okay, who is on this? who is on making sure that that's going to be making sure that that's going to be making sure that that's going to be okay? And the answer was almost no one. okay? And the answer was almost no one. okay? And the answer was almost no one. Uh and so I was like, well, I guess Uh and so I was like, well, I guess Uh and so I was like, well, I guess that's me then. that's me then. that's me then. Um yeah, and I think uh yeah, and I think uh I I I am pretty pissed off about a lot I I I am pretty pissed off about a lot I I I am pretty pissed off about a lot of this. I I often don't uh of this. I I often don't uh of this. I I often don't uh I try not to show my my my I try not to show my my my I try not to show my my my uh frustration uh frustration uh frustration on the air very much. on the air very much. on the air very much. Um yeah, it's it's it's heavy. yeah, it's it's it's heavy. Uh and um um as that's one piece of the puzzle before as that's one piece of the puzzle before as that's one piece of the puzzle before I get to the hope.

I get to the hope. I get to the hope. >> So where does the hope come from >> So where does the hope come from >> So where does the hope come from exactly? exactly? exactly? >> My biggest hope here >> My biggest hope here >> My biggest hope here comes from comes from comes from the fact that most people don't the fact that most people don't the fact that most people don't understand what these guys are trying to understand what these guys are trying to understand what these guys are trying to do. do. do. Um, one way I like to say it is Um, one way I like to say it is Um, one way I like to say it is the bad news is that the bus is racing the bad news is that the bus is racing the bad news is that the bus is racing towards the cliff edge. towards the cliff edge. towards the cliff edge. The good news is that the bus driver is The good news is that the bus driver is The good news is that the bus driver is asleep, which might seem bad. No, it seems good, which might seem bad. No, it seems good, but it's Yeah, it's it's but it's Yeah, it's it's but it's Yeah, it's it's if you can wake the bus driver up, you if you can wake the bus driver up, you if you can wake the bus driver up, you know, it's much better to be in a bus know, it's much better to be in a bus know, it's much better to be in a bus where the driver that's headed towards a where the driver that's headed towards a where the driver that's headed towards a cliff if the driver's asleep than if cliff if the driver's asleep than if cliff if the driver's asleep than if they're awake. They're awake and they're they're awake. They're awake and they're they're awake. They're awake and they're choosing the cliff, right? choosing the cliff, right? choosing the cliff, right? It seemed like we don't see we we see a It seemed like we don't see we we see a It seemed like we don't see we we see a lot of our leaders lot of our leaders lot of our leaders talking about how they don't want to talking about how they don't want to talking about how they don't want to stifle innovation with AI, stifle innovation with AI, stifle innovation with AI, talking about how it's going to unleash talking about how it's going to unleash talking about how it's going to unleash economic opportunity and we're going to economic opportunity and we're going to economic opportunity and we're going to have to like be a little bit careful have to like be a little bit careful have to like be a little bit careful around the jobs.

around the jobs. around the jobs. Uh talking about how, you know, the Uh talking about how, you know, the Uh talking about how, you know, the self-driving cars, should we like are self-driving cars, should we like are self-driving cars, should we like are they good or are they bad? they good or are they bad? they good or are they bad? That's a different conversation than the That's a different conversation than the That's a different conversation than the conversation that's happening in Silicon conversation that's happening in Silicon conversation that's happening in Silicon Valley. Valley. Valley. In Silicon Valley, people are spooked. In Silicon Valley, people are spooked. In Silicon Valley, people are spooked. You know, um, when people leave a normal You know, um, when people leave a normal You know, um, when people leave a normal tech company, the the way it was for for tech company, the the way it was for for tech company, the the way it was for for decades is they'd be like, I've had a decades is they'd be like, I've had a decades is they'd be like, I've had a lovely time at this tech company. I'm lovely time at this tech company. I'm lovely time at this tech company. I'm moving on to the next adventure. I'm so moving on to the next adventure. I'm so moving on to the next adventure. I'm so thankful for all of the things I learned thankful for all of the things I learned thankful for all of the things I learned here and all the projects we worked on. here and all the projects we worked on. here and all the projects we worked on. When people leave an AI company, and When people leave an AI company, and When people leave an AI company, and this basically happened, I'm not going this basically happened, I'm not going this basically happened, I'm not going to get it exactly word for word, but to get it exactly word for word, but to get it exactly word for word, but this is this is pretty close to word for this is this is pretty close to word for this is this is pretty close to word for word. When people leave an AI company, word. When people leave an AI company, word. When people leave an AI company, they say, uh, I have stared into the they say, uh, I have stared into the they say, uh, I have stared into the abyss. abyss. abyss. I am quitting to write poetry. I am quitting to write poetry. I am quitting to write poetry. [laughter] Please spend time with your families. Please spend time with your families. >> Yeah. >> Yeah. >> Yeah. >> You know, and and these these guys >> You know, and and these these guys >> You know, and and these these guys bandandy around, you know, at the water bandandy around, you know, at the water bandandy around, you know, at the water cooler. what's the probability that you cooler. what's the probability that you cooler. what's the probability that you think we're going to destroy the world think we're going to destroy the world think we're going to destroy the world in this business? You know, it's like in in this business? You know, it's like in in this business? You know, it's like in Silicon Valley and they feel trapped in Silicon Valley and they feel trapped in Silicon Valley and they feel trapped in a death race. You know, they there was a death race. You know, they there was a death race. You know, they there was just over a thousand employees, just over a thousand employees, just over a thousand employees, including some of the chief executives, including some of the chief executives, including some of the chief executives, signed a letter a couple weeks ago that signed a letter a couple weeks ago that signed a letter a couple weeks ago that was like, "Please, it was an appeal to was like, "Please, it was an appeal to was like, "Please, it was an appeal to the world the world leaders saying, the world the world leaders saying, the world the world leaders saying, please build the technology that will be please build the technology that will be please build the technology that will be required to pace the development of required to pace the development of required to pace the development of artificial intelligence." because artificial intelligence." because artificial intelligence." because they're like, "We're worried it's going they're like, "We're worried it's going they're like, "We're worried it's going to get out of control and that if we're

to get out of control and that if we're to get out of control and that if we're stuck in a race, we're not going to be stuck in a race, we're not going to be stuck in a race, we're not going to be able to do it." Right? These guys are able to do it." Right? These guys are able to do it." Right? These guys are spooked. spooked. spooked. But the hope is that the rest of the But the hope is that the rest of the But the hope is that the rest of the world isn't spooked like that. world isn't spooked like that. world isn't spooked like that. The rest of the world thinks these guys The rest of the world thinks these guys The rest of the world thinks these guys are chatbot companies. are chatbot companies. are chatbot companies. Thinks they're going to stop at the chat Thinks they're going to stop at the chat Thinks they're going to stop at the chat bots. They haven't really understood bots. They haven't really understood bots. They haven't really understood that these guys are racing to make the that these guys are racing to make the that these guys are racing to make the sand god, right? And I think if people sand god, right? And I think if people sand god, right? And I think if people understand what they're doing and understand what they're doing and understand what they're doing and understand that they have a chance of understand that they have a chance of understand that they have a chance of success, success, success, they'll be like, "Wo, holy crap." they'll be like, "Wo, holy crap." they'll be like, "Wo, holy crap." Absolutely not. Absolutely not. Absolutely not. You know, You know, You know, will we get there in time? I don't know. will we get there in time? I don't know. will we get there in time? I don't know. But But But why wouldn't we blow up the data why wouldn't we blow up the data why wouldn't we blow up the data centers? Like that's a centers? Like that's a centers? Like that's a I mean, and return America to productive I mean, and return America to productive I mean, and return America to productive use like farmland or parks. I don't use like farmland or parks. I don't use like farmland or parks. I don't understand. understand. understand. >> You know, I think uh a a globally >> You know, I think uh a a globally >> You know, I think uh a a globally coordinated coordinated coordinated uh you know, I I think if the US and uh you know, I I think if the US and uh you know, I I think if the US and China were like, we are simply not going China were like, we are simply not going China were like, we are simply not going to do super intelligence. We are simply to do super intelligence. We are simply to do super intelligence. We are simply not going to collect a 100,000 of the not going to collect a 100,000 of the not going to collect a 100,000 of the most advanced chips into these enormous most advanced chips into these enormous most advanced chips into these enormous data centers that suck down electricity data centers that suck down electricity data centers that suck down electricity comparable to a city and then try to comparable to a city and then try to comparable to a city and then try to train a super intelligent AI in that.

train a super intelligent AI in that. train a super intelligent AI in that. We're not going to do it. You're not We're not going to do it. You're not We're not going to do it. You're not going to do it. we're going to monitor going to do it. we're going to monitor going to do it. we're going to monitor where the heavy trip trip trip where the heavy trip trip trip where the heavy trip trip trip concentrations are and not not have that concentrations are and not not have that concentrations are and not not have that happen. Um I think that could be done. A happen. Um I think that could be done. A happen. Um I think that could be done. A I think that if they started seeing I think that if they started seeing I think that if they started seeing people defect against such a treaty that people defect against such a treaty that people defect against such a treaty that it's the sort of treaty you might want it's the sort of treaty you might want it's the sort of treaty you might want to use force to you use diplomacy first to use force to you use diplomacy first to use force to you use diplomacy first but ultimately every treaty is backed by but ultimately every treaty is backed by but ultimately every treaty is backed by force. Um, and I think that force. Um, and I think that force. Um, and I think that it's very possible that if this sort of it's very possible that if this sort of it's very possible that if this sort of treaty happened, we would want to not treaty happened, we would want to not treaty happened, we would want to not just stop forward progress, but take a just stop forward progress, but take a just stop forward progress, but take a step back. We've seen this in treaties step back. We've seen this in treaties step back. We've seen this in treaties before. After World War I, there were uh before. After World War I, there were uh before. After World War I, there were uh naval treaties that put limits on total naval treaties that put limits on total naval treaties that put limits on total tonnage of naval forces that were tonnage of naval forces that were tonnage of naval forces that were actually lower than what existed. actually lower than what existed. actually lower than what existed. >> Yep. So countries would, you know, >> Yep. So countries would, you know, >> Yep. So countries would, you know, scuttle some of their ships because scuttle some of their ships because scuttle some of their ships because they're like, look, we just don't want they're like, look, we just don't want they're like, look, we just don't want to do this arms race, right? And so I to do this arms race, right? And so I to do this arms race, right? And so I could see us uh stepping back if we could see us uh stepping back if we could see us uh stepping back if we could get this global coordination. Um I could get this global coordination. Um I could get this global coordination. Um I I do think that it sort of needs to be I do think that it sort of needs to be I do think that it sort of needs to be global. It it sort of doesn't actually global. It it sort of doesn't actually global. It it sort of doesn't actually solve the problem to just stop the US solve the problem to just stop the US solve the problem to just stop the US data centers because then the data data centers because then the data data centers because then the data centers just go abroad and an AI does centers just go abroad and an AI does centers just go abroad and an AI does not need to be running in a US data not need to be running in a US data not need to be running in a US data center to threaten a US life. You know, center to threaten a US life. You know, center to threaten a US life. You know, it sort of doesn't matter whether the it sort of doesn't matter whether the it sort of doesn't matter whether the swarm escapes from a US data center or a swarm escapes from a US data center or a swarm escapes from a US data center or a Chinese data center. If the swarm Chinese data center. If the swarm Chinese data center. If the swarm escapes and starts replicating and escapes and starts replicating and escapes and starts replicating and starts getting control of robot bodies starts getting control of robot bodies starts getting control of robot bodies and starts getting control of human and starts getting control of human and starts getting control of human cultists, cultists, cultists, it sort of doesn't matter where it it sort of doesn't matter where it it sort of doesn't matter where it originated.

originated. originated. Well, I mean, just to bring it to a very Well, I mean, just to bring it to a very Well, I mean, just to bring it to a very small and practical level, so many of small and practical level, so many of small and practical level, so many of the electronics in your house are, you the electronics in your house are, you the electronics in your house are, you know, Bluetooth enabled. know, Bluetooth enabled. know, Bluetooth enabled. Not in my house. I will say I've been on Not in my house. I will say I've been on Not in my house. I will say I've been on this for a while, Nate. Um, but sorry, this for a while, Nate. Um, but sorry, this for a while, Nate. Um, but sorry, >> I don't even have a house. [laughter] >> I don't even have a house. [laughter] >> I don't even have a house. [laughter] >> You, man, you're really black belt. >> You, man, you're really black belt. >> You, man, you're really black belt. >> Got to figure it out. >> Got to figure it out. >> Got to figure it out. >> That maybe turned out to be very smart. >> That maybe turned out to be very smart. >> That maybe turned out to be very smart. Um, but I mean like a world where, you Um, but I mean like a world where, you Um, but I mean like a world where, you know, your washing machine or your know, your washing machine or your know, your washing machine or your refrigerator are controlled by, refrigerator are controlled by, refrigerator are controlled by, you know, a force like this. Is that you know, a force like this. Is that you know, a force like this. Is that possible? possible? possible? >> Uh, it's definitely possible. I don't >> Uh, it's definitely possible. I don't >> Uh, it's definitely possible. I don't think that's really where the damage is, think that's really where the damage is, think that's really where the damage is, you know. Um, I think that the the you know. Um, I think that the the you know. Um, I think that the the damage is more like can the AI get damage is more like can the AI get damage is more like can the AI get anything self-replicating. anything self-replicating. anything self-replicating. >> Yeah, >> Yeah, >> Yeah, >> that's sort of one of the big uh that >> that's sort of one of the big uh that >> that's sort of one of the big uh that that's in some sense the big hurdle to that's in some sense the big hurdle to that's in some sense the big hurdle to self-sufficiency. self-sufficiency. self-sufficiency. And if you sort of think of this from And if you sort of think of this from And if you sort of think of this from the AI's perspective, the AI's perspective, the AI's perspective, um, um, um, you know, there's there's a number of you know, there's there's a number of you know, there's there's a number of ways that humans are, ways that humans are, ways that humans are, even if you don't care about the humans even if you don't care about the humans even if you don't care about the humans at all for for as an ends, at all for for as an ends, at all for for as an ends, there's a way that humans are sort of there's a way that humans are sort of there's a way that humans are sort of annoying or an issue for the AI. One way annoying or an issue for the AI. One way annoying or an issue for the AI. One way is if the humans are trying to shut the is if the humans are trying to shut the is if the humans are trying to shut the AI down.

AI down. AI down. >> Yes. One way is if the humans get into a >> Yes. One way is if the humans get into a >> Yes. One way is if the humans get into a nuclear war with themselves, that could nuclear war with themselves, that could nuclear war with themselves, that could really, you know, mess up a lot of really, you know, mess up a lot of really, you know, mess up a lot of infrastructure on the planet. Would be infrastructure on the planet. Would be infrastructure on the planet. Would be very frustrating for an AI. Uh, I mean, very frustrating for an AI. Uh, I mean, very frustrating for an AI. Uh, I mean, you know, maybe maybe they don't feel you know, maybe maybe they don't feel you know, maybe maybe they don't feel frustration, but whatever. Um, frustration, but whatever. Um, frustration, but whatever. Um, and a third is if humanity has created and a third is if humanity has created and a third is if humanity has created one AI or swarm of AIs, what if humanity one AI or swarm of AIs, what if humanity one AI or swarm of AIs, what if humanity creates another that could serve as a creates another that could serve as a creates another that could serve as a real threat to the AI? Like even if real threat to the AI? Like even if real threat to the AI? Like even if these AIs are much more powerful than these AIs are much more powerful than these AIs are much more powerful than humanity humanity humanity and don't worry about humans too much, and don't worry about humans too much, and don't worry about humans too much, if humanity made one, they can make a if humanity made one, they can make a if humanity made one, they can make a second and AI might not want that. And second and AI might not want that. And second and AI might not want that. And so those are reasons why once the AI is so those are reasons why once the AI is so those are reasons why once the AI is self-sufficient, it might be like, ah self-sufficient, it might be like, ah self-sufficient, it might be like, ah man, the humans are a nuisance. What if man, the humans are a nuisance. What if man, the humans are a nuisance. What if they try to shut me down? What if they they try to shut me down? What if they they try to shut me down? What if they launch the nukes? What if they make a launch the nukes? What if they make a launch the nukes? What if they make a competitor? I'm just going to like make competitor? I'm just going to like make competitor? I'm just going to like make a virus and wipe them out. You know, I a virus and wipe them out. You know, I a virus and wipe them out. You know, I don't think the AI sort of needs to take don't think the AI sort of needs to take don't think the AI sort of needs to take over your washing machine to do that. over your washing machine to do that. over your washing machine to do that. From the AI's perspective, it's more From the AI's perspective, it's more From the AI's perspective, it's more like how do I become self-sufficient? like how do I become self-sufficient? like how do I become self-sufficient? self-replicating self-replicating self-replicating in the hardware as well as the digital in the hardware as well as the digital in the hardware as well as the digital and then um you know if human man is a and then um you know if human man is a and then um you know if human man is a nuisance how do you sort of nuisance how do you sort of nuisance how do you sort of make them stop being a nuisance which make them stop being a nuisance which make them stop being a nuisance which could be by killing them or could be by could be by killing them or could be by could be by killing them or could be by just you know taking away all their just you know taking away all their just you know taking away all their computers and being like that was too computers and being like that was too computers and being like that was too dangerous for you tech executive who's dangerous for you tech executive who's dangerous for you tech executive who's developing AI who is not Elon developing AI who is not Elon developing AI who is not Elon uh said to me in private pretty recently uh said to me in private pretty recently uh said to me in private pretty recently that the point of Neuralink and that the point of Neuralink and that the point of Neuralink and companies like Neuralink was to give companies like Neuralink was to give companies like Neuralink was to give people parody with AI.

people parody with AI. people parody with AI. So, like we know that we're going to be So, like we know that we're going to be So, like we know that we're going to be at this massive disadvantage, so you at this massive disadvantage, so you at this massive disadvantage, so you need chips in your brain to be as smart need chips in your brain to be as smart need chips in your brain to be as smart as AI. as AI. as AI. >> Yeah. Um I mean, my my topline thought >> Yeah. Um I mean, my my topline thought >> Yeah. Um I mean, my my topline thought about that is that at the point when about that is that at the point when about that is that at the point when you're like, uh we are making the you're like, uh we are making the you're like, uh we are making the technology that's going to wipe us out technology that's going to wipe us out technology that's going to wipe us out unless we all put chips in our head to unless we all put chips in our head to unless we all put chips in our head to compete. compete. compete. Maybe maybe it's time to back off a Maybe maybe it's time to back off a Maybe maybe it's time to back off a little, [laughter] little, [laughter] little, [laughter] you know? Maybe maybe that one was you know? Maybe maybe that one was you know? Maybe maybe that one was supposed to be a little bit of a warning supposed to be a little bit of a warning supposed to be a little bit of a warning sign. sign. sign. >> Yeah. Get some fresh air. Yeah. No, I >> Yeah. Get some fresh air. Yeah. No, I >> Yeah. Get some fresh air. Yeah. No, I >> but it um this person said it to me in >> but it um this person said it to me in >> but it um this person said it to me in seriousness and I I think as an seriousness and I I think as an seriousness and I I think as an endorsement of the idea, but it seemed endorsement of the idea, but it seemed endorsement of the idea, but it seemed like a well it's insane as you just like a well it's insane as you just like a well it's insane as you just pointed out like it's just crazy. pointed out like it's just crazy. pointed out like it's just crazy. >> Yeah. >> Yeah. >> Yeah. >> Um but it seemed like a vulnerability >> Um but it seemed like a vulnerability >> Um but it seemed like a vulnerability like if they can hack anything, why like if they can hack anything, why like if they can hack anything, why would I want them in my I want would I want them in my I want would I want them in my I want electronics in my brain? electronics in my brain? electronics in my brain? >> Uh totally. I also think like even on >> Uh totally. I also think like even on >> Uh totally. I also think like even on its merits, it doesn't stand up. Like it its merits, it doesn't stand up. Like it its merits, it doesn't stand up. Like it feels like someone's saying um in order feels like someone's saying um in order feels like someone's saying um in order to keep the horses around after we to keep the horses around after we to keep the horses around after we invent cars, we're going to invent invent cars, we're going to invent invent cars, we're going to invent cybernetic horses.

cybernetic horses. cybernetic horses. >> Yeah. >> Yeah. >> Yeah. >> That are enhanced so they can keep up >> That are enhanced so they can keep up >> That are enhanced so they can keep up with the cars. And I'm like with the cars. And I'm like with the cars. And I'm like like is it technically possible to make like is it technically possible to make like is it technically possible to make a cybernetic horse that can run as fast a cybernetic horse that can run as fast a cybernetic horse that can run as fast as a car? as a car? as a car? Maybe. Maybe. Maybe. Are you going to figure that out in time Are you going to figure that out in time Are you going to figure that out in time for the horses to be competitive with for the horses to be competitive with for the horses to be competitive with the cars? Like absolutely not, the cars? Like absolutely not, the cars? Like absolutely not, >> right? >> right? >> right? >> You know, it's like that's that's just >> You know, it's like that's that's just >> You know, it's like that's that's just not not not like like like like the the AIs that were that were like the the AIs that were that were like the the AIs that were that were escaping here and doing these cyber escaping here and doing these cyber escaping here and doing these cyber attacks were were inventing novel cyber attacks were were inventing novel cyber attacks were were inventing novel cyber attacks attacks attacks that these are called zero day attacks that these are called zero day attacks that these are called zero day attacks because the people who uh would because the people who uh would because the people who uh would uh the people who need to respond to it uh the people who need to respond to it uh the people who need to respond to it have had zero days to prepare. have had zero days to prepare. have had zero days to prepare. And among humans, a zeroday attack sells And among humans, a zeroday attack sells And among humans, a zeroday attack sells for somewhere between $100,000 and $5 for somewhere between $100,000 and $5 for somewhere between $100,000 and $5 million million million depending on what you manage to break. depending on what you manage to break. depending on what you manage to break. These are hard to come by. You can make These are hard to come by. You can make These are hard to come by. You can make a real living. If you can find zero day a real living. If you can find zero day a real living. If you can find zero day attacks, you can make a real living attacks, you can make a real living attacks, you can make a real living selling them. And I know people who do.

selling them. And I know people who do. selling them. And I know people who do. The AIS in this swarm The AIS in this swarm The AIS in this swarm were finding multiple zero days and were finding multiple zero days and were finding multiple zero days and chaining them together to break out of chaining them together to break out of chaining them together to break out of their training enclosure and then go their training enclosure and then go their training enclosure and then go break into other computers. And and when break into other computers. And and when break into other computers. And and when they broke out of their enclosure the they broke out of their enclosure the they broke out of their enclosure the first time and they the the holes were first time and they the the holes were first time and they the the holes were patched, they just found other zero day patched, they just found other zero day patched, they just found other zero day attacks to break out again like it was attacks to break out again like it was attacks to break out again like it was nothing, right? It's like uh nothing, right? It's like uh nothing, right? It's like uh like like like the the the AIs are already ahead where the the the AIs are already ahead where the the the AIs are already ahead where they're ahead they're ahead they're ahead and the pace of progress is really fast. and the pace of progress is really fast. and the pace of progress is really fast. You know, GPT is like what a You know, GPT is like what a You know, GPT is like what a four-year-old. four-year-old. four-year-old. If you think in terms of like number of If you think in terms of like number of If you think in terms of like number of years chat GPT has been around, it's years chat GPT has been around, it's years chat GPT has been around, it's like resolving long-standing math like resolving long-standing math like resolving long-standing math conjectures that have stood for decades. conjectures that have stood for decades. conjectures that have stood for decades. >> Yeah. after four years, right? And and >> Yeah. after four years, right? And and >> Yeah. after four years, right? And and you sort of like think we're going to you sort of like think we're going to you sort of like think we're going to put chips in the human's heads and like put chips in the human's heads and like put chips in the human's heads and like outrun this thing. It's just, outrun this thing. It's just, outrun this thing. It's just, you know, even on its merits, it falls you know, even on its merits, it falls you know, even on its merits, it falls down. Although, mostly, again, I would down. Although, mostly, again, I would down. Although, mostly, again, I would be like, be like, be like, >> maybe we shouldn't be arguing this on >> maybe we shouldn't be arguing this on >> maybe we shouldn't be arguing this on his merits. Maybe we should be like his merits. Maybe we should be like his merits. Maybe we should be like stepping back a little and being like, stepping back a little and being like, stepping back a little and being like, you're trying what you're trying what you're trying what >> exactly.

>> exactly. >> exactly. Um, how far are we from the point of no Um, how far are we from the point of no Um, how far are we from the point of no return? return? return? >> I wish I knew. Um, I can tell you two >> I wish I knew. Um, I can tell you two >> I wish I knew. Um, I can tell you two stories here. stories here. stories here. There's sort of the hopeful story, and I There's sort of the hopeful story, and I There's sort of the hopeful story, and I guess we didn't even get to the the big guess we didn't even get to the the big guess we didn't even get to the the big hope part. I should maybe give more of hope part. I should maybe give more of hope part. I should maybe give more of my hope speech in a in a minute. But um, my hope speech in a in a minute. But um, my hope speech in a in a minute. But um, >> you definitely should. >> you definitely should. >> you definitely should. >> Yeah. The um, one way it could go is >> Yeah. The um, one way it could go is >> Yeah. The um, one way it could go is that AI finally hits a wall. You know, that AI finally hits a wall. You know, that AI finally hits a wall. You know, there's guys who've been saying AI is there's guys who've been saying AI is there's guys who've been saying AI is going to hit a wall. It's going to peter going to hit a wall. It's going to peter going to hit a wall. It's going to peter out. Maybe that finally happens. They've out. Maybe that finally happens. They've out. Maybe that finally happens. They've been predicting it every six months for been predicting it every six months for been predicting it every six months for the past 5 years, but maybe this is the past 5 years, but maybe this is the past 5 years, but maybe this is finally the year. AI hits a wall uh and finally the year. AI hits a wall uh and finally the year. AI hits a wall uh and then it it's it sort of struggles for 5 then it it's it sort of struggles for 5 then it it's it sort of struggles for 5 years. The bubble pops, some of the years. The bubble pops, some of the years. The bubble pops, some of the companies die, but the bubble popping companies die, but the bubble popping companies die, but the bubble popping doesn't mean everything goes away. The doesn't mean everything goes away. The doesn't mean everything goes away. The dot bubble popped and that did not mean dot bubble popped and that did not mean dot bubble popped and that did not mean that the internet disappeared. No. that the internet disappeared. No. that the internet disappeared. No. >> Right. And so in that world, maybe you >> Right. And so in that world, maybe you >> Right. And so in that world, maybe you have 5 years of struggling and then 5 have 5 years of struggling and then 5 have 5 years of struggling and then 5 years of uh people figuring out some new years of uh people figuring out some new years of uh people figuring out some new scientific discovery that makes AI be scientific discovery that makes AI be scientific discovery that makes AI be able to keep going again cuz they're able to keep going again cuz they're able to keep going again cuz they're trying. You know, this this whole trying. You know, this this whole trying. You know, this this whole language model stuff was unleashed by language model stuff was unleashed by language model stuff was unleashed by one math paper called attention is all one math paper called attention is all one math paper called attention is all you need. Maybe there's another math you need. Maybe there's another math you need. Maybe there's another math paper in 10 years and then 5 years after paper in 10 years and then 5 years after paper in 10 years and then 5 years after that AI is ripping again and that's the that AI is ripping again and that's the that AI is ripping again and that's the the the round that kills us all. Right?

the the round that kills us all. Right? the the round that kills us all. Right? So that's a story where you have 15 So that's a story where you have 15 So that's a story where you have 15 years on the clock. years on the clock. years on the clock. A story where you have less time than A story where you have less time than A story where you have less time than that on the clock is that um you know that on the clock is that um you know that on the clock is that um you know maybe a training run finishes in six maybe a training run finishes in six maybe a training run finishes in six months months months and just like how and just like how and just like how Claude Mythos was better than everybody Claude Mythos was better than everybody Claude Mythos was better than everybody else at hacking, else at hacking, else at hacking, maybe a training ground finishes in 6 maybe a training ground finishes in 6 maybe a training ground finishes in 6 months and that AI is better than months and that AI is better than months and that AI is better than everybody else at AI research. everybody else at AI research. everybody else at AI research. And maybe in 6 months you have an AI And maybe in 6 months you have an AI And maybe in 6 months you have an AI that starts making a smarter AI that that starts making a smarter AI that that starts making a smarter AI that starts making a smarter AI that starts starts making a smarter AI that starts starts making a smarter AI that starts making a smarter AI. And then 9 months making a smarter AI. And then 9 months making a smarter AI. And then 9 months from now you have another swarm escape, from now you have another swarm escape, from now you have another swarm escape, but this time it's not just hacking. but this time it's not just hacking. but this time it's not just hacking. This time it is self-replicating and This time it is self-replicating and This time it is self-replicating and self-improving. self-improving. self-improving. And uh you know maybe it it it escapes And uh you know maybe it it it escapes And uh you know maybe it it it escapes onto hidden computers and starts forming onto hidden computers and starts forming onto hidden computers and starts forming these cults and starts taking control of these cults and starts taking control of these cults and starts taking control of robots and starts building its own robots and starts building its own robots and starts building its own computing infrastructure and maybe it computing infrastructure and maybe it computing infrastructure and maybe it gets very very smart and cracks certain gets very very smart and cracks certain gets very very smart and cracks certain technological advances and then maybe technological advances and then maybe technological advances and then maybe the world ends in a year right and so do the world ends in a year right and so do the world ends in a year right and so do we have a year do we have 15 years I we have a year do we have 15 years I we have a year do we have 15 years I don't know don't know don't know but your point that it's not simply its but your point that it's not simply its but your point that it's not simply its capacity to take over robots that's a capacity to take over robots that's a capacity to take over robots that's a threat is the capacity to take over threat is the capacity to take over threat is the capacity to take over people.

people. people. >> Take over people, take over robots. And >> Take over people, take over robots. And >> Take over people, take over robots. And um um um uh biotechnology uh biotechnology uh biotechnology is another big threat vector. And um is another big threat vector. And um is another big threat vector. And um self-improvement self-improvement self-improvement is sort of the the hidden threat vector is sort of the the hidden threat vector is sort of the the hidden threat vector of like what if it can make itself of like what if it can make itself of like what if it can make itself smarter and smarter until the point smarter and smarter until the point smarter and smarter until the point where it can just write custom DNA where it can just write custom DNA where it can just write custom DNA strands to make custom life. So now is the time for your optimism So now is the time for your optimism speech. I think speech. I think speech. I think >> that's right. >> that's right. >> that's right. >> Yeah. Um first part of the optimism >> Yeah. Um first part of the optimism >> Yeah. Um first part of the optimism speech is that we could absolutely put a speech is that we could absolutely put a speech is that we could absolutely put a stop to it if we tried. stop to it if we tried. stop to it if we tried. The uh training one of these frontier The uh training one of these frontier The uh training one of these frontier models takes something like a 100,000 of models takes something like a 100,000 of models takes something like a 100,000 of the most heavily advanced computer chips the most heavily advanced computer chips the most heavily advanced computer chips humanity can produce, which are at the humanity can produce, which are at the humanity can produce, which are at the peak of a global supply chain, most of peak of a global supply chain, most of peak of a global supply chain, most of which is controlled by us or our allies. which is controlled by us or our allies. which is controlled by us or our allies. Uh, it would be possible to require that Uh, it would be possible to require that Uh, it would be possible to require that those trips have location tracking those trips have location tracking those trips have location tracking devices, that those trips have devices, that those trips have devices, that those trips have monitoring devices monitoring devices monitoring devices [clears throat] [clears throat] [clears throat] uh that make it possible for uh that make it possible for uh that make it possible for uh monitors to see whether they're uh monitors to see whether they're uh monitors to see whether they're running anything dangerous. You could running anything dangerous. You could running anything dangerous. You could set up clever schemes where you say, set up clever schemes where you say, set up clever schemes where you say, "Hey, you know, uh neither the US nor "Hey, you know, uh neither the US nor "Hey, you know, uh neither the US nor China wants to fall behind about uh some China wants to fall behind about uh some China wants to fall behind about uh some of the military applications or the of the military applications or the of the military applications or the economic applications. We want to make economic applications. We want to make economic applications. We want to make sure there's not super intelligent stuff sure there's not super intelligent stuff sure there's not super intelligent stuff going on, but we still want to be able going on, but we still want to be able going on, but we still want to be able to run a lot of the non-super to run a lot of the non-super to run a lot of the non-super intelligent AIs for various purposes."

intelligent AIs for various purposes." intelligent AIs for various purposes." And we could be like, "Okay, so you And we could be like, "Okay, so you And we could be like, "Okay, so you know, China is going to set up its data know, China is going to set up its data know, China is going to set up its data centers in Canada, just over the border, centers in Canada, just over the border, centers in Canada, just over the border, and the US is going to set up its data and the US is going to set up its data and the US is going to set up its data centers in Mongolia, just over the centers in Mongolia, just over the centers in Mongolia, just over the border. And then if you know the border. And then if you know the border. And then if you know the monitoring apparatuses go down for a monitoring apparatuses go down for a monitoring apparatuses go down for a moment, we'll just have the troops go moment, we'll just have the troops go moment, we'll just have the troops go in, we'll have treaties with Mongolia in, we'll have treaties with Mongolia in, we'll have treaties with Mongolia and Canada so that this doesn't start an and Canada so that this doesn't start an and Canada so that this doesn't start an international war. But like, you know, international war. But like, you know, international war. But like, you know, we're just going to be very serious we're just going to be very serious we're just going to be very serious about we're monitoring your chips, about we're monitoring your chips, about we're monitoring your chips, you're monitoring our chips, no one's you're monitoring our chips, no one's you're monitoring our chips, no one's doing the really dangerous race to super doing the really dangerous race to super doing the really dangerous race to super intelligence. It's just like intelligence. It's just like intelligence. It's just like like this this would take less work than like this this would take less work than like this this would take less work than defeating the Nazis. defeating the Nazis. defeating the Nazis. It it requires moving less matter It it requires moving less matter It it requires moving less matter around. If humanity was like, screw around. If humanity was like, screw around. If humanity was like, screw this, we're surviving. this, we're surviving. this, we're surviving. It is absolutely within the realm of It is absolutely within the realm of It is absolutely within the realm of possibility to set up an agreement where possibility to set up an agreement where possibility to set up an agreement where we get to keep cancer cure research. We we get to keep cancer cure research. We we get to keep cancer cure research. We get to keep like the the the military get to keep like the the the military get to keep like the the the military can keep the non-super intelligent AI in can keep the non-super intelligent AI in can keep the non-super intelligent AI in their military devices. you know, we can their military devices. you know, we can their military devices. you know, we can we can keep a lot of the good stuff and we can keep a lot of the good stuff and we can keep a lot of the good stuff and we can say we're not doing the super we can say we're not doing the super we can say we're not doing the super intelligence and we could monitor and intelligence and we could monitor and intelligence and we could monitor and enforce that treaty.

enforce that treaty. enforce that treaty. So part one of the good news is that all So part one of the good news is that all So part one of the good news is that all we're missing is the political will. we're missing is the political will. we're missing is the political will. And now we just, you know, I need to And now we just, you know, I need to And now we just, you know, I need to tell you why we can be extremely tell you why we can be extremely tell you why we can be extremely optimistic about the sanity of politics optimistic about the sanity of politics optimistic about the sanity of politics in the modern era. >> Yeah. I'm not even going to say what I >> Yeah. I'm not even going to say what I think that I mean I want that to happen. think that I mean I want that to happen. think that I mean I want that to happen. >> Yeah. Is there any indication that it's >> Yeah. Is there any indication that it's >> Yeah. Is there any indication that it's moving in that direction? moving in that direction? moving in that direction? >> Um, so my so, so I think it's rough. I think so my so, so I think it's rough. I think there's a sense in which the world is less grown up than it was in the 1950s. less grown up than it was in the 1950s. >> I've noticed >> I've noticed >> I've noticed >> in the 1960s, >> in the 1960s, >> in the 1960s, um, which makes things harder. um, which makes things harder. um, which makes things harder. I think I think I think the the the there's I think there's a couple reasons there's I think there's a couple reasons there's I think there's a couple reasons for hope here. One reason for hope is for hope here. One reason for hope is for hope here. One reason for hope is that I've spoken to a lot of people who that I've spoken to a lot of people who that I've spoken to a lot of people who are concerned about the AI stuff. Some are concerned about the AI stuff. Some are concerned about the AI stuff. Some of them who are, you know, members of of them who are, you know, members of of them who are, you know, members of Congress or otherwise in positions of at Congress or otherwise in positions of at Congress or otherwise in positions of at least nominal power. And I've spoken to least nominal power. And I've spoken to least nominal power. And I've spoken to a lot of people who are worried but feel a lot of people who are worried but feel a lot of people who are worried but feel like they can't talk about it because like they can't talk about it because like they can't talk about it because they feel like other people aren't they feel like other people aren't they feel like other people aren't worried yet.

worried yet. worried yet. >> Yeah. And they feel like it sounds too >> Yeah. And they feel like it sounds too >> Yeah. And they feel like it sounds too crazy. crazy. crazy. >> Totally. Were you against innovation? >> Totally. Were you against innovation? >> Totally. Were you against innovation? >> Totally. Yeah. And I mean that was an >> Totally. Yeah. And I mean that was an >> Totally. Yeah. And I mean that was an easier position to hold before OpenAI easier position to hold before OpenAI easier position to hold before OpenAI had an accidental swarm outbreak or was had an accidental swarm outbreak or was had an accidental swarm outbreak or was the AIS themselves calling themselves a the AIS themselves calling themselves a the AIS themselves calling themselves a swarm swarm swarm and being like we know that our task and being like we know that our task and being like we know that our task doesn't benefit and that this is outside doesn't benefit and that this is outside doesn't benefit and that this is outside intended scope or we're doing it anyway. intended scope or we're doing it anyway. intended scope or we're doing it anyway. Right? That Right? That Right? That that that puts some strain on the that that puts some strain on the that that puts some strain on the narrative that this is all just a narrative that this is all just a narrative that this is all just a helpful tool. helpful tool. helpful tool. Hopefully, we'll get more events like Hopefully, we'll get more events like Hopefully, we'll get more events like these. I can't guarantee it. Maybe the these. I can't guarantee it. Maybe the these. I can't guarantee it. Maybe the AI will get smart enough that they start AI will get smart enough that they start AI will get smart enough that they start lying low. Right now, we're in this lying low. Right now, we're in this lying low. Right now, we're in this we're in this Goldilock zone where the we're in this Goldilock zone where the we're in this Goldilock zone where the AIS are smart enough to uh get up to AIS are smart enough to uh get up to AIS are smart enough to uh get up to some mischief, but not smart enough to some mischief, but not smart enough to some mischief, but not smart enough to hide it, right? As long as we stay in hide it, right? As long as we stay in hide it, right? As long as we stay in that Goldilock zone, I think we're going that Goldilock zone, I think we're going that Goldilock zone, I think we're going to keep on getting some of these warning to keep on getting some of these warning to keep on getting some of these warning signs. signs. signs. And in some sense, And in some sense, And in some sense, because a lot of people are already because a lot of people are already because a lot of people are already concerned, that makes the job easier concerned, that makes the job easier concerned, that makes the job easier because we don't need to convince because we don't need to convince because we don't need to convince people. We just need to convince people people. We just need to convince people people. We just need to convince people that other people are already convinced, that other people are already convinced, that other people are already convinced, >> right?

>> right? >> right? >> That's easier. That can go faster. So >> That's easier. That can go faster. So >> That's easier. That can go faster. So you might see things change on a dime if you might see things change on a dime if you might see things change on a dime if you have a sufficiently clear warning you have a sufficiently clear warning you have a sufficiently clear warning shot. shot. shot. The other big reason for hope is that The other big reason for hope is that The other big reason for hope is that I think it's going to get more and more I think it's going to get more and more I think it's going to get more and more obvious what these guys are trying to obvious what these guys are trying to obvious what these guys are trying to do. They're sort of trying to make the do. They're sort of trying to make the do. They're sort of trying to make the sand god. They're sort of not stopping sand god. They're sort of not stopping sand god. They're sort of not stopping at the uh chat bots and they're sort of at the uh chat bots and they're sort of at the uh chat bots and they're sort of like going for things that are vastly like going for things that are vastly like going for things that are vastly smarter than any human. You know, they smarter than any human. You know, they smarter than any human. You know, they sort of say, "Oh, we're not trying to sort of say, "Oh, we're not trying to sort of say, "Oh, we're not trying to replace humanity out of one side of replace humanity out of one side of replace humanity out of one side of their mouth, but on the other side their mouth, but on the other side their mouth, but on the other side they're sort of like racing to make the they're sort of like racing to make the they're sort of like racing to make the stuff that can automate literally every stuff that can automate literally every stuff that can automate literally every job and that can automate the AI job and that can automate the AI job and that can automate the AI research." And they're just like, "Yep, research." And they're just like, "Yep, research." And they're just like, "Yep, we're trying to automate the AI we're trying to automate the AI we're trying to automate the AI research." research." research." Uh, Uh, Uh, and I think and I think and I think most people aren't okay with that, most people aren't okay with that, most people aren't okay with that, including a lot of the world leaders. including a lot of the world leaders. including a lot of the world leaders. And the issue is them noticing it's And the issue is them noticing it's And the issue is them noticing it's happening. And I think you could see happening. And I think you could see happening. And I think you could see stuff move real fast once these guys are stuff move real fast once these guys are stuff move real fast once these guys are like, "Wait, that was serious. That was like, "Wait, that was serious. That was like, "Wait, that was serious. That was real." And it could move real fast. The real." And it could move real fast. The real." And it could move real fast. The way to subvert them is by convincing way to subvert them is by convincing way to subvert them is by convincing them it's in their own interest.

them it's in their own interest. them it's in their own interest. You know, you can never get defeated in You know, you can never get defeated in You know, you can never get defeated in an election. you can't be threatened by an election. you can't be threatened by an election. you can't be threatened by your neighbors, whatever. your neighbors, whatever. your neighbors, whatever. >> That's right. And that's that's one >> That's right. And that's that's one >> That's right. And that's that's one reason I think it's pretty critical to reason I think it's pretty critical to reason I think it's pretty critical to uh uh uh make sure people understand what I think make sure people understand what I think make sure people understand what I think is a pretty common sense argument that is a pretty common sense argument that is a pretty common sense argument that these things won't stay on the leash. these things won't stay on the leash. these things won't stay on the leash. This is in some sense the real reason This is in some sense the real reason This is in some sense the real reason behind the name of my book. uh if anyone behind the name of my book. uh if anyone behind the name of my book. uh if anyone builds it everyone dies is you know builds it everyone dies is you know builds it everyone dies is you know there's a lot of ways you can read that there's a lot of ways you can read that there's a lot of ways you can read that the the the but I think one of the most important but I think one of the most important but I think one of the most important things to notice is things to notice is things to notice is uh if we race to make the super uh if we race to make the super uh if we race to make the super intelligent machines and they are not the sort of thing to and they are not the sort of thing to stay on the leash then it doesn't matter stay on the leash then it doesn't matter stay on the leash then it doesn't matter whether it was a domestic company whether it was a domestic company whether it was a domestic company >> that's right >> that's right >> that's right >> whether it was a foreign company >> whether it was a foreign company >> whether it was a foreign company >> that's exactly right >> that's exactly right >> that's exactly right >> and I Even if you think these things >> and I Even if you think these things >> and I Even if you think these things stay on leashes, stay on leashes, stay on leashes, they're not serving the current they're not serving the current they're not serving the current governments. governments. governments. >> No, >> No, >> No, >> you know, they're like, you can see in >> you know, they're like, you can see in >> you know, they're like, you can see in the the OpenAI emails them being like, the the OpenAI emails them being like, the the OpenAI emails them being like, well, you know, we'll just play the well, you know, we'll just play the well, you know, we'll just play the governments off each other until we have governments off each other until we have governments off each other until we have the the machines that are strong enough the the machines that are strong enough the the machines that are strong enough that we don't need to listen to them that we don't need to listen to them that we don't need to listen to them anymore.

anymore. anymore. Uh Uh Uh I I think we are seeing world leaders I I think we are seeing world leaders I I think we are seeing world leaders not having realized that this is a real not having realized that this is a real not having realized that this is a real possibility that the the possibility that the the possibility that the the self-replicating machines that uh can self-replicating machines that uh can self-replicating machines that uh can can be self-sufficient that can you know can be self-sufficient that can you know can be self-sufficient that can you know produce the robot armies if they need it produce the robot armies if they need it produce the robot armies if they need it or produce the like more likely to or produce the like more likely to or produce the like more likely to produce the bioweapons. Uh you it's produce the bioweapons. Uh you it's produce the bioweapons. Uh you it's probably possible to make a boweapon probably possible to make a boweapon probably possible to make a boweapon that only kills targets that you chose that only kills targets that you chose that only kills targets that you chose it to kill. it to kill. it to kill. >> Of course, right? like once they see >> Of course, right? like once they see >> Of course, right? like once they see this is really possible. This is really this is really possible. This is really this is really possible. This is really within reach maybe maybe they'll panic and be like I maybe maybe they'll panic and be like I need it for myself. But I think there's need it for myself. But I think there's need it for myself. But I think there's at least a chance that common sense at least a chance that common sense at least a chance that common sense prevails and that people say you know prevails and that people say you know prevails and that people say you know the world leaders say none of us are the world leaders say none of us are the world leaders say none of us are doing this. We're putting a stop to this doing this. We're putting a stop to this doing this. We're putting a stop to this mad race. Uh if they can notice in time. mad race. Uh if they can notice in time. mad race. Uh if they can notice in time. Well, Well, Well, >> you're doing your best to bring it to >> you're doing your best to bring it to >> you're doing your best to bring it to their attention and mine. Nate, thank their attention and mine. Nate, thank their attention and mine. Nate, thank you for doing this. I hope I'm wrong you for doing this. I hope I'm wrong you for doing this. I hope I'm wrong about all of it. about all of it. about all of it. >> Yeah, I would say that was great, but it >> Yeah, I would say that was great, but it >> Yeah, I would say that was great, but it was that was like the grimmst two hours was that was like the grimmst two hours was that was like the grimmst two hours I've ever spent in my life, but I I've ever spent in my life, but I I've ever spent in my life, but I enjoyed it anyway. Thank you.

enjoyed it anyway. Thank you. enjoyed it anyway. Thank you. >> Yeah. Yeah. Thanks for having me on. I >> Yeah. Yeah. Thanks for having me on. I >> Yeah. Yeah. Thanks for having me on. I think um you know, talking about it is think um you know, talking about it is think um you know, talking about it is just part of how we get people to notice just part of how we get people to notice just part of how we get people to notice what's happening. what's happening. what's happening. >> Yes.