The New Primitives: Building AI Native Software — Kwindla Kramer, Daily

>> Good morning. I know a lot of you in >> Good morning. I know a lot of you in this room. It's great to see you. this room. It's great to see you. this room. It's great to see you. Welcome to the voice track at AI Welcome to the voice track at AI Welcome to the voice track at AI Engineer World's Fair. Engineer World's Fair. Engineer World's Fair. For those of you who don't know me, my For those of you who don't know me, my For those of you who don't know me, my name is Quinn La Holman Cramer. name is Quinn La Holman Cramer. name is Quinn La Holman Cramer. I work at a company called Daily. We I work at a company called Daily. We I work at a company called Daily. We make developer infrastructure for make developer infrastructure for make developer infrastructure for real-time audio, video, and AI. And real-time audio, video, and AI. And real-time audio, video, and AI. And we're the team behind Pipe Cat, which is we're the team behind Pipe Cat, which is we're the team behind Pipe Cat, which is the most widely used framework for the most widely used framework for the most widely used framework for building voice agents today. Pipe Cat is building voice agents today. Pipe Cat is building voice agents today. Pipe Cat is open source and vendor neutral. open source and vendor neutral. open source and vendor neutral. It's used by companies like AWS and It's used by companies like AWS and It's used by companies like AWS and Nvidia and Anthropic and thousands of Nvidia and Anthropic and thousands of Nvidia and Anthropic and thousands of startups and scale-ups and enterprises. startups and scale-ups and enterprises. startups and scale-ups and enterprises. And today I'm going to talk about what And today I'm going to talk about what And today I'm going to talk about what kind of agents we're building today, kind of agents we're building today, kind of agents we're building today, including voice agents, but not just including voice agents, but not just including voice agents, but not just voice agents, and what I'm interested in voice agents, and what I'm interested in voice agents, and what I'm interested in building next. And I'm going to try to building next. And I'm going to try to building next. And I'm going to try to put all this in the context of the put all this in the context of the put all this in the context of the roughly 80-year history of digital roughly 80-year history of digital roughly 80-year history of digital computing so far. computing so far. computing so far. So, we've got a lot to cover. We're So, we've got a lot to cover. We're So, we've got a lot to cover. We're going to go fast. going to go fast. going to go fast. But we're going to start in 1945 with an But we're going to start in 1945 with an But we're going to start in 1945 with an essay called As We May Think, written by essay called As We May Think, written by essay called As We May Think, written by an engineer, an academic, a civil an engineer, an academic, a civil an engineer, an academic, a civil servant named Vannevar Bush.

servant named Vannevar Bush. servant named Vannevar Bush. Bush deeply understood technologies Bush deeply understood technologies Bush deeply understood technologies ranging from analog computers to ranging from analog computers to ranging from analog computers to photography to radio to radar. photography to radio to radar. photography to radio to radar. As We May Think is a extraordinary piece As We May Think is a extraordinary piece As We May Think is a extraordinary piece of writing. of writing. of writing. The essay predicts the development of, The essay predicts the development of, The essay predicts the development of, among other things, document display on among other things, document display on among other things, document display on a screen and document scanning and OCR a screen and document scanning and OCR a screen and document scanning and OCR and speech-to-text and text-to-speech and speech-to-text and text-to-speech and speech-to-text and text-to-speech and programming languages and hypertext and programming languages and hypertext and programming languages and hypertext and search engines and data networks. and search engines and data networks. and search engines and data networks. Something like the GoPro camera, Something like the GoPro camera, Something like the GoPro camera, something weirdly like the Amazon Kindle something weirdly like the Amazon Kindle something weirdly like the Amazon Kindle store, and voice interfaces and brain store, and voice interfaces and brain store, and voice interfaces and brain computer interfaces. computer interfaces. computer interfaces. And I've been thinking a lot about As We And I've been thinking a lot about As We And I've been thinking a lot about As We May Think lately because Bush wrote this May Think lately because Bush wrote this May Think lately because Bush wrote this essay right at the very beginning of the essay right at the very beginning of the essay right at the very beginning of the computing age. computing age. computing age. And I think it feels to most of us like And I think it feels to most of us like And I think it feels to most of us like we're working right at the beginning of we're working right at the beginning of we're working right at the beginning of a new age, the intelligence age. So, a new age, the intelligence age. So, a new age, the intelligence age. So, what will we build? what will we build? what will we build? Well, at the moment we're building Well, at the moment we're building Well, at the moment we're building agents and we're having a lot of fun agents and we're having a lot of fun agents and we're having a lot of fun doing it. doing it. doing it. And a lot of the AI engineering work And a lot of the AI engineering work And a lot of the AI engineering work we're all talking about this week is we're all talking about this week is we're all talking about this week is focused on building a full coherent focused on building a full coherent focused on building a full coherent software stack for AI agents.

software stack for AI agents. software stack for AI agents. Here is Satya Nadella talking a couple Here is Satya Nadella talking a couple Here is Satya Nadella talking a couple weeks ago on a crossover episode of the weeks ago on a crossover episode of the weeks ago on a crossover episode of the No Priors and Latent Space Pod about the No Priors and Latent Space Pod about the No Priors and Latent Space Pod about the challenges of building agents in 2026. challenges of building agents in 2026. challenges of building agents in 2026. >> That's sort of >> That's sort of >> That's sort of >> That's right. So, so in some sense you >> That's right. So, so in some sense you >> That's right. So, so in some sense you kind of want to harness to define the kind of want to harness to define the kind of want to harness to define the models, the the data, uh and the tools. models, the the data, uh and the tools. models, the the data, uh and the tools. And so that you have a loop across those And so that you have a loop across those And so that you have a loop across those three. And so what we are trying to three. And so what we are trying to three. And so what we are trying to first of all make sure is each of our first of all make sure is each of our first of all make sure is each of our products that we build, right? Whether products that we build, right? Whether products that we build, right? Whether it's GitHub Copilot or the security it's GitHub Copilot or the security it's GitHub Copilot or the security copilot the stuff we showed with M dash copilot the stuff we showed with M dash copilot the stuff we showed with M dash or even the discovery for science, it or even the discovery for science, it or even the discovery for science, it doesn't matter. All of them are doesn't matter. All of them are doesn't matter. All of them are multimodal harnesses um with tools multimodal harnesses um with tools multimodal harnesses um with tools access so that you can do this access so that you can do this access so that you can do this progressive uh disclosure of tools even progressive uh disclosure of tools even progressive uh disclosure of tools even so that they're token efficient. Uh and so that they're token efficient. Uh and so that they're token efficient. Uh and then you're feeding it with very rich then you're feeding it with very rich then you're feeding it with very rich context. context. context. >> So, if you were here last year at AI >> So, if you were here last year at AI >> So, if you were here last year at AI Engineer World's Fair, you could draw a Engineer World's Fair, you could draw a Engineer World's Fair, you could draw a through line from the things we were through line from the things we were through line from the things we were talking about last year to loops and talking about last year to loops and talking about last year to loops and tool calls and context engineering and tool calls and context engineering and tool calls and context engineering and the stuff we're focused on this year the stuff we're focused on this year the stuff we're focused on this year to some emerging ideas. Uh you can hear to some emerging ideas. Uh you can hear to some emerging ideas. Uh you can hear that in Nadella's clip just there. I that in Nadella's clip just there. I that in Nadella's clip just there. I think of this is kind of agents plus think of this is kind of agents plus think of this is kind of agents plus plus, like multi-model harnesses and plus, like multi-model harnesses and plus, like multi-model harnesses and software copilot embedded in every software copilot embedded in every software copilot embedded in every single piece of software, organization single piece of software, organization single piece of software, organization level harnesses.

level harnesses. level harnesses. So, how do we go from agents to agents So, how do we go from agents to agents So, how do we go from agents to agents plus plus to the next thing beyond plus plus to the next thing beyond plus plus to the next thing beyond agents. agents. agents. Well, the last time we had this kind of Well, the last time we had this kind of Well, the last time we had this kind of massive change in how we write software massive change in how we write software massive change in how we write software and what we write software for and to do and what we write software for and to do and what we write software for and to do was the early days of the World Wide was the early days of the World Wide was the early days of the World Wide Web. And I was around for the early days Web. And I was around for the early days Web. And I was around for the early days of the World Wide Web. I was a baby of the World Wide Web. I was a baby of the World Wide Web. I was a baby programmer in 1995 and the thing we programmer in 1995 and the thing we programmer in 1995 and the thing we talked about all the time in 1995, the talked about all the time in 1995, the talked about all the time in 1995, the way we talk about agents today, is web way we talk about agents today, is web way we talk about agents today, is web pages. pages. pages. I spent a lot of time writing HTML by I spent a lot of time writing HTML by I spent a lot of time writing HTML by hand and building web server software in hand and building web server software in hand and building web server software in C and indexing and search software in C C and indexing and search software in C C and indexing and search software in C and authoring tooling and management and authoring tooling and management and authoring tooling and management infrastructure for web pages in Pearl. infrastructure for web pages in Pearl. infrastructure for web pages in Pearl. I was as excited about HTML in 1995 as I I was as excited about HTML in 1995 as I I was as excited about HTML in 1995 as I am about agents today. am about agents today. am about agents today. And the web page is still with us and And the web page is still with us and And the web page is still with us and it's still important and useful. But it's still important and useful. But it's still important and useful. But today we talk a lot more about web today we talk a lot more about web today we talk a lot more about web applications and native mobile applications and native mobile applications and native mobile applications than we talk about web applications than we talk about web applications than we talk about web pages. pages. pages. So, just like we went from web pages to So, just like we went from web pages to So, just like we went from web pages to full-blown web and native mobile, full-blown web and native mobile, full-blown web and native mobile, clearly we're going to chart a path to a clearly we're going to chart a path to a clearly we're going to chart a path to a new fully AI native software that comes new fully AI native software that comes new fully AI native software that comes after agents and agents plus plus.

after agents and agents plus plus. after agents and agents plus plus. So, let's keep going back in time to in So, let's keep going back in time to in So, let's keep going back in time to in order to think about this future. order to think about this future. order to think about this future. Here's a timeline Vannevar Bush lays out Here's a timeline Vannevar Bush lays out Here's a timeline Vannevar Bush lays out and as we may think, he talks about the and as we may think, he talks about the and as we may think, he talks about the abacus, which was both an immensely abacus, which was both an immensely abacus, which was both an immensely useful device for doing practical useful device for doing practical useful device for doing practical everyday mathematical calculations and everyday mathematical calculations and everyday mathematical calculations and also an incredibly important theoretical also an incredibly important theoretical also an incredibly important theoretical tool that led to ideas like numeric tool that led to ideas like numeric tool that led to ideas like numeric place value and the concept of zero. place value and the concept of zero. place value and the concept of zero. And Bush talks about the massive jump And Bush talks about the massive jump And Bush talks about the massive jump from the abacus to the state-of-the-art from the abacus to the state-of-the-art from the abacus to the state-of-the-art electromechanical keyboard calculating electromechanical keyboard calculating electromechanical keyboard calculating machines that he had in 1945. machines that he had in 1945. machines that he had in 1945. And then he posits that we're about or And then he posits that we're about or And then he posits that we're about or he is about to witness and help create witness and help create an equally large leap to what he calls an equally large leap to what he calls an equally large leap to what he calls the arithmetical machine. And then he the arithmetical machine. And then he the arithmetical machine. And then he goes a step even further than that and goes a step even further than that and goes a step even further than that and he invents or designs in that essay a he invents or designs in that essay a he invents or designs in that essay a device he calls the memex. device he calls the memex. device he calls the memex. And we have a little bit of an advantage And we have a little bit of an advantage And we have a little bit of an advantage over over Bush in 1945. We've seen 80 over over Bush in 1945. We've seen 80 over over Bush in 1945. We've seen 80 years of computing play out. So, we can years of computing play out. So, we can years of computing play out. So, we can modify his timeline a little bit. We can modify his timeline a little bit. We can modify his timeline a little bit. We can go from the abacus to the stored program go from the abacus to the stored program go from the abacus to the stored program computer computer computer to 40 years later the personal computer to 40 years later the personal computer to 40 years later the personal computer and 40 years after that this AI agents and 40 years after that this AI agents and 40 years after that this AI agents era that we're all collectively helping era that we're all collectively helping era that we're all collectively helping to invent and create and bring into to invent and create and bring into to invent and create and bring into being.

being. being. So, the question for me is what did we So, the question for me is what did we So, the question for me is what did we build to go from those very first build to go from those very first build to go from those very first digital computers in the 1940s to the digital computers in the 1940s to the digital computers in the 1940s to the personal computer in the 1980s? personal computer in the 1980s? personal computer in the 1980s? Well, in the 1950s the big job was to Well, in the 1950s the big job was to Well, in the 1950s the big job was to figure out more effective ways of figure out more effective ways of figure out more effective ways of transmitting human intent to these new transmitting human intent to these new transmitting human intent to these new computing machines. computing machines. computing machines. We built the first programming We built the first programming We built the first programming languages. We wrote the first compilers. languages. We wrote the first compilers. languages. We wrote the first compilers. And the the theoretical underpinnings And the the theoretical underpinnings And the the theoretical underpinnings here were figuring out how to combine here were figuring out how to combine here were figuring out how to combine the elegance of mathematical formalisms the elegance of mathematical formalisms the elegance of mathematical formalisms with something a little bit more like with something a little bit more like with something a little bit more like natural language. natural language. natural language. And then building on that in the 1960s And then building on that in the 1960s And then building on that in the 1960s the challenge was to make these machines the challenge was to make these machines the challenge was to make these machines interactive. Make these machines capable interactive. Make these machines capable interactive. Make these machines capable of a two-way dialogue with humans. of a two-way dialogue with humans. of a two-way dialogue with humans. The '60s also saw the birth of graphical The '60s also saw the birth of graphical The '60s also saw the birth of graphical programming with systems like Ivan programming with systems like Ivan programming with systems like Ivan Sutherland's Sketchpad. Sutherland's Sketchpad. Sutherland's Sketchpad. And the '60s were an amazing era for And the '60s were an amazing era for And the '60s were an amazing era for science fiction. science fiction. science fiction. Even though almost nobody had access to Even though almost nobody had access to Even though almost nobody had access to a computer a computer a computer the computer became a big part of the the computer became a big part of the the computer became a big part of the popular imagination. The idea of a popular imagination. The idea of a popular imagination. The idea of a computer really resonated with people computer really resonated with people computer really resonated with people and ideas matter.

and ideas matter. and ideas matter. For example, here is the idea of the For example, here is the idea of the For example, here is the idea of the computer in Star Trek. computer in Star Trek. computer in Star Trek. >> Put her on record. >> Put her on record. >> Put her on record. >> Recording. Captain's log supplemental. Captain's log supplemental. Engineering officer Scott informs warp Engineering officer Scott informs warp Engineering officer Scott informs warp engines engines engines Can be made operational and Can be made operational and Can be made operational and re-energized. re-energized. re-energized. >> Computed and recorded, dear. >> Computed and recorded, dear. >> Computed and recorded, dear. >> Computer, you will not address me in >> Computer, you will not address me in >> Computer, you will not address me in that manner. that manner. that manner. Computer. Computer. Computer. >> Computed, dear. >> I love the background sound of punch >> I love the background sound of punch cards going through a punch card reader. cards going through a punch card reader. cards going through a punch card reader. So, like you know the computer is So, like you know the computer is So, like you know the computer is working even though it's talking to you working even though it's talking to you working even though it's talking to you about what it's actually computing. about what it's actually computing. about what it's actually computing. There were of course a bunch of other There were of course a bunch of other There were of course a bunch of other talking computers in in science fiction talking computers in in science fiction talking computers in in science fiction of the '60s and the next year after of the '60s and the next year after of the '60s and the next year after this, the Kubrick movie that was a this, the Kubrick movie that was a this, the Kubrick movie that was a interpretation of Arthur C. Clarke's interpretation of Arthur C. Clarke's interpretation of Arthur C. Clarke's 2001: A Space Odyssey had the HAL 9000 2001: A Space Odyssey had the HAL 9000 2001: A Space Odyssey had the HAL 9000 computer. This is a much, much more computer. This is a much, much more computer. This is a much, much more dystopian view of a talking computer dystopian view of a talking computer dystopian view of a talking computer than the Star Trek computers.

than the Star Trek computers. than the Star Trek computers. And by the 1970s, computers had become And by the 1970s, computers had become And by the 1970s, computers had become powerful enough that the next big job powerful enough that the next big job powerful enough that the next big job was designing abstractions that could was designing abstractions that could was designing abstractions that could scale to much larger amounts of data and scale to much larger amounts of data and scale to much larger amounts of data and much more powerful computing substrates. much more powerful computing substrates. much more powerful computing substrates. We got relational databases, which We got relational databases, which We got relational databases, which introduced new theoretical underpinnings introduced new theoretical underpinnings introduced new theoretical underpinnings for data manipulation. for data manipulation. for data manipulation. And we got declarative languages, which And we got declarative languages, which And we got declarative languages, which leveraged those new theoretical leveraged those new theoretical leveraged those new theoretical insights. And programming languages in insights. And programming languages in insights. And programming languages in general continued to evolve in what to general continued to evolve in what to general continued to evolve in what to me at least are really amazing ways. We me at least are really amazing ways. We me at least are really amazing ways. We got Smalltalk and object-oriented got Smalltalk and object-oriented got Smalltalk and object-oriented programming in the '70s. programming in the '70s. programming in the '70s. And all of this set the stage for the And all of this set the stage for the And all of this set the stage for the personal computer in the 1980s. The personal computer in the 1980s. The personal computer in the 1980s. The Macintosh shipped in 1984. Windows 1.0 Macintosh shipped in 1984. Windows 1.0 Macintosh shipped in 1984. Windows 1.0 shipped in 1985. And Microsoft's mission shipped in 1985. And Microsoft's mission shipped in 1985. And Microsoft's mission statement was a computer on every desk statement was a computer on every desk statement was a computer on every desk and in every home. And incredibly, and in every home. And incredibly, and in every home. And incredibly, Microsoft delivered on that mission Microsoft delivered on that mission Microsoft delivered on that mission statement. statement. statement. And we got a computer on every desk and And we got a computer on every desk and And we got a computer on every desk and in every home because these new personal in every home because these new personal in every home because these new personal computers delivered real, amazing, computers delivered real, amazing, computers delivered real, amazing, tangible benefits. tangible benefits. tangible benefits. Take VisiCalc, for example, which was Take VisiCalc, for example, which was Take VisiCalc, for example, which was the first spreadsheet program. the first spreadsheet program. the first spreadsheet program. A truly new abstraction for doing A truly new abstraction for doing A truly new abstraction for doing computation, numerical computing, computation, numerical computing, computation, numerical computing, two-dimensional, interactive, so durable two-dimensional, interactive, so durable two-dimensional, interactive, so durable and so useful that probably most of us and so useful that probably most of us and so useful that probably most of us in this room use a direct descendant of in this room use a direct descendant of in this room use a direct descendant of VisiCalc regularly, Google Google Sheets VisiCalc regularly, Google Google Sheets VisiCalc regularly, Google Google Sheets or Microsoft Excel or whatever.

Or put another way, this was a Or put another way, this was a spreadsheet in 1957. spreadsheet in 1957. spreadsheet in 1957. And this was a spreadsheet in 1985. And this was a spreadsheet in 1985. And this was a spreadsheet in 1985. And I think a lot about VisiCalc these And I think a lot about VisiCalc these And I think a lot about VisiCalc these days too because I think VisiCalc is an days too because I think VisiCalc is an days too because I think VisiCalc is an example of how transformative new example of how transformative new example of how transformative new technologies can be in the way of technologies can be in the way of technologies can be in the way of delivering a capability that used to delivering a capability that used to delivering a capability that used to require a lot of specialized people and require a lot of specialized people and require a lot of specialized people and specialized knowledge and making it specialized knowledge and making it specialized knowledge and making it generally accessible. And I think generally accessible. And I think generally accessible. And I think VisiCalc is a potentially a VisiCalc is a potentially a VisiCalc is a potentially a counter-argument to counter-argument to counter-argument to the argument or the fear or the concern the argument or the fear or the concern the argument or the fear or the concern that AI is going to lead to mass that AI is going to lead to mass that AI is going to lead to mass unemployment. Because VisiCalc didn't unemployment. Because VisiCalc didn't unemployment. Because VisiCalc didn't put accountants out of business. put accountants out of business. put accountants out of business. Instead, it made much much much much Instead, it made much much much much Instead, it made much much much much more accounting-like work possible. And more accounting-like work possible. And more accounting-like work possible. And it made new categories of work possible it made new categories of work possible it made new categories of work possible that we couldn't even really conceive of that we couldn't even really conceive of that we couldn't even really conceive of when a spreadsheet or doing when a spreadsheet or doing when a spreadsheet or doing a screen's worth of calculations as we a screen's worth of calculations as we a screen's worth of calculations as we think about it today took a roomful of think about it today took a roomful of think about it today took a roomful of people. people. people. So if we were here in the Moscone Center So if we were here in the Moscone Center So if we were here in the Moscone Center in 1985 in 1985 in 1985 and these two interfaces, the Macintosh and these two interfaces, the Macintosh and these two interfaces, the Macintosh System 2 and Windows 1.0 were state of System 2 and Windows 1.0 were state of System 2 and Windows 1.0 were state of the art, what would we have said the the art, what would we have said the the art, what would we have said the world would look like in 10 or 20 or 30 world would look like in 10 or 20 or 30 world would look like in 10 or 20 or 30 or 40 years?

or 40 years? or 40 years? Well, we actually have a really great Well, we actually have a really great Well, we actually have a really great example of a prediction from that time. example of a prediction from that time. example of a prediction from that time. Like as we may think another famous Like as we may think another famous Like as we may think another famous document in the history of human document in the history of human document in the history of human computer interaction, concept video from computer interaction, concept video from computer interaction, concept video from Apple made in 1987 called Knowledge Apple made in 1987 called Knowledge Apple made in 1987 called Knowledge Navigator. This is very much worth Navigator. This is very much worth Navigator. This is very much worth tracking down online and watching all of tracking down online and watching all of tracking down online and watching all of if you haven't seen it. I'm just going if you haven't seen it. I'm just going if you haven't seen it. I'm just going to play about 20 seconds from the to play about 20 seconds from the to play about 20 seconds from the middle. >> You have three messages. >> You have three messages. Your graduate research team in Your graduate research team in Your graduate research team in Guatemala, just checking in. Guatemala, just checking in. Guatemala, just checking in. Robert Jordan, a second semester junior, Robert Jordan, a second semester junior, Robert Jordan, a second semester junior, requesting a second extension on his requesting a second extension on his requesting a second extension on his term paper term paper term paper and your mother reminding you about your and your mother reminding you about your and your mother reminding you about your father's father's father's >> surprise birthday party next Sunday. >> So, the video shows a foldable tablet, a >> So, the video shows a foldable tablet, a touchscreen interface, [clears throat] a touchscreen interface, [clears throat] a touchscreen interface, [clears throat] a conversational voice assistant with a conversational voice assistant with a conversational voice assistant with a really strong personality, access to really strong personality, access to really strong personality, access to both global and personal information, both global and personal information, both global and personal information, real-time video generation, real-time real-time video generation, real-time real-time video generation, real-time computer vision, seamless video call computer vision, seamless video call computer vision, seamless video call integration, delegation of complex tasks integration, delegation of complex tasks integration, delegation of complex tasks for autonomous execution, and what we for autonomous execution, and what we for autonomous execution, and what we might call today continual learning.

might call today continual learning. might call today continual learning. And it's really, really clearly And it's really, really clearly And it's really, really clearly influenced by AS we may think, but it's influenced by AS we may think, but it's influenced by AS we may think, but it's also quite different. It really is also quite different. It really is also quite different. It really is updated for 40 years of progress, and it updated for 40 years of progress, and it updated for 40 years of progress, and it really does sort of presage this AI really does sort of presage this AI really does sort of presage this AI agent era we're in now in a way that agent era we're in now in a way that agent era we're in now in a way that Vannevar Bush's Memex didn't and maybe Vannevar Bush's Memex didn't and maybe Vannevar Bush's Memex didn't and maybe couldn't. couldn't. couldn't. The Knowledge Navigator video divides The Knowledge Navigator video divides The Knowledge Navigator video divides our timeline, I think, quite neatly in our timeline, I think, quite neatly in our timeline, I think, quite neatly in half, and hold that thought cuz we're half, and hold that thought cuz we're half, and hold that thought cuz we're going to come back to it. going to come back to it. going to come back to it. The 1990s were about the network, first The 1990s were about the network, first The 1990s were about the network, first local area networks and dial-up, and local area networks and dial-up, and local area networks and dial-up, and then the internet and the web, then the internet and the web, then the internet and the web, and with the benefit of hindsight, I now and with the benefit of hindsight, I now and with the benefit of hindsight, I now think that the single most important think that the single most important think that the single most important thing about the web was that it was thing about the web was that it was thing about the web was that it was multimodal from the very beginning. multimodal from the very beginning. multimodal from the very beginning. More even than the GUIs of the 1980s, More even than the GUIs of the 1980s, More even than the GUIs of the 1980s, the web anticipated that text and audio the web anticipated that text and audio the web anticipated that text and audio and video and data were not different and video and data were not different and video and data were not different things to be used in different programs, things to be used in different programs, things to be used in different programs, they belonged together. And in a real they belonged together. And in a real they belonged together. And in a real sense, the web was an attempt, and a sense, the web was an attempt, and a sense, the web was an attempt, and a conscious attempt on the part of a lot conscious attempt on the part of a lot conscious attempt on the part of a lot of people building the web to make that of people building the web to make that of people building the web to make that Knowledge Navigator video real.

Knowledge Navigator video real. Knowledge Navigator video real. Then in the first decade of the new Then in the first decade of the new Then in the first decade of the new millennium, the big job was to make all millennium, the big job was to make all millennium, the big job was to make all of this computing stuff mobile and of this computing stuff mobile and of this computing stuff mobile and continually connected, to put this new continually connected, to put this new continually connected, to put this new multimodal networked computer multimodal networked computer multimodal networked computer in your pocket, literally, to give a in your pocket, literally, to give a in your pocket, literally, to give a supercomputer to everybody in the world supercomputer to everybody in the world supercomputer to everybody in the world that they could carry around in their that they could carry around in their that they could carry around in their hand. hand. hand. And as with the 1960s, there was an And as with the 1960s, there was an And as with the 1960s, there was an efflorescence of like futurism on screen efflorescence of like futurism on screen efflorescence of like futurism on screen in the first few years of the new in the first few years of the new in the first few years of the new millennium. And I think it was because millennium. And I think it was because millennium. And I think it was because computers you could carry around with computers you could carry around with computers you could carry around with you and cameras everywhere and a kind of you and cameras everywhere and a kind of you and cameras everywhere and a kind of Moore's law for pixels making screens Moore's law for pixels making screens Moore's law for pixels making screens super cheap really gave us a chance to super cheap really gave us a chance to super cheap really gave us a chance to think through what we thought the future think through what we thought the future think through what we thought the future would look like in a new way. would look like in a new way. would look like in a new way. A lot of stuff we could almost but not A lot of stuff we could almost but not A lot of stuff we could almost but not quite build was cohering in the minds of quite build was cohering in the minds of quite build was cohering in the minds of people working on these machines. people working on these machines. people working on these machines. And the best and most famous Hollywood And the best and most famous Hollywood And the best and most famous Hollywood computers from that era were created by computers from that era were created by computers from that era were created by John Underkoffler [clears throat] for John Underkoffler [clears throat] for John Underkoffler [clears throat] for the films Minority Report and Iron Man. the films Minority Report and Iron Man. the films Minority Report and Iron Man. Here's Minority Report from 2002.

>> It's no longer there. >> It's no longer there. >> Time frame? >> Time frame? >> Time frame? >> 13 minutes. >> 13 minutes. >> 13 minutes. >> Hey Chief, >> Hey Chief, >> Hey Chief, investigator from the Feds here. investigator from the Feds here. investigator from the Feds here. >> Yeah, I don't need some twink from the >> Yeah, I don't need some twink from the >> Yeah, I don't need some twink from the Fed poking around right now. Fed poking around right now. Fed poking around right now. >> John, I wrote it down on your calendar. >> John, I wrote it down on your calendar. >> John, I wrote it down on your calendar. I left you a message at your house. I left you a message at your house. I left you a message at your house. >> Check in with the Favors ahead of Ford >> Check in with the Favors ahead of Ford >> Check in with the Favors ahead of Ford and see if the neighbors knew where they and see if the neighbors knew where they and see if the neighbors knew where they went. Check all relations. went. Check all relations. went. Check all relations. >> Check the neighbors and relations. >> Check the neighbors and relations. >> Check the neighbors and relations. >> But John >> But John >> But John >> It's actually >> It's actually >> It's actually >> Just get him some coffee. Tell him some >> Just get him some coffee. Tell him some >> Just get him some coffee. Tell him some stories how I save [music] your ass stories how I save [music] your ass stories how I save [music] your ass every day and you can't do without me. every day and you can't do without me. every day and you can't do without me. >> I got coffee. Thank you. >> I got coffee. Thank you. >> I got coffee. Thank you. >> Danny Witwer. >> Danny Witwer. >> Danny Witwer. Twink from the Fed. Twink from the Fed. Twink from the Fed. Oops. Gone. Oops. Gone. Oops. Gone. >> So the gestural interface in Minority >> So the gestural interface in Minority >> So the gestural interface in Minority Report was implemented on screen as Report was implemented on screen as Report was implemented on screen as special effects, but it was actually special effects, but it was actually special effects, but it was actually based on John's PhD work at the MIT based on John's PhD work at the MIT based on John's PhD work at the MIT Media Lab. In a real sense, this was Media Lab. In a real sense, this was Media Lab. In a real sense, this was real technology. real technology. real technology. John had brought the UI out of the small John had brought the UI out of the small John had brought the UI out of the small screen and into the world with us in a screen and into the world with us in a screen and into the world with us in a bunch of really interesting and lovely bunch of really interesting and lovely bunch of really interesting and lovely ways. ways. ways. John also consulted on Iron Man, which John also consulted on Iron Man, which John also consulted on Iron Man, which is a very different view of the future is a very different view of the future is a very different view of the future than you Minority Report, which was than you Minority Report, which was than you Minority Report, which was Spielberg working in like the American Spielberg working in like the American Spielberg working in like the American Kubrick dystopian tradition. Iron Man is Kubrick dystopian tradition. Iron Man is Kubrick dystopian tradition. Iron Man is really squarely in that Star Trek goofy really squarely in that Star Trek goofy really squarely in that Star Trek goofy futurist tradition. But I think you futurist tradition. But I think you futurist tradition. But I think you could see the common elements in the UI could see the common elements in the UI could see the common elements in the UI depicted on screen. It's still from the depicted on screen. It's still from the depicted on screen. It's still from the same era.

>> Wake up, Daddy Sean. >> Wake up, Daddy Sean. >> Welcome home, sir. [music] >> Welcome home, sir. [music] >> Welcome home, sir. [music] Congratulations on the opening Congratulations on the opening Congratulations on the opening ceremonies. They were such a success. As ceremonies. They were such a success. As ceremonies. They were such a success. As was your summit hearing. was your summit hearing. was your summit hearing. >> [music] >> [music] >> [music] >> And may I say how refreshing it is to >> And may I say how refreshing it is to >> And may I say how refreshing it is to finally you in a video with your finally you in a video with your finally you in a video with your clothing on, sir. You! You! I swear to god, I'll dismantle you or I swear to god, I'll dismantle you or I swear to god, I'll dismantle you or short your motherboard. I'll turn you short your motherboard. I'll turn you short your motherboard. I'll turn you into a wine rack. >> I co-founded a startup with John in 2006 >> I co-founded a startup with John in 2006 to make the Minority Report interface to make the Minority Report interface to make the Minority Report interface into a commercial product. This is our into a commercial product. This is our into a commercial product. This is our demo reel from 2012, 6 years into that demo reel from 2012, 6 years into that demo reel from 2012, 6 years into that work. work. work. >> [music] >> This long project to build the

>> This long project to build the multi-modal, multi-device, multi-screen, multi-modal, multi-device, multi-screen, multi-modal, multi-device, multi-screen, multi-player, ubiquitously connected multi-player, ubiquitously connected multi-player, ubiquitously connected computer is still what I'm working on 15 computer is still what I'm working on 15 computer is still what I'm working on 15 years later. years later. years later. In 2010s, we built out the cloud, which In 2010s, we built out the cloud, which In 2010s, we built out the cloud, which laid the groundwork for the laid the groundwork for the laid the groundwork for the infrastructure and data centers and data infrastructure and data centers and data infrastructure and data centers and data capacity we would need to scale up AI capacity we would need to scale up AI capacity we would need to scale up AI training and inference, training and inference, training and inference, which brings us to now. which brings us to now. which brings us to now. We're building agents. We're building agents. We're building agents. And we're starting to think about agents And we're starting to think about agents And we're starting to think about agents plus plus. plus plus. plus plus. But I think we can also start to think But I think we can also start to think But I think we can also start to think about the next thing, the AI native about the next thing, the AI native about the next thing, the AI native software that is to agents what today's software that is to agents what today's software that is to agents what today's internet is to the web pages of 1995. internet is to the web pages of 1995. internet is to the web pages of 1995. And one way to think about the story is And one way to think about the story is And one way to think about the story is this. We went from the calculator to the this. We went from the calculator to the this. We went from the calculator to the computer to the personal computer to the computer to the personal computer to the computer to the personal computer to the global cloud computer. And now we global cloud computer. And now we global cloud computer. And now we actually have the ability actually have the ability actually have the ability to build the Memex and Jarvis from Iron to build the Memex and Jarvis from Iron to build the Memex and Jarvis from Iron Man and knowledge navigator from that Man and knowledge navigator from that Man and knowledge navigator from that 1987 video for real, completely working. 1987 video for real, completely working. 1987 video for real, completely working. And by building those things, we'll And by building those things, we'll And by building those things, we'll figure out what we want to build next. figure out what we want to build next. figure out what we want to build next. A couple of weeks ago the team at Tavis A couple of weeks ago the team at Tavis A couple of weeks ago the team at Tavis released a reimagined knowledge released a reimagined knowledge released a reimagined knowledge navigator video this time entirely built navigator video this time entirely built navigator video this time entirely built on real and available technology.

on real and available technology. on real and available technology. I'll just play another 20 seconds of I'll just play another 20 seconds of I'll just play another 20 seconds of this, but like the original knowledge this, but like the original knowledge this, but like the original knowledge navigator video, this is worth tracking navigator video, this is worth tracking navigator video, this is worth tracking down and watching in full. >> Good evening, Hassaan. I adore that >> Good evening, Hassaan. I adore that houndstooth jacket you're wearing today. houndstooth jacket you're wearing today. houndstooth jacket you're wearing today. Anything I can help with or would you Anything I can help with or would you Anything I can help with or would you like to review tomorrow's schedule? like to review tomorrow's schedule? like to review tomorrow's schedule? >> Thanks so much, Tom. Yeah, let's review >> Thanks so much, Tom. Yeah, let's review >> Thanks so much, Tom. Yeah, let's review tomorrow's schedule and see how busy it tomorrow's schedule and see how busy it tomorrow's schedule and see how busy it is. is. is. >> Opening your calendar now. Here is the >> Opening your calendar now. Here is the >> Opening your calendar now. Here is the quick version since it is late. Tomorrow quick version since it is late. Tomorrow quick version since it is late. Tomorrow morning is slammed. Investor meeting at morning is slammed. Investor meeting at morning is slammed. Investor meeting at 9:00, then back-to-back one-on-ones and 9:00, then back-to-back one-on-ones and 9:00, then back-to-back one-on-ones and internal meetings until 1:00. internal meetings until 1:00. internal meetings until 1:00. >> So, the full 4-minute video is one take, >> So, the full 4-minute video is one take, >> So, the full 4-minute video is one take, completely real. And when you watch it, completely real. And when you watch it, completely real. And when you watch it, it really does feel both like the it really does feel both like the it really does feel both like the knowledge navigator video from 1987, knowledge navigator video from 1987, knowledge navigator video from 1987, familiar but built on real technology, familiar but built on real technology, familiar but built on real technology, and like something brand new. and like something brand new. and like something brand new. And I'll just close with a massively And I'll just close with a massively And I'll just close with a massively multiplayer game project I've been multiplayer game project I've been multiplayer game project I've been working on with some friends as a canvas working on with some friends as a canvas working on with some friends as a canvas to really think about what AI native to really think about what AI native to really think about what AI native software can be.

software can be. software can be. This game is built from the ground up This game is built from the ground up This game is built from the ground up with LLMs as the core of every with LLMs as the core of every with LLMs as the core of every interaction. interaction. interaction. At every moment in the game, there are At every moment in the game, there are At every moment in the game, there are hundreds of inference calls happening hundreds of inference calls happening hundreds of inference calls happening and we couldn't have built anything like and we couldn't have built anything like and we couldn't have built anything like this even a year ago. this even a year ago. this even a year ago. Oh, sorry. >> Welcome to Gradient Bang, a multiplayer >> Welcome to Gradient Bang, a multiplayer game [music] that showcases real-time game [music] that showcases real-time game [music] that showcases real-time agent orchestration. agent orchestration. agent orchestration. Gradient Bang demonstrates several Gradient Bang demonstrates several Gradient Bang demonstrates several patterns for AI sub agents such as patterns for AI sub agents such as patterns for AI sub agents such as asynchronous non-blocking context asynchronous non-blocking context asynchronous non-blocking context compression. compression. compression. >> Okay, make a note for later. We are >> Okay, make a note for later. We are >> Okay, make a note for later. We are going to eliminate Heliotrope from going to eliminate Heliotrope from going to eliminate Heliotrope from existence. existence. existence. >> Noted. >> Noted. >> Noted. >> Long-running sub agents that share >> Long-running sub agents that share >> Long-running sub agents that share context. [music] context. [music] context. [music] >> Eagle is on five trade loops. Hawk and >> Eagle is on five trade loops. Hawk and >> Eagle is on five trade loops. Hawk and Raptor are on five exploration loops Raptor are on five exploration loops Raptor are on five exploration loops [music] each. Your fleet is busy. [music] each. Your fleet is busy. [music] each. Your fleet is busy. >> Progressive skills loading. >> Progressive skills loading. >> Progressive skills loading. >> How much does your average ship cost? >> How much does your average ship cost? >> How much does your average ship cost? >> Ships range quite [music] a bit, >> Ships range quite [music] a bit, >> Ships range quite [music] a bit, Captain. Captain. Captain. >> Dynamic user interface generation. >> Dynamic user interface generation. >> Dynamic user interface generation. >> Show my task history. >> Show my task history. >> Show my task history. >> Certainly. >> Certainly. >> Certainly. >> Uh hide the map. >> Uh hide the map. >> Uh hide the map. >> Okay.

>> Okay. >> Okay. >> And conversational [music] voice. >> And conversational [music] voice. >> And conversational [music] voice. >> No, I don't want to exchange it. I just >> No, I don't want to exchange it. I just >> No, I don't want to exchange it. I just want to sell it for cold, hard cash, want to sell it for cold, hard cash, want to sell it for cold, hard cash, please. please. please. >> I'm afraid [music] the galaxy doesn't >> I'm afraid [music] the galaxy doesn't >> I'm afraid [music] the galaxy doesn't allow you to be shipless and hitchhike. allow you to be shipless and hitchhike. allow you to be shipless and hitchhike. >> So, I went long after wrap-up, but I >> So, I went long after wrap-up, but I >> So, I went long after wrap-up, but I will say that the first version of this will say that the first version of this will say that the first version of this new draft talk was an hour. So, I have a new draft talk was an hour. So, I have a new draft talk was an hour. So, I have a lot more things I'm super excited to lot more things I'm super excited to lot more things I'm super excited to talk about with all of you. So, if you talk about with all of you. So, if you talk about with all of you. So, if you are interested in this stuff, come find are interested in this stuff, come find are interested in this stuff, come find me. We have a booth on the show floor. me. We have a booth on the show floor. me. We have a booth on the show floor. I'm online everywhere and I'm excited to I'm online everywhere and I'm excited to I'm online everywhere and I'm excited to build agents build agents build agents cuz agents are awesome, but also to cuz agents are awesome, but also to cuz agents are awesome, but also to build the next next thing, too. Thank build the next next thing, too. Thank build the next next thing, too. Thank you. you. you. >> [applause] [music] [music] >> Mhm.