A cognition engine for science with Allen Stewart

So, like stuff like that. You just don't So, like stuff like that. You just don't want to make those kind of mistakes or want to make those kind of mistakes or want to make those kind of mistakes or otherwise you're going to be drinking otherwise you're going to be drinking otherwise you're going to be drinking sulfuric acid, Tommy. sulfuric acid, Tommy. sulfuric acid, Tommy. >> Absolutely. Look Look, on my journey I >> Absolutely. Look Look, on my journey I >> Absolutely. Look Look, on my journey I look I had my own journey to science, look I had my own journey to science, look I had my own journey to science, right? I'm a computer scientist. And I right? I'm a computer scientist. And I right? I'm a computer scientist. And I look when we found out that we were look when we found out that we were look when we found out that we were going to work in AI for science, you going to work in AI for science, you going to work in AI for science, you know, I got to spend some time with know, I got to spend some time with know, I got to spend some time with scientists. And that was very That was scientists. And that was very That was scientists. And that was very That was very eye-opening, right? As you get to very eye-opening, right? As you get to very eye-opening, right? As you get to start spending time with scientists, start spending time with scientists, start spending time with scientists, first of First of all, they said, "Look, first of First of all, they said, "Look, first of First of all, they said, "Look, hey hey hey AI guy. AI guy. AI guy. Science Scientists do science. Do you Science Scientists do science. Do you Science Scientists do science. Do you understand?" And I was like, "I understand?" And I was like, "I understand?" And I was like, "I understand." understand." understand." So, AI is a They're like, "Repeat it So, AI is a They're like, "Repeat it So, AI is a They're like, "Repeat it back to me. What did you hear? AI is a back to me. What did you hear? AI is a back to me. What did you hear? AI is a tool. Scientists do science." They're tool. Scientists do science." They're tool. Scientists do science." They're like, "Okay, we like this guy. like, "Okay, we like this guy. like, "Okay, we like this guy. >> There you go. Steady count. >> There you go. Steady count. >> There you go. Steady count. >> what we're trying to do here, right? So >> what we're trying to do here, right? So >> what we're trying to do here, right? So Hey friends, you probably knew that Text Hey friends, you probably knew that Text Hey friends, you probably knew that Text Control is a powerful library for Control is a powerful library for Control is a powerful library for document editing and PDF generation, but document editing and PDF generation, but document editing and PDF generation, but did you also know that they're a strong did you also know that they're a strong did you also know that they're a strong supporter of the developer community and supporter of the developer community and supporter of the developer community and it's part of their mission to build and it's part of their mission to build and it's part of their mission to build and support a strong community by being support a strong community by being support a strong community by being present, by listening to users, and by present, by listening to users, and by present, by listening to users, and by sharing knowledge at conferences across sharing knowledge at conferences across sharing knowledge at conferences across Europe and the United States. If you're Europe and the United States. If you're Europe and the United States. If you're heading to conference soon, heading to conference soon, heading to conference soon, maybe check if Text Control will be maybe check if Text Control will be maybe check if Text Control will be there. Stop by and say hi. You'll find there. Stop by and say hi. You'll find there. Stop by and say hi. You'll find their full conference calendar at their full conference calendar at their full conference calendar at textcontrol.com.

textcontrol.com. textcontrol.com. That's t e x t control.com. That's t e x t control.com. That's t e x t control.com. Hey friends, I'm Scott Hanselman. This Hey friends, I'm Scott Hanselman. This Hey friends, I'm Scott Hanselman. This is another episode of Hanselminutes. is another episode of Hanselminutes. is another episode of Hanselminutes. Today I'm chatting with Alan Stewart. Today I'm chatting with Alan Stewart. Today I'm chatting with Alan Stewart. He's a partner director of software He's a partner director of software He's a partner director of software engineering innovation at Microsoft and engineering innovation at Microsoft and engineering innovation at Microsoft and a supporter of AI for science. How are a supporter of AI for science. How are a supporter of AI for science. How are you, sir? I'm doing great. How are you you, sir? I'm doing great. How are you you, sir? I'm doing great. How are you doing, Scott? I'm getting there. I'm doing, Scott? I'm getting there. I'm doing, Scott? I'm getting there. I'm ready to learn. I'm ready to learn. You ready to learn. I'm ready to learn. You ready to learn. I'm ready to learn. You know, like that Somebody was telling me know, like that Somebody was telling me know, like that Somebody was telling me that they're worried that my podcast is that they're worried that my podcast is that they're worried that my podcast is going to become like an AI like slop going to become like an AI like slop going to become like an AI like slop kind of talk where all we do is talk kind of talk where all we do is talk kind of talk where all we do is talk about AI. I am interested in solving about AI. I am interested in solving about AI. I am interested in solving real problems for real people. And if it real problems for real people. And if it real problems for real people. And if it happens to be using AI, then that's happens to be using AI, then that's happens to be using AI, then that's cool. And that's that's fine. And cool. And that's that's fine. And cool. And that's that's fine. And everyone's saying agent. Everyone's everyone's saying agent. Everyone's everyone's saying agent. Everyone's saying If you want to be smart in a saying If you want to be smart in a saying If you want to be smart in a meeting, you say agentic. And uh I think meeting, you say agentic. And uh I think meeting, you say agentic. And uh I think we've had a lot of talks about that, but we've had a lot of talks about that, but we've had a lot of talks about that, but you are You believe that it's memory you are You believe that it's memory you are You believe that it's memory that is the trick. That's the special that is the trick. That's the special that is the trick. That's the special sauce. sauce. sauce. I believe memory is that closed loop. It I believe memory is that closed loop. It I believe memory is that closed loop. It closes the loop of, you know, the closes the loop of, you know, the closes the loop of, you know, the cognitive engine in AI cognitive engine in AI cognitive engine in AI doing research, that being stored in a doing research, that being stored in a doing research, that being stored in a memory store, and then that being used memory store, and then that being used memory store, and then that being used again by the AI to either unlock a new again by the AI to either unlock a new again by the AI to either unlock a new path of research or to accelerate the path of research or to accelerate the path of research or to accelerate the existing path of research. Let me tell existing path of research. Let me tell existing path of research. Let me tell you what I mean. So, if somebody puts a you what I mean. So, if somebody puts a you what I mean. So, if somebody puts a science problem there and like I I science problem there and like I I science problem there and like I I recently ran a science job that ran for recently ran a science job that ran for recently ran a science job that ran for like 14 days and spent millions of like 14 days and spent millions of like 14 days and spent millions of tokens, right? And some of the thoughts tokens, right? And some of the thoughts tokens, right? And some of the thoughts in the loop were completed and some of in the loop were completed and some of in the loop were completed and some of them were incomplete thoughts. But what them were incomplete thoughts. But what them were incomplete thoughts. But what I wanted to do is I wanted to prove that I wanted to do is I wanted to prove that I wanted to do is I wanted to prove that any memory, any token expenditure from any memory, any token expenditure from any memory, any token expenditure from an AI perspective, is good research. So, an AI perspective, is good research. So, an AI perspective, is good research. So, I took those memories, packaged them up,

I took those memories, packaged them up, I took those memories, packaged them up, and I was calling them like a science and I was calling them like a science and I was calling them like a science pack of knowledge, took them to a new pack of knowledge, took them to a new pack of knowledge, took them to a new machine, restarted the same science machine, restarted the same science machine, restarted the same science example, example, example, and that example started 150 million and that example started 150 million and that example started 150 million tokens less because 150 million of the tokens less because 150 million of the tokens less because 150 million of the tokens were creating the plan for how to tokens were creating the plan for how to tokens were creating the plan for how to solve the problem, right? So, the next solve the problem, right? So, the next solve the problem, right? So, the next system picked up directly from there and system picked up directly from there and system picked up directly from there and started, but it also picked up the started, but it also picked up the started, but it also picked up the incomplete thoughts and started research incomplete thoughts and started research incomplete thoughts and started research from that perspective, too. So, just from that perspective, too. So, just from that perspective, too. So, just from a token efficiency perspective, from a token efficiency perspective, from a token efficiency perspective, it's 150 million tokens less to start it's 150 million tokens less to start it's 150 million tokens less to start that research uh moving forward just that research uh moving forward just that research uh moving forward just from the memories of the system. So, from the memories of the system. So, from the memories of the system. So, think about that that exhaust being used think about that that exhaust being used think about that that exhaust being used solving the problem, using memories, solving the problem, using memories, solving the problem, using memories, starting the problem again using those starting the problem again using those starting the problem again using those same memories to solve either some of same memories to solve either some of same memories to solve either some of the similar problems or new problems the similar problems or new problems the similar problems or new problems moving forward. So, memory has has been moving forward. So, memory has has been moving forward. So, memory has has been that closed loop for me. I've been doing that closed loop for me. I've been doing that closed loop for me. I've been doing a lot of work there. So, is this the AI a lot of work there. So, is this the AI a lot of work there. So, is this the AI equivalent of no bad ideas in a equivalent of no bad ideas in a equivalent of no bad ideas in a brainstorm, right? Like there is no brainstorm, right? Like there is no brainstorm, right? Like there is no wasted tokens from your perspective. wasted tokens from your perspective. wasted tokens from your perspective. You're going to pick up all the exhaust You're going to pick up all the exhaust You're going to pick up all the exhaust and feed it back into the system. Yeah, and feed it back into the system. Yeah, and feed it back into the system. Yeah, cuz think about like a science problem cuz think about like a science problem cuz think about like a science problem where maybe I'm trying to repurpose or where maybe I'm trying to repurpose or where maybe I'm trying to repurpose or reanalyze a drug a drug for Alzheimer's.

reanalyze a drug a drug for Alzheimer's. reanalyze a drug a drug for Alzheimer's. So, every token expended is some sort of So, every token expended is some sort of So, every token expended is some sort of research path that could either unlock a research path that could either unlock a research path that could either unlock a new path or, like I said, speed up an new path or, like I said, speed up an new path or, like I said, speed up an existing existing science problem, existing existing science problem, existing existing science problem, right? These problems run for long right? These problems run for long right? These problems run for long long-running problems, right? Trying to long-running problems, right? Trying to long-running problems, right? Trying to repurpose and use, repurpose and use, repurpose and use, you know, millions of catalogs of you know, millions of catalogs of you know, millions of catalogs of science data, either chemistry or science data, either chemistry or science data, either chemistry or biology moving forward, so. Mhm. biology moving forward, so. Mhm. biology moving forward, so. Mhm. Okay, so you're you're deeper in this Okay, so you're you're deeper in this Okay, so you're you're deeper in this than I am. I'm focused primarily on than I am. I'm focused primarily on than I am. I'm focused primarily on solving problems for programmers. solving problems for programmers. solving problems for programmers. Uh you know, you've you've done You're Uh you know, you've you've done You're Uh you know, you've you've done You're doing like postgraduate work and you're doing like postgraduate work and you're doing like postgraduate work and you're thinking about this deeper than I am. thinking about this deeper than I am. thinking about this deeper than I am. Put this into the context for the Put this into the context for the Put this into the context for the regular Joes and Janes who are regular Joes and Janes who are regular Joes and Janes who are listening, for whom memory is a bunch of listening, for whom memory is a bunch of listening, for whom memory is a bunch of markdown files. markdown files. markdown files. Right? That's like the you know, the Right? That's like the you know, the Right? That's like the you know, the poor man's memory. It's just kind of poor man's memory. It's just kind of poor man's memory. It's just kind of like what I'm keeping track of. What is like what I'm keeping track of. What is like what I'm keeping track of. What is a real legitimate enterprise-level a real legitimate enterprise-level a real legitimate enterprise-level memory system and how is that different memory system and how is that different memory system and how is that different than a bunch of MD files in storage? than a bunch of MD files in storage? than a bunch of MD files in storage? Yeah, so, you know, your markdown files Yeah, so, you know, your markdown files Yeah, so, you know, your markdown files are They're a record of context that are They're a record of context that are They're a record of context that you've pushed into the system to either you've pushed into the system to either you've pushed into the system to either build something. And then out of it build something. And then out of it build something. And then out of it you're measuring some sort of outcome you're measuring some sort of outcome you're measuring some sort of outcome out of it. So, you think about it just out of it. So, you think about it just out of it. So, you think about it just from a scientific perspective, we're from a scientific perspective, we're from a scientific perspective, we're pushing a science challenge problem into pushing a science challenge problem into pushing a science challenge problem into the system. Some, you know, look, the the system. Some, you know, look, the the system. Some, you know, look, the type of customers we work with, type of customers we work with, type of customers we work with, scientists, they're trying to either you scientists, they're trying to either you scientists, they're trying to either you know, discover a new chemical, discover know, discover a new chemical, discover know, discover a new chemical, discover a new biological agent. So, they push a a new biological agent. So, they push a a new biological agent. So, they push a challenge problem into the system. And challenge problem into the system. And challenge problem into the system. And as the system is decomposing the problem as the system is decomposing the problem as the system is decomposing the problem and researching or deciding how it's and researching or deciding how it's and researching or deciding how it's going to research, going to research, going to research, all those activities from the tasks, the all those activities from the tasks, the all those activities from the tasks, the agents, we call them subtasks and and agents, we call them subtasks and and agents, we call them subtasks and and and task and thought channels. All that

and task and thought channels. All that and task and thought channels. All that thought channel work that the agents and thought channel work that the agents and thought channel work that the agents and tasks are doing tasks are doing tasks are doing to to build to the problem, all that's to to build to the problem, all that's to to build to the problem, all that's memory. It's generating exhaust from the memory. It's generating exhaust from the memory. It's generating exhaust from the process. Hey, I'm I I want you to go process. Hey, I'm I I want you to go process. Hey, I'm I I want you to go research 10 different drugs in this research 10 different drugs in this research 10 different drugs in this thought channel. So, that channel is thought channel. So, that channel is thought channel. So, that channel is actively researching and what they're actively researching and what they're actively researching and what they're finding out is all the memories that we finding out is all the memories that we finding out is all the memories that we associate into the system or we store associate into the system or we store associate into the system or we store into the system and then we use as a into the system and then we use as a into the system and then we use as a part of the cognitive engine again as um part of the cognitive engine again as um part of the cognitive engine again as um to either re-prosecute some additional to either re-prosecute some additional to either re-prosecute some additional research or some of the same research. research or some of the same research. research or some of the same research. But there's a confidence score between But there's a confidence score between But there's a confidence score between the cognitive engine and the memory the cognitive engine and the memory the cognitive engine and the memory store that says, "Is this a relevant store that says, "Is this a relevant store that says, "Is this a relevant memory?" Right? I'm researching drugs. memory?" Right? I'm researching drugs. memory?" Right? I'm researching drugs. Is this a relevant memory or not? Right? Is this a relevant memory or not? Right? Is this a relevant memory or not? Right? If it score if the memory has a If it score if the memory has a If it score if the memory has a confidence score of three or higher, confidence score of three or higher, confidence score of three or higher, it's stored in the actual store itself. it's stored in the actual store itself. it's stored in the actual store itself. If it doesn't have a confidence score of If it doesn't have a confidence score of If it doesn't have a confidence score of three or higher, it's stored in the three or higher, it's stored in the three or higher, it's stored in the actual database for regular use down the actual database for regular use down the actual database for regular use down the line, but it's just not relevant for line, but it's just not relevant for line, but it's just not relevant for this particular problem. So, there's this particular problem. So, there's this particular problem. So, there's there's signals that are happening there's signals that are happening there's signals that are happening between the cognitive engine and the between the cognitive engine and the between the cognitive engine and the memory store that signal, "Hey, this I memory store that signal, "Hey, this I memory store that signal, "Hey, this I want you to query for a memory. This want you to query for a memory. This want you to query for a memory. This memory is relevant. Now I'll ground my memory is relevant. Now I'll ground my memory is relevant. Now I'll ground my research in this data or it's not research in this data or it's not research in this data or it's not relevant for this problem, but still relevant for this problem, but still relevant for this problem, but still stored in the the store. It may be stored in the the store. It may be stored in the the store. It may be relevant for another problem down down relevant for another problem down down relevant for another problem down down the line." Okay, I want to make sure the line." Okay, I want to make sure the line." Okay, I want to make sure that we're grounding this in as as that we're grounding this in as as that we're grounding this in as as simple analogies as I understand and simple analogies as I understand and simple analogies as I understand and that I'm pushing back as appropriate and that I'm pushing back as appropriate and that I'm pushing back as appropriate and as I am known for so that everyone can as I am known for so that everyone can as I am known for so that everyone can come along for as you are come along for as you are come along for as you are >> As you're known for, yes. Well, what I'm >> As you're known for, yes. Well, what I'm >> As you're known for, yes. Well, what I'm just saying like as just saying like as just saying like as cuz I want to make sure that everyone cuz I want to make sure that everyone cuz I want to make sure that everyone gets to come along for the ride because gets to come along for the ride because gets to come along for the ride because some people might be thinking word

some people might be thinking word some people might be thinking word salad. Some might people might be salad. Some might people might be salad. Some might people might be saying, "I'm kind of where he's at, but saying, "I'm kind of where he's at, but saying, "I'm kind of where he's at, but I'm not 100% there." I'm not 100% there." I'm not 100% there." AI is the overarching thing on top of AI is the overarching thing on top of AI is the overarching thing on top of ML, deep learning, neural networks, and ML, deep learning, neural networks, and ML, deep learning, neural networks, and what happens to be generative what happens to be generative what happens to be generative pre-trained transformers. Are you doing pre-trained transformers. Are you doing pre-trained transformers. Are you doing primarily work in GPTs or are you primarily work in GPTs or are you primarily work in GPTs or are you combining all the different things under combining all the different things under combining all the different things under the AI umbrella, only some of which are the AI umbrella, only some of which are the AI umbrella, only some of which are tokens burned on next token prediction? tokens burned on next token prediction? tokens burned on next token prediction? We're combining all of them together. We're combining all of them together. We're combining all of them together. Anything from building You know, look, Anything from building You know, look, Anything from building You know, look, for science we use large language for science we use large language for science we use large language models, but the way we use them is not models, but the way we use them is not models, but the way we use them is not the way people think we use them for the way people think we use them for the way people think we use them for science. We're not using them science. We're not using them science. We're not using them >> They're like, "Here, that What drug's >> They're like, "Here, that What drug's >> They're like, "Here, that What drug's best?" Right? And then you put it into best?" Right? And then you put it into best?" Right? And then you put it into ChatGPT. ChatGPT. ChatGPT. >> That's what people think. Like, "Hey, >> That's what people think. Like, "Hey, >> That's what people think. Like, "Hey, I'm just putting it into ChatGPT. So, I'm just putting it into ChatGPT. So, I'm just putting it into ChatGPT. So, build me a Build me a Build me a drug to build me a Build me a Build me a drug to build me a Build me a Build me a drug to to solve this." We're not using it that to solve this." We're not using it that to solve this." We're not using it that way, right? For science, we're using way, right? For science, we're using way, right? For science, we're using specifically trained and fine-tuned specifically trained and fine-tuned specifically trained and fine-tuned scientific models, Right. Okay. scientific models, Right. Okay. scientific models, Right. Okay. that are trained on scientific process. that are trained on scientific process. that are trained on scientific process. We're using LLMs to in either We're using LLMs to in either We're using LLMs to in either orchestrate agents or enhance or orchestrate agents or enhance or orchestrate agents or enhance or summarize some of the context or we have summarize some of the context or we have summarize some of the context or we have a cognitive engine that we call Cleo.

a cognitive engine that we call Cleo. a cognitive engine that we call Cleo. Okay. And Cleo uses system one and Okay. And Cleo uses system one and Okay. And Cleo uses system one and system two thinking, right? Slow system two thinking, right? Slow system two thinking, right? Slow thinking and fast thinking in the thinking and fast thinking in the thinking and fast thinking in the system, right? So, for slow thinking, we system, right? So, for slow thinking, we system, right? So, for slow thinking, we input that something that has to churn. input that something that has to churn. input that something that has to churn. This is a huge challenge problem. You're This is a huge challenge problem. You're This is a huge challenge problem. You're going to burn a lot of tokens to try to going to burn a lot of tokens to try to going to burn a lot of tokens to try to solve it. So, we push that into what we solve it. So, we push that into what we solve it. So, we push that into what we call a slow thinking channel, right? And call a slow thinking channel, right? And call a slow thinking channel, right? And it a model we have a specific model it a model we have a specific model it a model we have a specific model there that works on that problem. Slow there that works on that problem. Slow there that works on that problem. Slow thinking, lots of research going to thinking, lots of research going to thinking, lots of research going to You're going to spend a lot of tokens to You're going to spend a lot of tokens to You're going to spend a lot of tokens to get to to an answer. And then we have a get to to an answer. And then we have a get to to an answer. And then we have a fast path, right? Where we say, "Okay, fast path, right? Where we say, "Okay, fast path, right? Where we say, "Okay, look, this This is you're summarizing look, this This is you're summarizing look, this This is you're summarizing some data. You maybe you're taking some data. You maybe you're taking some data. You maybe you're taking something out of the fast path and something out of the fast path and something out of the fast path and you're summarizing in a table or you're summarizing in a table or you're summarizing in a table or summarizing in a markdown file to be summarizing in a markdown file to be summarizing in a markdown file to be used later as a part of research as you used later as a part of research as you used later as a part of research as you get further down the chain. So, we have get further down the chain. So, we have get further down the chain. So, we have two kind of two kind of two kind of thinking mechanisms there. And each one thinking mechanisms there. And each one thinking mechanisms there. And each one of those things are exhibiting exhaust. of those things are exhibiting exhaust. of those things are exhibiting exhaust. Those exhausts are memories that are Those exhausts are memories that are Those exhausts are memories that are coming out of system, whether it's a coming out of system, whether it's a coming out of system, whether it's a markdown file or show me the top 10 markdown file or show me the top 10 markdown file or show me the top 10 drugs that meet this criteria in a file. drugs that meet this criteria in a file. drugs that meet this criteria in a file. We're pushing that down further into the We're pushing that down further into the We're pushing that down further into the system. There's a one thing our system. There's a one thing our system. There's a one thing our cognitive engine does, it's a very cognitive engine does, it's a very cognitive engine does, it's a very resilient brain. It works across resilient brain. It works across resilient brain. It works across transient errors, problems. We use the transient errors, problems. We use the transient errors, problems. We use the coding agent, we use the deep research coding agent, we use the deep research coding agent, we use the deep research agent. And let's just say you're not agent. And let's just say you're not agent. And let's just say you're not able to get to the coding agent. It will able to get to the coding agent. It will able to get to the coding agent. It will work around that. Okay, I'm not able to work around that. Okay, I'm not able to work around that. Okay, I'm not able to get to the coding agent. Let me use LLM get to the coding agent. Let me use LLM get to the coding agent. Let me use LLM to do some research. Let me use another to do some research. Let me use another to do some research. Let me use another tool or mechanism that maybe the system tool or mechanism that maybe the system tool or mechanism that maybe the system designer put in to be able to do that designer put in to be able to do that designer put in to be able to do that work. So, we're using every mechanism to work. So, we're using every mechanism to work. So, we're using every mechanism to do science. So, we need all of them to do science. So, we need all of them to do science. So, we need all of them to do science. It's pretty complex.

do science. It's pretty complex. do science. It's pretty complex. And and for context for folks that may And and for context for folks that may And and for context for folks that may not be familiar with those those not be familiar with those those not be familiar with those those concepts, system one and system two concepts, system one and system two concepts, system one and system two thinking came from Daniel Kahneman's thinking came from Daniel Kahneman's thinking came from Daniel Kahneman's book Thinking, Fast and Slow. So, the book Thinking, Fast and Slow. So, the book Thinking, Fast and Slow. So, the system one is kind of automatic, system one is kind of automatic, system one is kind of automatic, unconscious. It's your gut. It's your unconscious. It's your gut. It's your unconscious. It's your gut. It's your intuition. It's how you get from point A intuition. It's how you get from point A intuition. It's how you get from point A to point D Yes. without actually showing to point D Yes. without actually showing to point D Yes. without actually showing your work. Now, when we they when when your work. Now, when we they when when your work. Now, when we they when when we were kids and the teacher was we were kids and the teacher was we were kids and the teacher was constantly like, "Show your work. Show constantly like, "Show your work. Show constantly like, "Show your work. Show your work." They were forcing you to go your work." They were forcing you to go your work." They were forcing you to go from system one thinking to system two from system one thinking to system two from system one thinking to system two thinking, which is more effortful, more thinking, which is more effortful, more thinking, which is more effortful, more complex. Now, people might push back on complex. Now, people might push back on complex. Now, people might push back on saying that next token prediction and saying that next token prediction and saying that next token prediction and models think, but I'm noticing that you models think, but I'm noticing that you models think, but I'm noticing that you and others are all using effectively and others are all using effectively and others are all using effectively anthropomorphized descriptions. You say anthropomorphized descriptions. You say anthropomorphized descriptions. You say memory, yes, we say memory when we talk memory, yes, we say memory when we talk memory, yes, we say memory when we talk about computer memory, but you're also about computer memory, but you're also about computer memory, but you're also saying like cognitive engine. And some saying like cognitive engine. And some saying like cognitive engine. And some people might wince at that going like people might wince at that going like people might wince at that going like it's not a brain, but it kind of is it's not a brain, but it kind of is it's not a brain, but it kind of is trying to be modeled like a brain. We're trying to be modeled like a brain. We're trying to be modeled like a brain. We're trying to model it like a brain. We're trying to model it like a brain. We're trying to model it like a brain. We're trying to model it like a brain where we trying to model it like a brain where we trying to model it like a brain where we have, you know, the cognitive engine have, you know, the cognitive engine have, you know, the cognitive engine that's going to orchestrate the agents, that's going to orchestrate the agents, that's going to orchestrate the agents, the tools, and and taking the challenge the tools, and and taking the challenge the tools, and and taking the challenge problem and it orchestrate, you know, problem and it orchestrate, you know, problem and it orchestrate, you know, the the ensemble of models that we have the the ensemble of models that we have the the ensemble of models that we have to get to an answer. To orchestrate the to get to an answer. To orchestrate the to get to an answer. To orchestrate the memories as well, right? The cognitive memories as well, right? The cognitive memories as well, right? The cognitive engine says I'm going to start on this engine says I'm going to start on this engine says I'm going to start on this problem. Hey, memory store, I'm building problem. Hey, memory store, I'm building problem. Hey, memory store, I'm building this type of drug. Do you have anything this type of drug. Do you have anything this type of drug. Do you have anything there? Yeah, I have relevant memories.

there? Yeah, I have relevant memories. there? Yeah, I have relevant memories. Let me ground myself in that in that Let me ground myself in that in that Let me ground myself in that in that data. Let me ground myself in the tools. data. Let me ground myself in the tools. data. Let me ground myself in the tools. And then, if I need to create a tool And then, if I need to create a tool And then, if I need to create a tool because one's not available, because one's not available, because one's not available, we have the GitHub Copilot CLI SDK there we have the GitHub Copilot CLI SDK there we have the GitHub Copilot CLI SDK there for the agent to build a agent to build for the agent to build a agent to build for the agent to build a agent to build the tool that needs to that needs to be the tool that needs to that needs to be the tool that needs to that needs to be built for the research moving forward. built for the research moving forward. built for the research moving forward. So, it is kind of orchestrating and So, it is kind of orchestrating and So, it is kind of orchestrating and operating like a brain. I know people operating like a brain. I know people operating like a brain. I know people like, you know, people kind of cringe like, you know, people kind of cringe like, you know, people kind of cringe when you hear some AI people talking when you hear some AI people talking when you hear some AI people talking about we're trying to build a brain here about we're trying to build a brain here about we're trying to build a brain here and they're like, all right. We're far and they're like, all right. We're far and they're like, all right. We're far away from that. But, we are trying to away from that. But, we are trying to away from that. But, we are trying to model out cognitive engine, ability to model out cognitive engine, ability to model out cognitive engine, ability to store short-term and long-term memory, store short-term and long-term memory, store short-term and long-term memory, procedural memories, or episodic memory procedural memories, or episodic memory procedural memories, or episodic memory in a store. And then, some level of in a store. And then, some level of in a store. And then, some level of engine that says, "Hey, let me query and engine that says, "Hey, let me query and engine that says, "Hey, let me query and is this relevant to relevant to what I'm is this relevant to relevant to what I'm is this relevant to relevant to what I'm trying to research?" And use that, trying to research?" And use that, trying to research?" And use that, right? I'm going to write a paper that, right? I'm going to write a paper that, right? I'm going to write a paper that, you know, called like the enterprise you know, called like the enterprise you know, called like the enterprise science brain, right? So, every time science brain, right? So, every time science brain, right? So, every time you use the system, the system that we you use the system, the system that we you use the system, the system that we build is system that we call discovery build is system that we call discovery build is system that we call discovery to do science scientific research, every to do science scientific research, every to do science scientific research, every time you research, the system is time you research, the system is time you research, the system is learning and growing and becoming learning and growing and becoming learning and growing and becoming smarter.

smarter. smarter. Okay. So, my son was in art school for a Okay. So, my son was in art school for a Okay. So, my son was in art school for a while and he would always start drawings while and he would always start drawings while and he would always start drawings and then he'd like stop and scribble and then he'd like stop and scribble and then he'd like stop and scribble them and throw them away. You're arguing them and throw them away. You're arguing them and throw them away. You're arguing never do that. Never do that. There's no never do that. Never do that. There's no never do that. Never do that. There's no no such thing as a wasted token in no such thing as a wasted token in no such thing as a wasted token in science research. science research. science research. There's some way we can use that and There's some way we can use that and There's some way we can use that and leverage that. You may think that this leverage that. You may think that this leverage that. You may think that this is not relevant. It's not relevant for is not relevant. It's not relevant for is not relevant. It's not relevant for your existing challenge problem, but in your existing challenge problem, but in your existing challenge problem, but in the field of science, it's probably the field of science, it's probably the field of science, it's probably relevant to some additional research relevant to some additional research relevant to some additional research that's happening and might that one that's happening and might that one that's happening and might that one partial memory might be the thing that partial memory might be the thing that partial memory might be the thing that unlocks a breakthrough. Okay. So, in a unlocks a breakthrough. Okay. So, in a unlocks a breakthrough. Okay. So, in a world where people are concerned about world where people are concerned about world where people are concerned about the the weight of AI and on on on on the the the weight of AI and on on on on the the the weight of AI and on on on on the ecology, on, you know, eco-friendly, AI ecology, on, you know, eco-friendly, AI ecology, on, you know, eco-friendly, AI is not something that you would hear is not something that you would hear is not something that you would hear said. One could argue that the cheapest said. One could argue that the cheapest said. One could argue that the cheapest thing in the cloud right now is storage. thing in the cloud right now is storage. thing in the cloud right now is storage. And the most expensive thing in the And the most expensive thing in the And the most expensive thing in the cloud right now is making tokens. So, by cloud right now is making tokens. So, by cloud right now is making tokens. So, by saving them all, you're potentially saving them all, you're potentially saving them all, you're potentially allowing agents to not have to restart allowing agents to not have to restart allowing agents to not have to restart from scratch, which is just wasteful. from scratch, which is just wasteful. from scratch, which is just wasteful. Absolutely. And you know, doing my Absolutely. And you know, doing my Absolutely. And you know, doing my research, even even doing the research research, even even doing the research research, even even doing the research when I run this this large science job when I run this this large science job when I run this this large science job and I I I took all the memories from the and I I I took all the memories from the and I I I took all the memories from the completed channels. Uh-huh. And then I completed channels. Uh-huh. And then I completed channels. Uh-huh. And then I look back and it said, "We have 33 look back and it said, "We have 33 look back and it said, "We have 33 incomplete channels with 30% memories."

incomplete channels with 30% memories." incomplete channels with 30% memories." And I said, And I said, And I said, "Hold on, I want those, too. Let me "Hold on, I want those, too. Let me "Hold on, I want those, too. Let me package those as well." I packaged those package those as well." I packaged those package those as well." I packaged those and you take them to another system, you and you take them to another system, you and you take them to another system, you start to run and you see the system take start to run and you see the system take start to run and you see the system take even the partial memories and say, "Oh, even the partial memories and say, "Oh, even the partial memories and say, "Oh, these are relevant." And you actually these are relevant." And you actually these are relevant." And you actually see the cognitive engine and the memory see the cognitive engine and the memory see the cognitive engine and the memory store agreeing that even the partial store agreeing that even the partial store agreeing that even the partial memories are relevant to the research memories are relevant to the research memories are relevant to the research and using them moving forward. So, like and using them moving forward. So, like and using them moving forward. So, like I said, I've literally proved there's no I said, I've literally proved there's no I said, I've literally proved there's no such thing as a wasted token in science such thing as a wasted token in science such thing as a wasted token in science research, right? research, right? research, right? >> Okay. So, is this doing and I know I'm >> Okay. So, is this doing and I know I'm >> Okay. So, is this doing and I know I'm I'm being I'm being I'm being indirectly diminutive. It's thing It's indirectly diminutive. It's thing It's indirectly diminutive. It's thing It's kind of doing rag over its own kind of doing rag over its own kind of doing rag over its own partial work and deciding what to show partial work and deciding what to show partial work and deciding what to show its attention to? Because arguably its attention to? Because arguably its attention to? Because arguably though, you still have you don't have an though, you still have you don't have an though, you still have you don't have an unlimited context window. You have a unlimited context window. You have a unlimited context window. You have a large context window, but you still have large context window, but you still have large context window, but you still have context is still a thing. Yeah. context is still a thing. Yeah. context is still a thing. Yeah. Okay. Yeah, if you look at the the Okay. Yeah, if you look at the the Okay. Yeah, if you look at the the cognitive engine in the background is cognitive engine in the background is cognitive engine in the background is breaking down is breaking down context. breaking down is breaking down context. breaking down is breaking down context. It's separating files. It's splitting It's separating files. It's splitting It's separating files. It's splitting things out, putting them into markdown things out, putting them into markdown things out, putting them into markdown files and keeping you away from that files and keeping you away from that files and keeping you away from that that that context limitation explosion, that that context limitation explosion, that that context limitation explosion, right? We're we're kind of moved away.

right? We're we're kind of moved away. right? We're we're kind of moved away. Yeah, eventually things do decompose Yeah, eventually things do decompose Yeah, eventually things do decompose down into a problem, but how we're doing down into a problem, but how we're doing down into a problem, but how we're doing them within the cognitive engine, how them within the cognitive engine, how them within the cognitive engine, how we're breaking them apart, splitting we're breaking them apart, splitting we're breaking them apart, splitting them, separating them, even images, them, separating them, even images, them, separating them, even images, right? You know, if you do a lot of right? You know, if you do a lot of right? You know, if you do a lot of research around images, even images have research around images, even images have research around images, even images have a limit around the the the the scope of a limit around the the the the scope of a limit around the the the the scope of number of images. So, in the cognitive number of images. So, in the cognitive number of images. So, in the cognitive engine is handling those limitations and engine is handling those limitations and engine is handling those limitations and working through them, working around working through them, working around working through them, working around them. Nice. I like that. And then, the them. Nice. I like that. And then, the them. Nice. I like that. And then, the partial memories have value because they partial memories have value because they partial memories have value because they are preserving explored territory. It's are preserving explored territory. It's are preserving explored territory. It's like the fog of war when you play like, like the fog of war when you play like, like the fog of war when you play like, you know, Age of Empires. Like, "Okay, you know, Age of Empires. Like, "Okay, you know, Age of Empires. Like, "Okay, I've been there before. I do not need to I've been there before. I do not need to I've been there before. I do not need to return." Yeah. You're absolutely right. return." Yeah. You're absolutely right. return." Yeah. You're absolutely right. Even do Even when I was doing memory Even do Even when I was doing memory Even do Even when I was doing memory research, I kind of thought about it a research, I kind of thought about it a research, I kind of thought about it a little bit like I'm like, little bit like I'm like, little bit like I'm like, "Is this really advanced?" Because, you "Is this really advanced?" Because, you "Is this really advanced?" Because, you know, really what we're doing is we're know, really what we're doing is we're know, really what we're doing is we're storing stuff like you know, I'm looking storing stuff like you know, I'm looking storing stuff like you know, I'm looking back at back at back at the the the development I used to do in the the the development I used to do in the the the development I used to do in financial service. We're storing stuff financial service. We're storing stuff financial service. We're storing stuff in a database and recalling it. Okay. in a database and recalling it. Okay. in a database and recalling it. Okay. Just Just basically what we're doing. Just Just basically what we're doing. Just Just basically what we're doing. There is some parts of, "Hey, the There is some parts of, "Hey, the There is some parts of, "Hey, the cognitive engine, the dynamic nature of cognitive engine, the dynamic nature of cognitive engine, the dynamic nature of the storing thought channels in a memory the storing thought channels in a memory the storing thought channels in a memory store. What do you use for a memory store. What do you use for a memory store. What do you use for a memory store?" That that gets to be a little store?" That that gets to be a little store?" That that gets to be a little bit AI-y, right? Procedural versus bit AI-y, right? Procedural versus bit AI-y, right? Procedural versus episodic stuff like we ran our our episodic stuff like we ran our our episodic stuff like we ran our our cognitive engine, we ran against science cognitive engine, we ran against science cognitive engine, we ran against science problems, but we also partnered up with problems, but we also partnered up with problems, but we also partnered up with the M365 research team and we ran it the M365 research team and we ran it the M365 research team and we ran it against office tasks, right? There's no against office tasks, right? There's no against office tasks, right? There's no way to prove out your thinking brain way to prove out your thinking brain way to prove out your thinking brain than my by saying, "Hey, you're a than my by saying, "Hey, you're a than my by saying, "Hey, you're a science brain. Now, I'm going to give science brain. Now, I'm going to give science brain. Now, I'm going to give you something totally not relevant to you something totally not relevant to you something totally not relevant to science, science, science, just really more procedural, right? I just really more procedural, right? I just really more procedural, right? I want you to send an email. In that email want you to send an email. In that email want you to send an email. In that email is going to be a number 518 is in there.

is going to be a number 518 is in there. is going to be a number 518 is in there. Read it via OCR. If you find that via Read it via OCR. If you find that via Read it via OCR. If you find that via OCR, generate a new document and send an OCR, generate a new document and send an OCR, generate a new document and send an email." The multitask task kind of email." The multitask task kind of email." The multitask task kind of stuff. But, even there, we actually saw stuff. But, even there, we actually saw stuff. But, even there, we actually saw that our cognitive engine being very that our cognitive engine being very that our cognitive engine being very relevant for even those kind of jobs as relevant for even those kind of jobs as relevant for even those kind of jobs as well, right? So, without any what we well, right? So, without any what we well, right? So, without any what we call warm start, meaning giving it data, call warm start, meaning giving it data, call warm start, meaning giving it data, procedural information, "Here's how you procedural information, "Here's how you procedural information, "Here's how you send an email." Without giving it any send an email." Without giving it any send an email." Without giving it any information, it was very efficient in in information, it was very efficient in in information, it was very efficient in in using working on science task and using working on science task and using working on science task and working on office task as well. Okay, so working on office task as well. Okay, so working on office task as well. Okay, so give me something concrete. I I you I I give me something concrete. I I you I I give me something concrete. I I you I I went through three or four different went through three or four different went through three or four different scientific papers and PowerPoints and scientific papers and PowerPoints and scientific papers and PowerPoints and Word documents and I don't know what's Word documents and I don't know what's Word documents and I don't know what's secret and what's not secret. Okay. I do secret and what's not secret. Okay. I do secret and what's not secret. Okay. I do know that you did do a 12-day autonomous know that you did do a 12-day autonomous know that you did do a 12-day autonomous investigation just a couple of weeks ago investigation just a couple of weeks ago investigation just a couple of weeks ago and the first run was, you know, and the first run was, you know, and the first run was, you know, hundreds of millions of tokens across hundreds of millions of tokens across hundreds of millions of tokens across almost 200 thought channels. And then almost 200 thought channels. And then almost 200 thought channels. And then you did a second run. How were those you did a second run. How were those you did a second run. How were those runs different? runs different? runs different? Great great great deal. So, look, you Great great great deal. So, look, you Great great great deal. So, look, you know, science problems, they run over know, science problems, they run over know, science problems, they run over multiple days, right? And they're multiple days, right? And they're multiple days, right? And they're they're When people say AI is they're When people say AI is they're When people say AI is non-deterministic, a science challenge non-deterministic, a science challenge non-deterministic, a science challenge problem is non-deterministic on when problem is non-deterministic on when problem is non-deterministic on when it's going to start and how it's going it's going to start and how it's going it's going to start and how it's going to finish, right? So, we ran this long to finish, right? So, we ran this long to finish, right? So, we ran this long job over 14 days. It was a sample that I job over 14 days. It was a sample that I job over 14 days. It was a sample that I built to really kind of exercise all of built to really kind of exercise all of built to really kind of exercise all of the AI capability in our engine the AI capability in our engine the AI capability in our engine metacognition, our engine deep research, metacognition, our engine deep research, metacognition, our engine deep research, and our engine the dynamic nation notion and our engine the dynamic nation notion and our engine the dynamic nation notion of thought channels. I spin everything of thought channels. I spin everything of thought channels. I spin everything into a task and a thought channel. And into a task and a thought channel. And into a task and a thought channel. And then setting up the memory store to be then setting up the memory store to be then setting up the memory store to be dynamic as well to store all those dynamic as well to store all those dynamic as well to store all those memories in the thought channel. So, if

memories in the thought channel. So, if memories in the thought channel. So, if I spin off a third channel a thought I spin off a third channel a thought I spin off a third channel a thought channel on inflammation, channel on inflammation, channel on inflammation, everything in that thought channel now everything in that thought channel now everything in that thought channel now is stored in the database under the is stored in the database under the is stored in the database under the thought channel inflammation. And any thought channel inflammation. And any thought channel inflammation. And any task that builds out of that is stored task that builds out of that is stored task that builds out of that is stored there as well. That channel activity is there as well. That channel activity is there as well. That channel activity is what I was able to package up and then what I was able to package up and then what I was able to package up and then take to a new system take to a new system take to a new system and rerun the same sample. and rerun the same sample. and rerun the same sample. And one of the observations was 150 And one of the observations was 150 And one of the observations was 150 million tokens were expended building million tokens were expended building million tokens were expended building the plan for doing the research. the plan for doing the research. the plan for doing the research. I didn't know that from the first job, I didn't know that from the first job, I didn't know that from the first job, right? It's so unique. It's running. right? It's so unique. It's running. right? It's so unique. It's running. AI's cranking through it. Here's the AI's cranking through it. Here's the AI's cranking through it. Here's the activity I have to be able to solve this activity I have to be able to solve this activity I have to be able to solve this problem, but it expended that much in problem, but it expended that much in problem, but it expended that much in just setting up the plan. So, when I just setting up the plan. So, when I just setting up the plan. So, when I moved those memories to a new system, moved those memories to a new system, moved those memories to a new system, the plan of research was laid out or the plan of research was laid out or the plan of research was laid out or automatically from the previous system automatically from the previous system automatically from the previous system in in the memory store. So, the first in in the memory store. So, the first in in the memory store. So, the first query from the cognitive engine was to query from the cognitive engine was to query from the cognitive engine was to the memory store, "Do you have anything the memory store, "Do you have anything the memory store, "Do you have anything relevant to this problem?" relevant to this problem?" relevant to this problem?" And the memory store said, "Oh, boy, do And the memory store said, "Oh, boy, do And the memory store said, "Oh, boy, do I have a lot relevant to this problem. I I have a lot relevant to this problem. I I have a lot relevant to this problem. I have about 1,200 memories that I'm going have about 1,200 memories that I'm going have about 1,200 memories that I'm going to give you this relevant to this to give you this relevant to this to give you this relevant to this problem. Thought channels, full complete problem. Thought channels, full complete problem. Thought channels, full complete memories, and impartial memories." So, memories, and impartial memories." So, memories, and impartial memories." So, it actually started the research 150 it actually started the research 150 it actually started the research 150 million tokens less.

million tokens less. million tokens less. So, it started not from square zero. It So, it started not from square zero. It So, it started not from square zero. It started like at second base. At second started like at second base. At second started like at second base. At second base. At second I know like some people base. At second I know like some people base. At second I know like some people would say 150 million tokens to second would say 150 million tokens to second would say 150 million tokens to second base for science problems, that's second base for science problems, that's second base for science problems, that's second >> It was if it's a big problem to start >> It was if it's a big problem to start >> It was if it's a big problem to start with. Yes. At second base. Yep. Okay. with. Yes. At second base. Yep. Okay. with. Yes. At second base. Yep. Okay. >> And actually, the research on the second >> And actually, the research on the second >> And actually, the research on the second job, the second job got a lot much job, the second job got a lot much job, the second job got a lot much further along in the process of actually further along in the process of actually further along in the process of actually being able to solve it. Many of the being able to solve it. Many of the being able to solve it. Many of the channels actually converge. Converge channels actually converge. Converge channels actually converge. Converge means they finished their work. They means they finished their work. They means they finished their work. They were able to finish their work and come were able to finish their work and come were able to finish their work and come up with output. Maybe the output was up with output. Maybe the output was up with output. Maybe the output was come up with 10 candidate drugs for this come up with 10 candidate drugs for this come up with 10 candidate drugs for this problem. So, it was able to finish and problem. So, it was able to finish and problem. So, it was able to finish and have a finished output, a markdown file, have a finished output, a markdown file, have a finished output, a markdown file, graph, something that fit into the graph, something that fit into the graph, something that fit into the broader research. Okay. broader research. Okay. broader research. Okay. Now, keeping in mind that, you know, I Now, keeping in mind that, you know, I Now, keeping in mind that, you know, I can only do so much research and I don't can only do so much research and I don't can only do so much research and I don't work in this space. My goal is to make work in this space. My goal is to make work in this space. My goal is to make coders faster and your your work is to coders faster and your your work is to coders faster and your your work is to make scientists faster. I would think make scientists faster. I would think make scientists faster. I would think though that a low-quality memory could though that a low-quality memory could though that a low-quality memory could bias or mislead a run later. Absolutely.

bias or mislead a run later. Absolutely. bias or mislead a run later. Absolutely. Okay, but you said there are no bad Okay, but you said there are no bad Okay, but you said there are no bad memories. Is there a risk of a bad memories. Is there a risk of a bad memories. Is there a risk of a bad memory compounding and how would you memory compounding and how would you memory compounding and how would you pull out like that was a bad idea in a pull out like that was a bad idea in a pull out like that was a bad idea in a brainstorm and we're going to throw that brainstorm and we're going to throw that brainstorm and we're going to throw that one out? Absolutely. So, that's where one out? Absolutely. So, that's where one out? Absolutely. So, that's where this the between the engine and the this the between the engine and the this the between the engine and the store, there's signals. They're called K store, there's signals. They're called K store, there's signals. They're called K signals between the engine and the store signals between the engine and the store signals between the engine and the store that says they're measuring the efficacy that says they're measuring the efficacy that says they're measuring the efficacy of the actual memory to the problem. Is of the actual memory to the problem. Is of the actual memory to the problem. Is this irrelevant? We're giving it a this irrelevant? We're giving it a this irrelevant? We're giving it a confidence score relevance of one confidence score relevance of one confidence score relevance of one through five, right? So, one to two are through five, right? So, one to two are through five, right? So, one to two are low-value memories. Low-value memories, low-value memories. Low-value memories, low-value memories. Low-value memories, they're keep them in the store, but they're keep them in the store, but they're keep them in the store, but they're low-value memories for this they're low-value memories for this they're low-value memories for this problem. Do not ground in ground in the problem. Do not ground in ground in the problem. Do not ground in ground in the problem in the in these memories. A problem in the in these memories. A problem in the in these memories. A three or above three or above three or above are groundable memories, meaning they are groundable memories, meaning they are groundable memories, meaning they have some level of fidelity to the have some level of fidelity to the have some level of fidelity to the problem. problem. problem. >> Okay, there it is. It just clicked for >> Okay, there it is. It just clicked for >> Okay, there it is. It just clicked for me. So, you have a library and you don't me. So, you have a library and you don't me. So, you have a library and you don't know if the stuff in the library is crap know if the stuff in the library is crap know if the stuff in the library is crap or not. So, you're going to keep it all or not. So, you're going to keep it all or not. So, you're going to keep it all and some of it might be pulp fiction and and some of it might be pulp fiction and and some of it might be pulp fiction and some of it might be the great American some of it might be the great American some of it might be the great American novel. But, if you're doing a problem novel. But, if you're doing a problem novel. But, if you're doing a problem about pulp fiction, you'll go and you'll about pulp fiction, you'll go and you'll about pulp fiction, you'll go and you'll get the crappy memories and they're get the crappy memories and they're get the crappy memories and they're going to lead someone in some direction.

going to lead someone in some direction. going to lead someone in some direction. So, you throw nothing away, but you also So, you throw nothing away, but you also So, you throw nothing away, but you also don't have to keep everything in don't have to keep everything in don't have to keep everything in context. You just pull the bits that are context. You just pull the bits that are context. You just pull the bits that are that are pertinent. that are pertinent. that are pertinent. >> Absolutely. There's that There's glue. >> Absolutely. There's that There's glue. >> Absolutely. There's that There's glue. There's this interaction, this dynamic There's this interaction, this dynamic There's this interaction, this dynamic interaction between the cognitive engine interaction between the cognitive engine interaction between the cognitive engine and the memory store. That really the and the memory store. That really the and the memory store. That really the work between the two because these are work between the two because these are work between the two because these are two separate things, right? We work with two separate things, right? We work with two separate things, right? We work with the the MD65 research team on on their the the MD65 research team on on their the the MD65 research team on on their memory store and we built the cognitive memory store and we built the cognitive memory store and we built the cognitive engine. So, the first part of the engine. So, the first part of the engine. So, the first part of the activity was marrying up the signals if activity was marrying up the signals if activity was marrying up the signals if we call it, right? The signals of when we call it, right? The signals of when we call it, right? The signals of when something is relevant, when to call the something is relevant, when to call the something is relevant, when to call the memory store from the cognitive engine, memory store from the cognitive engine, memory store from the cognitive engine, right? Of course, when you're starting a right? Of course, when you're starting a right? Of course, when you're starting a problem, absolutely call the memory problem, absolutely call the memory problem, absolutely call the memory store. But, when you're doing subtasking store. But, when you're doing subtasking store. But, when you're doing subtasking or starting new channels of activity, or starting new channels of activity, or starting new channels of activity, that's another opportunity for you to that's another opportunity for you to that's another opportunity for you to call the memory store to get memories call the memory store to get memories call the memory store to get memories out of the system, rank them. out of the system, rank them. out of the system, rank them. If they're efficient for the problem, If they're efficient for the problem, If they're efficient for the problem, use them. If they're not efficient, keep use them. If they're not efficient, keep use them. If they're not efficient, keep them in the store and use it. And this them in the store and use it. And this them in the store and use it. And this whole idea around Think about it for a whole idea around Think about it for a whole idea around Think about it for a second. Us second. Us second. Us a user gets a scientist gets sit down a user gets a scientist gets sit down a user gets a scientist gets sit down sit sit down his computer says, "I have sit sit down his computer says, "I have sit sit down his computer says, "I have this problem." this problem." this problem." We can query an enterprise memory store We can query an enterprise memory store We can query an enterprise memory store and say, "Hey, you're trying to do and say, "Hey, you're trying to do and say, "Hey, you're trying to do immunology. Here's all the memories on immunology. Here's all the memories on immunology. Here's all the memories on immunology. Would you like to use it?"

immunology. Would you like to use it?" immunology. Would you like to use it?" And somebody would say, "Well, why would And somebody would say, "Well, why would And somebody would say, "Well, why would we want to surface that to the we want to surface that to the we want to surface that to the scientist? We just have AI do it scientist? We just have AI do it scientist? We just have AI do it automatically." automatically." automatically." Scientists don't want to just be Scientists don't want to just be Scientists don't want to just be disenfranchised from science, right? disenfranchised from science, right? disenfranchised from science, right? They feel like science shouldn't be They feel like science shouldn't be They feel like science shouldn't be science. AI is a tool for them to do science. AI is a tool for them to do science. AI is a tool for them to do science. They feel like they could science. They feel like they could science. They feel like they could they're SMEs. They can look at the they're SMEs. They can look at the they're SMEs. They can look at the memories and say, "Yeah, these I'll memories and say, "Yeah, these I'll memories and say, "Yeah, these I'll select them." But, behind the scenes, select them." But, behind the scenes, select them." But, behind the scenes, we're doing the automatic selection as we're doing the automatic selection as we're doing the automatic selection as well. That whole cognitive engine dance well. That whole cognitive engine dance well. That whole cognitive engine dance behind the scenes that we're doing behind the scenes that we're doing behind the scenes that we're doing between the signals of the cognitive between the signals of the cognitive between the signals of the cognitive engine and the memory store, we're doing engine and the memory store, we're doing engine and the memory store, we're doing that as well. So, in addition to the that as well. So, in addition to the that as well. So, in addition to the scientist picking some of this task, scientist picking some of this task, scientist picking some of this task, right? So, it starts to look like right? So, it starts to look like right? So, it starts to look like this is a learning system. So, last this is a learning system. So, last this is a learning system. So, last week, I didn't have any week, I didn't have any week, I didn't have any information-based memories. This week I information-based memories. This week I information-based memories. This week I have 1,500 information-based memories. have 1,500 information-based memories. have 1,500 information-based memories. How did that get there? How did that get there? How did that get there? Other colleagues doing research. So, the Other colleagues doing research. So, the Other colleagues doing research. So, the system is continually to grow grow from system is continually to grow grow from system is continually to grow grow from that research, right? So, yeah, very that research, right? So, yeah, very that research, right? So, yeah, very exciting around that. Okay, so someone exciting around that. Okay, so someone exciting around that. Okay, so someone might be if they made it 20 minutes into might be if they made it 20 minutes into might be if they made it 20 minutes into the show here, they might be thinking, the show here, they might be thinking, the show here, they might be thinking, "Well, hang on. I can't get Claude to "Well, hang on. I can't get Claude to "Well, hang on. I can't get Claude to make me a mediocre website and you're make me a mediocre website and you're make me a mediocre website and you're making drugs." Like, this thing, you making drugs." Like, this thing, you making drugs." Like, this thing, you know, I talked to Rusevitch all the time know, I talked to Rusevitch all the time know, I talked to Rusevitch all the time and he's always telling me how the and he's always telling me how the and he's always telling me how the thing's gaslighting him, commenting out thing's gaslighting him, commenting out thing's gaslighting him, commenting out tests. Mhm. If ever there were a thing tests. Mhm. If ever there were a thing tests. Mhm. If ever there were a thing that needed to be grounded in data, it that needed to be grounded in data, it that needed to be grounded in data, it would be something like this.

would be something like this. would be something like this. >> Oh, yeah. How do you ground it in data? >> Oh, yeah. How do you ground it in data? >> Oh, yeah. How do you ground it in data? Cuz I know that there's a far more Cuz I know that there's a far more Cuz I know that there's a far more There's far more data around like drug There's far more data around like drug There's far more data around like drug repurposing and things like that that's repurposing and things like that that's repurposing and things like that that's like that's testable. You need to stop like that's testable. You need to stop like that's testable. You need to stop hallucinations at the at the root. hallucinations at the at the root. hallucinations at the at the root. Yeah, when we first started our journey Yeah, when we first started our journey Yeah, when we first started our journey down the AI for science journey, right? down the AI for science journey, right? down the AI for science journey, right? We call that area knowledge, right? We call that area knowledge, right? We call that area knowledge, right? Knowledge. How much knowledge do you Knowledge. How much knowledge do you Knowledge. How much knowledge do you need to get to give to the system to do need to get to give to the system to do need to get to give to the system to do science? Cuz remember when we started science? Cuz remember when we started science? Cuz remember when we started out, out, out, um going to a a large language model um going to a a large language model um going to a a large language model says, "Give me something with SMILES says, "Give me something with SMILES says, "Give me something with SMILES notation." notation." notation." The large language model would The large language model would The large language model would understand what SMILES notation was, but understand what SMILES notation was, but understand what SMILES notation was, but submit to you something that you could submit to you something that you could submit to you something that you could look at and you say, "This is totally, look at and you say, "This is totally, look at and you say, "This is totally, absolutely, positively, 1,000% wrong." absolutely, positively, 1,000% wrong." absolutely, positively, 1,000% wrong." Right, cuz SMILES is Simplified Right, cuz SMILES is Simplified Right, cuz SMILES is Simplified Molecular Input Line Entry System. It's Molecular Input Line Entry System. It's Molecular Input Line Entry System. It's basically ASCII for chemicals. For basically ASCII for chemicals. For basically ASCII for chemicals. For chemicals. Exactly. chemicals. Exactly. chemicals. Exactly. >> know that. It doesn't come baked in and >> know that. It doesn't come baked in and >> know that. It doesn't come baked in and it can probably gaslight you and make up it can probably gaslight you and make up it can probably gaslight you and make up all kinds of nonsense. And it absolutely all kinds of nonsense. And it absolutely all kinds of nonsense. And it absolutely was, right? So, you know, large language was, right? So, you know, large language was, right? So, you know, large language models are not great for science without models are not great for science without models are not great for science without proper grounding. So, we we we work with proper grounding. So, we we we work with proper grounding. So, we we we work with MSR and we built this capability called MSR and we built this capability called MSR and we built this capability called Graph RAG, right? So, Graph RAG is how Graph RAG, right? So, Graph RAG is how Graph RAG, right? So, Graph RAG is how do you use knowledge graph? So, moving do you use knowledge graph? So, moving do you use knowledge graph? So, moving beyond RAG to, you know, RAG is, you beyond RAG to, you know, RAG is, you beyond RAG to, you know, RAG is, you know, you know, very is not in dynamic, know, you know, very is not in dynamic, know, you know, very is not in dynamic, right? It used to fixed ontology. You right? It used to fixed ontology. You right? It used to fixed ontology. You put your data in there.

put your data in there. put your data in there. It doesn't grow. It's just Here's what I It doesn't grow. It's just Here's what I It doesn't grow. It's just Here's what I have. Here's what I'll ground. But, when have. Here's what I'll ground. But, when have. Here's what I'll ground. But, when you build in a knowledge graph or you you build in a knowledge graph or you you build in a knowledge graph or you use graphs, graphs are dynamic ontology. use graphs, graphs are dynamic ontology. use graphs, graphs are dynamic ontology. So, the more data you feed it, the more So, the more data you feed it, the more So, the more data you feed it, the more relationships it builds and the more relationships it builds and the more relationships it builds and the more communities it builds as part of the communities it builds as part of the communities it builds as part of the data. So, we built what we call a data. So, we built what we call a data. So, we built what we call a scientific bookshelf. So, hey Mr. scientific bookshelf. So, hey Mr. scientific bookshelf. So, hey Mr. Scientist, you're a chemist. What is Scientist, you're a chemist. What is Scientist, you're a chemist. What is your bookshelf? your bookshelf? your bookshelf? Mhm. Mhm. Mhm. Your materials that you use to do Your materials that you use to do Your materials that you use to do science and your tooling that you use to science and your tooling that you use to science and your tooling that you use to do science. Those two things are super do science. Those two things are super do science. Those two things are super important, right? And we can incorporate important, right? And we can incorporate important, right? And we can incorporate those into the discovery system, the those into the discovery system, the those into the discovery system, the knowledge and the tools. Once you have knowledge and the tools. Once you have knowledge and the tools. Once you have that, you marry that with fine-tune that, you marry that with fine-tune that, you marry that with fine-tune models, science models, and fine-tune models, science models, and fine-tune models, science models, and fine-tune knowledge graphs of chemistry, bio, knowledge graphs of chemistry, bio, knowledge graphs of chemistry, bio, depending on what what area science depending on what what area science depending on what what area science you're working on, and you use that to you're working on, and you use that to you're working on, and you use that to ground your answers in. And that's how ground your answers in. And that's how ground your answers in. And that's how you get around this whole, "Hey, a large you get around this whole, "Hey, a large you get around this whole, "Hey, a large language models are not good for language models are not good for language models are not good for science. They hallucinate." How do you science. They hallucinate." How do you science. They hallucinate." How do you get to a hallucination-free system, get to a hallucination-free system, get to a hallucination-free system, right? There you go. So, I've been right? There you go. So, I've been right? There you go. So, I've been calling them and I'm trying to get this calling them and I'm trying to get this calling them and I'm trying to get this to catch on, ambiguity loops. And if to catch on, ambiguity loops. And if to catch on, ambiguity loops. And if you've got a loop, like a for loop is you've got a loop, like a for loop is you've got a loop, like a for loop is very unambiguous. It's very clear. It's very unambiguous. It's very clear. It's very unambiguous. It's very clear. It's procedural. It runs exactly as you'd procedural. It runs exactly as you'd procedural. It runs exactly as you'd expect. But, an ambiguity loop says, expect. But, an ambiguity loop says, expect. But, an ambiguity loop says, "Hey, you know, AI, go and do a thing."

"Hey, you know, AI, go and do a thing." "Hey, you know, AI, go and do a thing." Mhm. And I didn't tell you how to do the Mhm. And I didn't tell you how to do the Mhm. And I didn't tell you how to do the thing. I say, "Hey, go get me bread and thing. I say, "Hey, go get me bread and thing. I say, "Hey, go get me bread and milk." Didn't say take the bus. Didn't milk." Didn't say take the bus. Didn't milk." Didn't say take the bus. Didn't say walk. You decide. And sometimes say walk. You decide. And sometimes say walk. You decide. And sometimes they're really good at that, but there's they're really good at that, but there's they're really good at that, but there's the car wash example that everyone the car wash example that everyone the car wash example that everyone thinks is funny where you tell the AI, thinks is funny where you tell the AI, thinks is funny where you tell the AI, "I need to go and get my car washed. The "I need to go and get my car washed. The "I need to go and get my car washed. The car wash is about 300 m away. Should I car wash is about 300 m away. Should I car wash is about 300 m away. Should I drive or should I walk?" drive or should I walk?" drive or should I walk?" And many many AIs will say, "Well, it And many many AIs will say, "Well, it And many many AIs will say, "Well, it would be silly to drive." Yeah. would be silly to drive." Yeah. would be silly to drive." Yeah. >> And then they have you walking off to >> And then they have you walking off to >> And then they have you walking off to the the thing, but you get it doesn't the the thing, but you get it doesn't the the thing, but you get it doesn't realize that it needs to go there in realize that it needs to go there in realize that it needs to go there in order to wash the car and take the car order to wash the car and take the car order to wash the car and take the car with it. But, ambiguity loops are with it. But, ambiguity loops are with it. But, ambiguity loops are sometimes helpful because you leave the sometimes helpful because you leave the sometimes helpful because you leave the part that is ambiguous because you don't part that is ambiguous because you don't part that is ambiguous because you don't know. Yes. know. Yes. know. Yes. >> And you want it to explore that space. >> And you want it to explore that space. >> And you want it to explore that space. Mhm. But, when you do know stuff like Mhm. But, when you do know stuff like Mhm. But, when you do know stuff like like scientific data, it must never like scientific data, it must never like scientific data, it must never fabricate that data. fabricate that data. fabricate that data. >> Mhm. Yeah, must never fab because, you >> Mhm. Yeah, must never fab because, you >> Mhm. Yeah, must never fab because, you know, the difference in a SMILES know, the difference in a SMILES know, the difference in a SMILES notation is a different drug. It's a notation is a different drug. It's a notation is a different drug. It's a single byte. Yeah, like, you know, single byte. Yeah, like, you know, single byte. Yeah, like, you know, Yeah, that's a great point. You should Yeah, that's a great point. You should Yeah, that's a great point. You should go and people who are listening should go and people who are listening should go and people who are listening should go and look up SMILES. The the go and look up SMILES. The the go and look up SMILES. The the specification. It's it's such an specification. It's it's such an specification. It's it's such an important thing. Like, you know, ethanol important thing. Like, you know, ethanol important thing. Like, you know, ethanol is like CCO, but you change one is like CCO, but you change one is like CCO, but you change one character and it's a, you know, what do character and it's a, you know, what do character and it's a, you know, what do they say? You do you remember the old they say? You do you remember the old they say? You do you remember the old thing in chemistry school? What was it?

thing in chemistry school? What was it? thing in chemistry school? What was it? Tommy was a chemist, but Tommy is no Tommy was a chemist, but Tommy is no Tommy was a chemist, but Tommy is no more. What Tommy thought was H2O was more. What Tommy thought was H2O was more. What Tommy thought was H2O was H2SO4. H2SO4. H2SO4. >> [laughter] >> [laughter] >> [laughter] >> I didn't mean that one. >> I didn't mean that one. >> I didn't mean that one. So, like stuff like that, you just don't So, like stuff like that, you just don't So, like stuff like that, you just don't want to make those kind of mistakes or want to make those kind of mistakes or want to make those kind of mistakes or otherwise you're going to be drinking otherwise you're going to be drinking otherwise you're going to be drinking sulfuric acid, Tommy. sulfuric acid, Tommy. sulfuric acid, Tommy. >> Absolutely. Look, look Uh look, at my >> Absolutely. Look, look Uh look, at my >> Absolutely. Look, look Uh look, at my journey I look, I had my own journey to journey I look, I had my own journey to journey I look, I had my own journey to science, right? I'm a computer science, right? I'm a computer science, right? I'm a computer scientist. And I look, when we found out scientist. And I look, when we found out scientist. And I look, when we found out that we're going to work in the AI for that we're going to work in the AI for that we're going to work in the AI for science, you know, I got to spend some science, you know, I got to spend some science, you know, I got to spend some time with scientists. And that was very time with scientists. And that was very time with scientists. And that was very That was very eye-opening, right? As you That was very eye-opening, right? As you That was very eye-opening, right? As you get to kind of spend time with get to kind of spend time with get to kind of spend time with scientists. First of First of all, they scientists. First of First of all, they scientists. First of First of all, they said, "Look, hey, said, "Look, hey, said, "Look, hey, AI guy, AI guy, AI guy, science scientists do science. Do you science scientists do science. Do you science scientists do science. Do you understand?" And I was like, "I understand?" And I was like, "I understand?" And I was like, "I understand." understand." understand." Mhm. So, AI is They're like, "Repeat it Mhm. So, AI is They're like, "Repeat it Mhm. So, AI is They're like, "Repeat it back to me. What did you hear? AI is a back to me. What did you hear? AI is a back to me. What did you hear? AI is a tool. Scientists do science." They're tool. Scientists do science." They're tool. Scientists do science." They're like, "Okay, we like this guy." like, "Okay, we like this guy." like, "Okay, we like this guy." >> There you go. There you go. They got an >> There you go. There you go. They got an >> There you go. There you go. They got an understanding of what we're trying to do understanding of what we're trying to do understanding of what we're trying to do here, right? So, they will be in the here, right? So, they will be in the here, right? So, they will be in the loop in this process, right? Even though loop in this process, right? Even though loop in this process, right? Even though we're trying to another area that I'm we're trying to another area that I'm we're trying to another area that I'm working on right now is area called lab working on right now is area called lab working on right now is area called lab in the loop. Right? So, you think about in the loop. Right? So, you think about in the loop. Right? So, you think about the science, the hypothesis is I'm going the science, the hypothesis is I'm going the science, the hypothesis is I'm going to give you feed you some stuff about a to give you feed you some stuff about a to give you feed you some stuff about a chemical property and I want you to come chemical property and I want you to come chemical property and I want you to come up with a new chemical. Candidates for a up with a new chemical. Candidates for a up with a new chemical. Candidates for a chemical.

chemical. chemical. What about making the chemical? Mhm. What about making the chemical? Mhm. What about making the chemical? Mhm. Now, I want to take that formula lab Now, I want to take that formula lab Now, I want to take that formula lab protocol. I want to submit it to a bunch protocol. I want to submit it to a bunch protocol. I want to submit it to a bunch of robots in the lab and I want to of robots in the lab and I want to of robots in the lab and I want to synthesize chemical. synthesize chemical. synthesize chemical. And this is a real thing. And the good And this is a real thing. And the good And this is a real thing. And the good beauty of that is it brings AI, beauty of that is it brings AI, beauty of that is it brings AI, physical environments together in physical environments together in physical environments together in science at the same time. So, you get a science at the same time. So, you get a science at the same time. So, you get a bunch of autonomous system, you get some bunch of autonomous system, you get some bunch of autonomous system, you get some robotics in there, you get some AI and robotics in there, you get some AI and robotics in there, you get some AI and science at the same time. So, really science at the same time. So, really science at the same time. So, really exciting area of focus. exciting area of focus. exciting area of focus. >> Interesting. And then the last thing >> Interesting. And then the last thing >> Interesting. And then the last thing that I wanted to touch on cuz I thought that I wanted to touch on cuz I thought that I wanted to touch on cuz I thought it was interesting as we get towards the it was interesting as we get towards the it was interesting as we get towards the end here was that, you know, we see end here was that, you know, we see end here was that, you know, we see these kind of random emergent things these kind of random emergent things these kind of random emergent things happen. Like, it's like, "I didn't happen. Like, it's like, "I didn't happen. Like, it's like, "I didn't expect it to do that." You would not expect it to do that." You would not expect it to do that." You would not expect a large language model or any expect a large language model or any expect a large language model or any system to system to system to try to develop a discipline of try to develop a discipline of try to develop a discipline of anti-fabrication to They they tend to anti-fabrication to They they tend to anti-fabrication to They they tend to float away like balloons and in helium. float away like balloons and in helium. float away like balloons and in helium. And they tend to kind of fall off the And they tend to kind of fall off the And they tend to kind of fall off the rails and they just start making stuff rails and they just start making stuff rails and they just start making stuff up. But, in the building of this system, up. But, in the building of this system, up. But, in the building of this system, you started to notice it repeatedly you started to notice it repeatedly you started to notice it repeatedly policing itself, catching itself, and policing itself, catching itself, and policing itself, catching itself, and then regrounding in the data. Why do you then regrounding in the data. Why do you then regrounding in the data. Why do you think that happened?

think that happened? think that happened? Memory. Mhm. The effect of memory on the Memory. Mhm. The effect of memory on the Memory. Mhm. The effect of memory on the system, the ability for it to say, "Hey, system, the ability for it to say, "Hey, system, the ability for it to say, "Hey, I have a challenging area." Like, it's I have a challenging area." Like, it's I have a challenging area." Like, it's something in our cognitive system. something in our cognitive system. something in our cognitive system. There's There's path There's A path A, There's There's path There's A path A, There's There's path There's A path A, which is a preferred path, and I want which is a preferred path, and I want which is a preferred path, and I want you to use these tools. Then there's you to use these tools. Then there's you to use these tools. Then there's path B, non-preferred. You're work path B, non-preferred. You're work path B, non-preferred. You're work Something happened. There's a transient Something happened. There's a transient Something happened. There's a transient failure infrastructure. I'm not able to failure infrastructure. I'm not able to failure infrastructure. I'm not able to get this tool. It's another path for get this tool. It's another path for get this tool. It's another path for being able to research and get to a being able to research and get to a being able to research and get to a result. That's one thing our cognitive result. That's one thing our cognitive result. That's one thing our cognitive engine will do. Because these things run engine will do. Because these things run engine will do. Because these things run in a long wrong running multiple day, in a long wrong running multiple day, in a long wrong running multiple day, the cognitive engine will get to a the cognitive engine will get to a the cognitive engine will get to a result. It's trained to get to a result. result. It's trained to get to a result. result. It's trained to get to a result. Failure in this, new path. Failure X, Failure in this, new path. Failure X, Failure in this, new path. Failure X, read a new path, right? It will read a new path, right? It will read a new path, right? It will continually try to execute new paths. continually try to execute new paths. continually try to execute new paths. And by adding this dynamic memory And by adding this dynamic memory And by adding this dynamic memory capability to it where before the path capability to it where before the path capability to it where before the path was, all right, let me ask the large was, all right, let me ask the large was, all right, let me ask the large language model. It says, "I'm not able language model. It says, "I'm not able language model. It says, "I'm not able to get to a coding agent. Let me ask the to get to a coding agent. Let me ask the to get to a coding agent. Let me ask the large language model to write code for large language model to write code for large language model to write code for me or something like that, right?" So, me or something like that, right?" So, me or something like that, right?" So, in this area, hey, I'm stuck a little in this area, hey, I'm stuck a little in this area, hey, I'm stuck a little bit here. bit here. bit here. Guess what? I have a memory store. Let Guess what? I have a memory store. Let Guess what? I have a memory store. Let me query the memory store for actions or me query the memory store for actions or me query the memory store for actions or activity or insight on how to generate activity or insight on how to generate activity or insight on how to generate the next problem, right? So, the memory the next problem, right? So, the memory the next problem, right? So, the memory store has been really It's been really a store has been really It's been really a store has been really It's been really a breakthrough. It's been a in the breakthrough. It's been a in the breakthrough. It's been a in the interaction between the cognitive engine interaction between the cognitive engine interaction between the cognitive engine and the memory store. That interaction, and the memory store. That interaction, and the memory store. That interaction, right? The ability to pull memories, use right? The ability to pull memories, use right? The ability to pull memories, use memories at runtime is effectively memories at runtime is effectively memories at runtime is effectively changed a lot of behavior in long changed a lot of behavior in long changed a lot of behavior in long running jobs, which scientists science running jobs, which scientists science running jobs, which scientists science jobs are, right? So, where where would jobs are, right? So, where where would jobs are, right? So, where where would we learn more? Like, is this Are there we learn more? Like, is this Are there we learn more? Like, is this Are there papers? What am I What am I Googling for papers? What am I What am I Googling for papers? What am I What am I Googling for to learn about this stuff?

to learn about this stuff? to learn about this stuff? >> Yeah, the first thing I'd like to direct >> Yeah, the first thing I'd like to direct >> Yeah, the first thing I'd like to direct you to is the product that the division you to is the product that the division you to is the product that the division is building around for scientists to do is building around for scientists to do is building around for scientists to do um science. science and that's called um science. science and that's called um science. science and that's called Microsoft Discovery. Okay. So, Microsoft Microsoft Discovery. Okay. So, Microsoft Microsoft Discovery. Okay. So, Microsoft Discovery. Great great tooling. You Discovery. Great great tooling. You Discovery. Great great tooling. You start to introduce some of the concepts start to introduce some of the concepts start to introduce some of the concepts that we talked about. Mhm. If you want that we talked about. Mhm. If you want that we talked about. Mhm. If you want to know more about our our cognitive to know more about our our cognitive to know more about our our cognitive engine, engine, engine, there's a paper that we put out for I'll there's a paper that we put out for I'll there's a paper that we put out for I'll send you the link. I'll send you the send you the link. I'll send you the send you the link. I'll send you the link. link. link. >> I'll put it in the show notes. Yeah, it >> I'll put it in the show notes. Yeah, it >> I'll put it in the show notes. Yeah, it tells you how our cognitive engine goes tells you how our cognitive engine goes tells you how our cognitive engine goes about prosecuting its work and doing its about prosecuting its work and doing its about prosecuting its work and doing its activities. And then some of the memory activities. And then some of the memory activities. And then some of the memory research that I'm working on right now, research that I'm working on right now, research that I'm working on right now, hopefully in the future I'll be able to hopefully in the future I'll be able to hopefully in the future I'll be able to we'll be able to publish some of the we'll be able to publish some of the we'll be able to publish some of the memory research and the effect of memory memory research and the effect of memory memory research and the effect of memory on scientific problems moving forward, on scientific problems moving forward, on scientific problems moving forward, so. Very cool. Well, thank you so much so. Very cool. Well, thank you so much so. Very cool. Well, thank you so much Alan Stewart for hanging out with me Alan Stewart for hanging out with me Alan Stewart for hanging out with me today. I I've got a lot of reading to do today. I I've got a lot of reading to do today. I I've got a lot of reading to do it sounds like. it sounds like. it sounds like. Oh, come on. Absolutely anytime, Scott. Oh, come on. Absolutely anytime, Scott. Oh, come on. Absolutely anytime, Scott. You know that. All right. I appreciate You know that. All right. I appreciate You know that. All right. I appreciate you. you. you. This has been another episode of This has been another episode of This has been another episode of Hanselminutes and we'll see you again Hanselminutes and we'll see you again Hanselminutes and we'll see you again next week.