I Read Goodfire's New Essay. AI May Be Far Easier To Understand Than We Thought.

I hear so much I hear so much bad news about AI. bad news about AI. bad news about AI. Today is a day of good Today is a day of good Today is a day of good news. A little-known news. A little-known news. A little-known organization organization organization called Goodfire may have called Goodfire may have called Goodfire may have given us a great given us a great given us a great reason reason reason to look with more optimism to look with more optimism to look with more optimism at what OpenAI and Anthropic at what OpenAI and Anthropic at what OpenAI and Anthropic will create next. will create next. will create next. Their new Their new Their new study study study claims that the claims that the claims that the inner workings of inner workings of inner workings of AI are much AI are much AI are much clearer than clearer than clearer than researchers researchers researchers previously believed. If this previously believed. If this previously believed. If this is confirmed, we is confirmed, we is confirmed, we will be able to better will be able to better will be able to better detect dangerous detect dangerous detect dangerous behavior and behavior and behavior and correct its correct its correct its causes, and I think causes, and I think causes, and I think this is so this is so this is so important that it changes important that it changes important that it changes the outlook for the outlook for the outlook for AI safety. And I expect you'll AI safety. And I expect you'll AI safety. And I expect you'll start seeing start seeing start seeing the effects of this work the effects of this work the effects of this work in the products you in the products you in the products you use every day use every day use every day next year, next year, next year, and maybe even sooner. and maybe even sooner. and maybe even sooner. This is a powerful signal of This is a powerful signal of This is a powerful signal of optimism for our optimism for our optimism for our future, and I want to future, and I want to future, and I want to show you why, show you why, show you why, to showcase this to showcase this to showcase this work, what I work, what I work, what I took away from it, and what it means took away from it, and what it means took away from it, and what it means for all of us in 2027 for all of us in 2027 . Imagine . Imagine . Imagine asking an AI to fix a asking an AI to fix a asking an AI to fix a bug in your bug in your software. He software. He will say that the work will say that the work will say that the work is done, but the error is done, but the error is done, but the error may remain, may remain, may remain, right? The problem of right? The problem of right? The problem of hallucinations. You hallucinations. You hallucinations. You want want want AI developers to be able to AI developers to be able to AI developers to be able to investigate what happened investigate what happened investigate what happened in such cases, in such cases, in such cases, find out if find out if find out if it learned to take the it learned to take the it learned to take the path of least path of least path of least resistance, and fix resistance, and fix resistance, and fix the cause in future the cause in future the cause in future models so it doesn't models so it doesn't models so it doesn't happen again. Better happen again. Better happen again. Better tools for tools for tools for this could this could this could make it much easier to trust make it much easier to trust make it much easier to trust AI in important work.

AI in important work. We have long been able to We have long been able to measure whether measure whether measure whether AI produces useful AI produces useful AI produces useful results, but we have spent a results, but we have spent a results, but we have spent a very long time very long time very long time trying to explain trying to explain trying to explain why it made a particular why it made a particular why it made a particular decision. This is not a decision. This is not a decision. This is not a new problem. Here is a new problem. Here is a new problem. Here is a truly impressive truly impressive truly impressive result. Nick result. Nick result. Nick Cammarata reports Cammarata reports that 99 to 100% of that 99 to 100% of the neurons in the neurons in the neurons in the final layer of the the final layer of the the final layer of the advanced advanced advanced vision model are vision model are vision model are interpretable. This interpretable. This interpretable. This means that we can means that we can means that we can understand all the understand all the understand all the components of a components of a components of a particular model. This does particular model. This does particular model. This does not mean that AI not mean that AI not mean that AI immediately became 100% immediately became 100% safe. The broader safe. The broader safe. The broader argument is argument is argument is that we that we that we can now understand can now understand can now understand models as models as models as they become they become they become more powerful. So more powerful. So more powerful. So the likelihood of the likelihood of the likelihood of AI pursuing the AI pursuing the AI pursuing the wrong goal is wrong goal is wrong goal is greatly reduced greatly reduced greatly reduced because we have a because we have a because we have a way of understanding the way of understanding the way of understanding the workings of each workings of each workings of each individual neuron. individual neuron. individual neuron. You may be wondering You may be wondering You may be wondering what GoodFire is. GoodFire is an what GoodFire is. GoodFire is an what GoodFire is. GoodFire is an AI research AI research company that builds company that builds company that builds tools to tools to tools to understand what's understand what's understand what's going on going on going on inside AI models. inside AI models. inside AI models. Its founders include Its founders include researchers who researchers who researchers who have worked on this have worked on this have worked on this problem at OpenAI, Google problem at OpenAI, Google problem at OpenAI, Google DeepMind, and elsewhere.

DeepMind, and elsewhere. DeepMind, and elsewhere. The goal is to The goal is to The goal is to help help help developers explain developers explain developers explain model behavior, model behavior, model behavior, identify problems, and identify problems, and identify problems, and make make make targeted targeted targeted fixes. I believe fixes. I believe this points to this points to this points to four changes to four changes to four changes to watch for as we head watch for as we head watch for as we head towards 2027. towards 2027. towards 2027. Number one: AI that is Number one: AI that is Number one: AI that is safer safer safer to delegate tasks to. to delegate tasks to. to delegate tasks to. Number two—more Number two—more Number two—more confidence in confidence in confidence in safely building safely building safely building powerful models. powerful models. powerful models. Number three— Number three— professional professional professional tools that tools that tools that help you help you help you research the research the research the decisions you make. And decisions you make. And decisions you make. And number four— number four— finally, everyday finally, everyday finally, everyday products that products that products that allow you allow you allow you to adjust what to adjust what to adjust what the algorithm has learned the algorithm has learned the algorithm has learned about you. We're about you. We're about you. We're going to figure going to figure going to figure all of this out. So, all of this out. So, all of this out. So, let me explain let me explain let me explain why this is so important from why this is so important from why this is so important from my perspective as my perspective as my perspective as someone who has someone who has someone who has worked within the worked within the worked within the AI ​​field for a long time, even before AI ​​field for a long time, even before AI ​​field for a long time, even before ChatGPT came along. When I worked ChatGPT came along. When I worked ChatGPT came along. When I worked on machine on machine on machine learning at Amazon, we learning at Amazon, we learning at Amazon, we could measure whether could measure whether could measure whether our our our recommendations worked. Did recommendations worked. Did recommendations worked. Did anyone watch anything? anyone watch anything? anyone watch anything? Did the system produce a Did the system produce a Did the system produce a useful result? useful result? useful result? But explaining why But explaining why was much was much was much more difficult or even more difficult or even more difficult or even impossible. impossible. impossible. We have had cases We have had cases We have had cases where we had where we had where we had customer groups, customer groups, customer groups, product groups, or product groups, or product groups, or name groups. And you might know name groups. And you might know that grouping is that grouping is that grouping is useful, but it useful, but it useful, but it was still hard was still hard was still hard to explain what exactly to explain what exactly to explain what exactly the model learned about the model learned about the model learned about these people or names these people or names these people or names to group them to group them to group them that way. This that way. This that way. This is called a "black is called a "black is called a "black box." This is a very box." This is a very box." This is a very well-known problem in well-known problem in well-known problem in machine learning.

machine learning. machine learning. This is certainly not This is certainly not This is certainly not unique to Amazon. unique to Amazon. unique to Amazon. This is how AI has worked for This is how AI has worked for This is how AI has worked for many years. The people who many years. The people who many years. The people who create these systems create these systems know a lot about them, but know a lot about them, but measuring a great measuring a great measuring a great outcome and outcome and outcome and explaining the process explaining the process are two different things. When are two different things. When are two different things. When the results become the results become the results become good enough, we good enough, we good enough, we usually continue usually continue usually continue to optimize them. And what to optimize them. And what to optimize them. And what really bothers me really bothers me is the idea that this is the is the idea that this is the is the idea that this is the result of laziness, result of laziness, result of laziness, commercialism, or commercialism, or commercialism, or that we simply didn't that we simply didn't that we simply didn't take the time to do it. I'm take the time to do it. I'm take the time to do it. I'm here to tell you: here to tell you: here to tell you: this is a feature of this is a feature of this is a feature of computer science. computer science. computer science. For a long time, we For a long time, we For a long time, we couldn't read how couldn't read how couldn't read how models made models made models made decisions, and this was an decisions, and this was an decisions, and this was an inherent problem in inherent problem in model research and science. It model research and science. It just was the way it just was the way it just was the way it was. Now that is was. Now that is was. Now that is changing. I can't changing. I can't changing. I can't express how express how express how important this is. And you know, important this is. And you know, important this is. And you know, you and I—we also lived with you and I—we also lived with you and I—we also lived with this system. You this system. You this system. You watch one video watch one video watch one video and your phone and your phone and your phone seems to decide that seems to decide that seems to decide that this is your entire this is your entire this is your entire personality. And you personality. And you personality. And you can reject the can reject the can reject the next next next recommendation, but you recommendation, but you recommendation, but you usually can't usually can't usually can't check the conclusion check the conclusion check the conclusion he made about he made about he made about you and correct that you and correct that you and correct that particular particular particular misunderstanding. So misunderstanding. So misunderstanding. So keep that thought in keep that thought in keep that thought in mind, because that's what mind, because that's what mind, because that's what we'll come back to we'll come back to we'll come back to when we talk when we talk when we talk about how this affects about how this affects about how this affects our daily our daily our daily lives. But it's lives. But it's lives. But it's related, right? Your related, right? Your related, right? Your TikTok is overloaded TikTok is overloaded , your YouTube , your YouTube , your YouTube is overloaded, whatever it is overloaded, whatever it is overloaded, whatever it is— is— the algorithm decides for the algorithm decides for the algorithm decides for you. This is a way for us to you. This is a way for us to you. This is a way for us to start responding.

start responding. start responding. The field that The field that The field that tries tries tries to understand how the to understand how the to understand how the internal internal internal mechanisms of large mechanisms of large mechanisms of large language models work language models work language models work is called is called is called interpretability. interpretability. interpretability. The developers are creating a The developers are creating a The developers are creating a system that learns system that learns system that learns during training; during training; during training; they haven't written a they haven't written a they haven't written a separate rule for separate rule for separate rule for each judgment each judgment each judgment it makes. That's why it makes. That's why it makes. That's why we say that models we say that models we say that models are grown, not are grown, not are grown, not made. Therefore, made. Therefore, made. Therefore, knowing knowing knowing the calculations performed or the calculations performed or the calculations performed or tracking tracking tracking the agent's actions does not the agent's actions does not the agent's actions does not automatically explain why automatically explain why automatically explain why the model did what it did the model did what it did the model did what it did . You may be . You may be . You may be asking what a asking what a asking what a neuron is. A neuron is not in neuron is. A neuron is not in neuron is. A neuron is not in your brain, in this your brain, in this your brain, in this context it is context it is context it is actually a small actually a small actually a small mathematical unit mathematical unit mathematical unit inside an AI model. inside an AI model. inside an AI model. It takes numbers, It takes numbers, It takes numbers, combines them combines them combines them using values using values using values learned during learned during learned during training, and passes the training, and passes the training, and passes the result to other result to other result to other mathematical mathematical mathematical units. Some units. Some units. Some neurons neurons neurons respond much more strongly to respond much more strongly to respond much more strongly to recognizable patterns, recognizable patterns, recognizable patterns, such as a certain shape in an such as a certain shape in an such as a certain shape in an image. Many image. Many image. Many neurons can neurons can neurons can work together, work together, work together, creating the overall creating the overall creating the overall behavior of the model. behavior of the model. behavior of the model. A neuron is not a whole A neuron is not a whole A neuron is not a whole thought or a biological thought or a biological thought or a biological brain cell. So, I brain cell. So, I brain cell. So, I just want us to be just want us to be just want us to be on the same level. on the same level. on the same level. The problem was The problem was The problem was that the individual that the individual that the individual components of the model components of the model components of the model often looked like they often looked like they often looked like they were performing a very were performing a very were performing a very confusing mix of confusing mix of confusing mix of tasks. Researchers tasks. Researchers tasks. Researchers have developed methods have developed methods have developed methods for finding meaningful for finding meaningful for finding meaningful patterns among patterns among patterns among combinations of neurons combinations of neurons combinations of neurons in models, and these methods in models, and these methods in models, and these methods have yielded good have yielded good have yielded good results, but a results, but a results, but a fundamental fundamental fundamental understanding of an entire understanding of an entire understanding of an entire powerful model powerful model powerful model remains a very remains a very remains a very difficult task.

difficult task. difficult task. So to see what So to see what So to see what we're looking at here, we're looking at here, we're looking at here, let's look at a let's look at a let's look at a famous neuron in the famous neuron in the famous neuron in the visual model visual model visual model that was discussed in the that was discussed in the that was discussed in the article that responds article that responds article that responds to both cats and to both cats and to both cats and cars. What cars. What cars. What the authors of the authors of the authors of this article found out is this article found out is this article found out is that the models have been that the models have been that the models have been interpretable interpretable interpretable all along. We all along. We all along. We simply didn't have simply didn't have simply didn't have the alphabet and the skills to the alphabet and the skills to the alphabet and the skills to read it. And read it. And read it. And so, the example I so, the example I so, the example I gave about the cat and the gave about the cat and the gave about the cat and the car, what was the car, what was the car, what was the main breakthrough? The main breakthrough? The main breakthrough? The fact that we humans, when fact that we humans, when fact that we humans, when we see the concept of a we see the concept of a we see the concept of a cat and the concept of a cat and the concept of a cat and the concept of a car, think, "Wow, car, think, "Wow, car, think, "Wow, they're completely different. they're completely different. they're completely different. They probably They probably They probably shouldn't be on the shouldn't be on the shouldn't be on the same neuron." The same neuron." The same neuron." The fact that they are on the fact that they are on the fact that they are on the same neuron same neuron same neuron means that the model does not means that the model does not means that the model does not have a logical have a logical have a logical representation of the data representation of the data representation of the data and is therefore difficult and is therefore difficult and is therefore difficult to interpret, which is to interpret, which is to interpret, which is bad for AI safety bad for AI safety . But the breakthrough is that . But the breakthrough is that . But the breakthrough is that if you understand if you understand if you understand that the basic that the basic that the basic organizing organizing organizing concept of the model for concept of the model for concept of the model for this neuron is this neuron is this neuron is not about cars or not about cars or not about cars or cats, but about the V-shaped cats, but about the V-shaped cats, but about the V-shaped posture of those images in the posture of those images in the posture of those images in the model, then everything falls model, then everything falls model, then everything falls into place. Then it's into place. Then it's into place. Then it's not confusing. Then you not confusing. Then you not confusing. Then you understand what understand what understand what the model is thinking when it places the model is thinking when it places the model is thinking when it places this on the neuron, and you this on the neuron, and you this on the neuron, and you realize later as you realize later as you realize later as you look at more and look at more and look at more and more examples, which more examples, which more examples, which the article provides in the article provides in the article provides in exhaustive numbers.

exhaustive numbers. exhaustive numbers. This is not a manipulation of This is not a manipulation of This is not a manipulation of facts. This means that facts. This means that facts. This means that the model has a logical the model has a logical the model has a logical way of arranging way of arranging way of arranging the data. And if the model the data. And if the model the data. And if the model has a logical way has a logical way has a logical way to structure the data in to structure the data in to structure the data in neurons, we can neurons, we can neurons, we can understand that logic. understand that logic. understand that logic. We can read We can read We can read this alphabet. This is the this alphabet. This is the this alphabet. This is the Rosetta Stone of how Rosetta Stone of how Rosetta Stone of how models models models read and think, and read and think, and read and think, and it allows us it allows us it allows us to provide to provide to provide security. Now you security. Now you security. Now you might be thinking, “ might be thinking, “ Hey, we don’t all Hey, we don’t all Hey, we don’t all use use use image models. We image models. We image models. We use language use language use language models." Don't models." Don't models." Don't worry, there are worry, there are worry, there are developments regarding Gwen developments regarding Gwen developments regarding Gwen 332B, the language model. And 332B, the language model. And 332B, the language model. And Camarada attributes Camarada attributes Camarada attributes approximately 90% of the poles of approximately 90% of the poles of approximately 90% of the poles of neurons, the positive neurons, the positive neurons, the positive or negative side of or negative side of or negative side of activations, to a specific activations, to a specific activations, to a specific category with a function. In category with a function. In category with a function. In other words, Comma other words, Comma other words, Comma is able to understand is able to understand is able to understand neurons inside LLM neurons inside LLM neurons inside LLM as well. This is as well. This is as well. This is scalable. The scalable. The comprehensible component comprehensible component gives researchers a gives researchers a gives researchers a starting point in the starting point in the starting point in the model. The next model. The next model. The next question is question is question is whether this whether this whether this understanding helps them understanding helps them understanding helps them predict and predict and predict and change behavior. change behavior. change behavior. And this is where the And this is where the And this is where the security argument gets security argument gets security argument gets really interesting. So really interesting. So , my first prediction is that , my first prediction is that , my first prediction is that if we can if we can if we can make this understanding make this understanding make this understanding robust and robust and robust and scale it scale it scale it with models, with models, with models, the chances of serious the chances of serious the chances of serious inconsistency will be inconsistency will be inconsistency will be greatly reduced. By greatly reduced. By greatly reduced. By this I mean AI this I mean AI this I mean AI that pursues a goal that pursues a goal that pursues a goal that deviates from what that deviates from what humans actually want, sometimes humans actually want, sometimes producing results producing results producing results that appear that appear that appear successful.

successful. successful. Remember the Remember the Remember the coding agent who says he coding agent who says he coding agent who says he fixed your fixed your fixed your mistake? One mistake? One mistake? One way this could way this could way this could happen is he changes happen is he changes happen is he changes the tests instead of the tests instead of the tests instead of fixing the fixing the software, and I did software, and I did n't make that up. This n't make that up. This n't make that up. This actually actually actually happens often. happens often. happens often. Now the test passes, Now the test passes, Now the test passes, but the problem but the problem but the problem remains, remains, remains, right? Researchers right? Researchers right? Researchers call this call this call this behavior behavior behavior reward-smashing, reward-smashing, reward-smashing, optimizing the optimizing the optimizing the success rate success rate success rate in a way that in a way that in a way that defeats the purpose of the defeats the purpose of the defeats the purpose of the task. So, the task. So, the coding example is coding example is an illustration, but not the an illustration, but not the an illustration, but not the only one. There are many only one. There are many only one. There are many examples similar examples similar examples similar to reward hacking in to reward hacking in to reward hacking in other areas. In a other areas. In a other areas. In a separate study, separate study, separate study, Goodfellow provided a real-world Goodfellow provided a real-world Goodfellow provided a real-world example of a example of a example of a purchase evaluation criterion. purchase evaluation criterion. purchase evaluation criterion. The agent recommended a The agent recommended a The agent recommended a warning warning warning signal as a replacement for signal as a replacement for signal as a replacement for a car alarm. a car alarm. a car alarm. His internal His internal His internal monitor detected an monitor detected an monitor detected an unwanted substitution unwanted substitution unwanted substitution when an observer when an observer when an observer reviewing reviewing reviewing the transcript the transcript the transcript missed it. This was a missed it. This was a missed it. This was a mistake. We want to mistake. We want to mistake. We want to be able to be able to be able to fix things like this. fix things like this. fix things like this. And it's very important, when And it's very important, when And it's very important, when you discover something like this you discover something like this you discover something like this during the during the during the learning process, to learning process, to respond appropriately. In respond appropriately. In the study of the study of the study of car car car signals and shopping, signals and shopping, signals and shopping, detectors detectors detectors using using using internal cues internal cues internal cues detected some detected some detected some reward breaks that reward breaks that observers observers reading the transcript missed, as reading the transcript missed, as reading the transcript missed, as I described.

I described. I described. The researchers also The researchers also The researchers also tested tested tested by amplifying the by amplifying the by amplifying the corresponding corresponding corresponding internal signal internal signal , which increased , which increased , which increased the use of the the use of the the use of the " " path of least path of least path of least resistance" they had established. So, what resistance" they had established. So, what resistance" they had established. So, what conclusion can we conclusion can we conclusion can we draw from this? draw from this? draw from this? Part of the problem Part of the problem Part of the problem was that was that was that the way we the way we the way we intervene in the training of the intervene in the training of the intervene in the training of the model matters a lot model matters a lot model matters a lot , and , and , and until now it has been difficult until now it has been difficult until now it has been difficult to predict and to predict and to predict and control. This doesn't control. This doesn't control. This doesn't mean that every mean that every mean that every attempt by the model attempt by the model attempt by the model to cheat will disappear, to cheat will disappear, to cheat will disappear, but we begin to but we begin to but we begin to get a get a get a plausible plausible plausible explanation for why explanation for why explanation for why the model performed a certain the model performed a certain the model performed a certain action, and we begin action, and we begin action, and we begin to understand how we to understand how we to understand how we can intervene in the can intervene in the can intervene in the learning process to learning process to learning process to correct such correct such correct such behavior. So, for a behavior. So, for a behavior. So, for a future code writing agent, a future code writing agent, a useful monitor useful monitor could create could create could create an opportunity an opportunity an opportunity to intervene before a to intervene before a to intervene before a bad decision bad decision bad decision leads to an leads to an leads to an unwanted unwanted unwanted merge request (PR), right? merge request (PR), right? merge request (PR), right? Or the manager could Or the manager could Or the manager could receive receive receive a warning, a warning, a warning, pause the pause the pause the task, and task, and task, and check what check what check what happened. Such a system happened. Such a system happened. Such a system must detect must detect must detect enough real enough real enough real problems without problems without problems without constantly interrupting constantly interrupting constantly interrupting legitimate work, and legitimate work, and legitimate work, and this requires a this requires a this requires a considerable considerable considerable amount of engineering amount of engineering amount of engineering work. But I think work. But I think work. But I think what you should take away what you should take away what you should take away from this is that we from this is that we from this is that we have the opportunity have the opportunity have the opportunity to prevent the acquisition of to prevent the acquisition of to prevent the acquisition of unwanted habits even unwanted habits even unwanted habits even at the learning stage. And at the learning stage. And at the learning stage. And this can happen this can happen this can happen during training, as I have during training, as I have during training, as I have already described. Also in already described. Also in already described. Also in the future this could the future this could the future this could happen happen happen directly in the directly in the directly in the workplace if workplace if workplace if we can we can we can modify the models modify the models modify the models while working, which is a while working, which is a while working, which is a separate task. I separate task. I separate task. I expect this to expect this to expect this to happen by 2027.

happen by 2027. happen by 2027. Let's look at Let's look at Let's look at another example. We another example. We another example. We all know about hacks all know about hacks all know about hacks like "my grandmother is like "my grandmother is like "my grandmother is dying" to bypass dying" to bypass dying" to bypass the fuses in the model. The the fuses in the model. The the fuses in the model. The learning process learning process learning process can be improved can be improved can be improved to detect some of to detect some of to detect some of these dangerous these dangerous these dangerous workarounds. I workarounds. I workarounds. I mean things mean things mean things related to related to related to biological weapons— biological weapons— things that we definitely don't things that we definitely don't things that we definitely don't want to put into want to put into want to put into people's hands, and we don't people's hands, and we don't people's hands, and we don't want the protective want the protective want the protective barriers to be broken. In barriers to be broken. In barriers to be broken. In this case, this case, this case, the ability to understand the ability to understand the ability to understand what is happening what is happening what is happening helps us helps us helps us build a safer build a safer build a safer world. Now let's world. Now let's world. Now let's go back to where go back to where go back to where we started. We we started. We we started. We want to find out want to find out want to find out why AI why AI why AI makes makes makes mistakes, and we want to mistakes, and we want to mistakes, and we want to fix what fix what fix what caused the caused the caused the problem. To do problem. To do problem. To do this effectively, this effectively, this effectively, the fix must the fix must the fix must work for more than just the work for more than just the work for more than just the original example original example that identified that identified that identified the problem. It should the problem. It should the problem. It should be generalized. be generalized. be generalized. So if the So if the So if the encoding agent stops encoding agent stops encoding agent stops changing tasks changing tasks changing tasks but finds another but finds another but finds another workaround, we haven't workaround, we haven't really solved the problem. We need to really solved the problem. We need to see how it see how it see how it performs the planned performs the planned performs the planned work in a way work in a way work in a way that remains that remains that remains valid as the challenges valid as the challenges valid as the challenges change. And that's the change. And that's the change. And that's the part I want part I want us to start us to start us to start paying attention to. If paying attention to. If paying attention to. If you want to know where you want to know where you want to know where we're headed next we're headed next we're headed next after such a big after such a big after such a big paper, it's finding paper, it's finding paper, it's finding ways to change our ways to change our model training protocols so that we can model training protocols so that we can effectively effectively effectively generalize our generalize our generalize our new understanding of how new understanding of how new understanding of how neurons work. My neurons work. My neurons work. My second prediction second prediction second prediction follows from this.

follows from this. follows from this. Successful Successful Successful interpretability interpretability interpretability should give us much should give us much should give us much more confidence that more confidence that more confidence that we can we can we can safely build safely build safely build larger and more capable larger and more capable larger and more capable models. And the difference models. And the difference models. And the difference is that the is that the is that the security tools security tools security tools we have now we have now we have now have a chance to have a chance to have a chance to continue to continue to continue to work as work as their their their responsibilities grow. responsibilities grow. responsibilities grow. Think about Think about Think about moving from an agent moving from an agent moving from an agent that fixes, that fixes, that fixes, say, a single bug say, a single bug , to an agent that , to an agent that , to an agent that maintains an entire maintains an entire software repository. He software repository. He will make many will make many will make many more decisions. It more decisions. It more decisions. It will work much will work much will work much longer. He longer. He longer. He will encounter will encounter will encounter situations that situations that situations that his his his developers did not foresee. A developers did not foresee. A developers did not foresee. A safety score safety score safety score for one for one for one small task small task small task does not indicate how it does not indicate how it does not indicate how it will behave will behave will behave with a much larger with a much larger with a much larger responsibility. responsibility. responsibility. If researchers If researchers If researchers can study a can study a can study a more powerful model, more powerful model, more powerful model, discover new discover new discover new ways it ways it ways it looks for easy paths, and looks for easy paths, and looks for easy paths, and make corrections make corrections make corrections that work reliably and at that work reliably and at that work reliably and at scale, they have a scale, they have a scale, they have a better basis for better basis for better basis for determining what determining what determining what can be trusted in that can be trusted in that can be trusted in that model. They model. They model. They may also discover may also discover may also discover behavior that behavior that behavior that cannot be reliably cannot be reliably cannot be reliably controlled, but controlled, but controlled, but knowing about it before knowing about it before knowing about it before implementation will give implementation will give implementation will give them a strong reason them a strong reason them a strong reason to analyze, to analyze, to analyze, understand, stop, understand, stop, understand, stop, resolve the problem, and resolve the problem, and resolve the problem, and figure out figure out figure out next steps more quickly. And that's how next steps more quickly. And that's how next steps more quickly. And that's how understanding understanding understanding can support can support can support much more ambitious much more ambitious much more ambitious models, right?

models, right? models, right? Because it helps Because it helps Because it helps us distinguish between us distinguish between us distinguish between capabilities we capabilities we capabilities we can can can use and use and use and those we are not yet those we are not yet those we are not yet ready to entrust to ready to entrust to ready to entrust to artificial intelligence. artificial intelligence. artificial intelligence. So I find that So I find that So I find that incredibly incredibly incredibly encouraging. I know encouraging. I know encouraging. I know that one of the questions that one of the questions that one of the questions I often hear every day I often hear every day I often hear every day outside of the cells is: how outside of the cells is: how outside of the cells is: how can we be can we be can we be safe? How can we safe? How can we safe? How can we be sure that be sure that be sure that models are becoming models are becoming models are becoming safer? This is one safer? This is one safer? This is one way we way we way we can do it. This is can do it. This is can do it. This is one way one way one way we we we can be sure can be sure can be sure that we are starting that we are starting that we are starting to scale to scale to scale security tools security tools security tools in parallel with in parallel with in parallel with scaling the scaling the scaling the capabilities of the models. capabilities of the models. capabilities of the models. My third prediction My third prediction is what I expect is what I expect is what I expect people will start to see in people will start to see in people will start to see in products. This is what products. This is what products. This is what you and I will begin to see. you and I will begin to see. you and I will begin to see. In 2027, I think we will In 2027, I think we will In 2027, I think we will see much more see much more see much more serious attempts to serious attempts to serious attempts to make AI decisions make AI decisions make AI decisions interpretable and interpretable and interpretable and understandable to the understandable to the understandable to the specialist specialist specialist responsible for that responsible for that responsible for that decision. I would first decision. I would first decision. I would first focus on focus on focus on areas with high areas with high compliance requirements, such as compliance requirements, such as law, law, law, healthcare, and finance. healthcare, and finance. healthcare, and finance. These are industries where an These are industries where an These are industries where an inexplicable mistake inexplicable mistake inexplicable mistake can be can be can be extremely costly, extremely costly, extremely costly, and someone must be and someone must be and someone must be held accountable for the held accountable for the held accountable for the decision made. There is a decision made. There is a decision made. There is a very practical very practical very practical reason to pay for a reason to pay for a reason to pay for a tool that tool that tool that helps us helps us helps us investigate what went investigate what went investigate what went wrong and whether the wrong and whether the fix worked. Imagine fix worked. Imagine a lawyer a lawyer a lawyer reviewing an AI- reviewing an AI- reviewing an AI- generated contract analysis generated contract analysis generated contract analysis . The AI ​​marks one . The AI ​​marks one . The AI ​​marks one item as a big item as a big item as a big problem, and in this problem, and in this problem, and in this example, it was example, it was example, it was wrong. He wrong. He wrong. He took a condition that took a condition that took a condition that only applies to only applies to only applies to the appendix as if the appendix as if the appendix as if it applied it applied it applied to the entire agreement.

to the entire agreement. to the entire agreement. The lawyer needs to The lawyer needs to The lawyer needs to identify this error and identify this error and identify this error and determine whether determine whether determine whether it affected the it affected the it affected the AI's conclusion, and then, AI's conclusion, and then, AI's conclusion, and then, ideally, the lawyer should ideally, the lawyer should ideally, the lawyer should provide some feedback, provide some feedback, provide some feedback, and we need to and we need to and we need to understand what we understand what we understand what we can do. can do. can do. Today, you Today, you Today, you can already ask can already ask can already ask a model to explain a model to explain a model to explain an answer, but an answer, but an answer, but the explanation the the explanation the the explanation the model generates is not model generates is not model generates is not reliable. So, from a reliable. So, from a reliable. So, from a future future future tool, I expect tool, I expect tool, I expect evidence of what evidence of what evidence of what influenced the analysis influenced the analysis influenced the analysis of the model, including of the model, including of the model, including which parts of which parts of which parts of the explanation were the explanation were the explanation were tested, which tested, which tested, which remain remain remain untested, and so that untested, and so that untested, and so that you can trace the you can trace the you can trace the path from the activated path from the activated path from the activated neurons to neurons to neurons to understanding the logic of understanding the logic of understanding the logic of the reasoning. The lawyer could the reasoning. The lawyer could the reasoning. The lawyer could then correct then correct then correct the scope of the condition and the scope of the condition and the scope of the condition and check the result check the result . Did the . Did the . Did the relevant conclusion change relevant conclusion change relevant conclusion change for the expected reason? for the expected reason? for the expected reason? Does the system handle Does the system handle Does the system handle the same difference in a the same difference in a the same difference in a differently differently differently worded worded worded contract? Did she contract? Did she contract? Did she simply accept simply accept simply accept the correction and the correction and the correction and issue another confident issue another confident issue another confident response? We can response? We can response? We can determine that. These are determine that. These are determine that. These are completely different completely different completely different results, and we results, and we results, and we will be able to adjust will be able to adjust will be able to adjust and steer and steer and steer the model in the right the model in the right the model in the right direction once we can direction once we can direction once we can implement it in the implement it in the implement it in the product. And this is a new product. And this is a new product. And this is a new interface, this is a new interface, this is a new interface, this is a new product of this type.

product of this type. product of this type. We don't have that yet. We don't have that yet. We don't have that yet. But, in my opinion, that's where we're heading But, in my opinion, that's where we're heading But, in my opinion, that's where we're heading . . . We will spend We will spend We will spend less time less time less time guessing why guessing why guessing why AI is failing, we will be able to AI is failing, we will be able to AI is failing, we will be able to make more specific make more specific make more specific fixes, and fixes, and fixes, and ultimately we will have a ultimately we will have a ultimately we will have a better way to better way to better way to verify quality. This will verify quality. This will verify quality. This will not eliminate the need for not eliminate the need for not eliminate the need for professional professional professional judgment; on the contrary, it will judgment; on the contrary, it will judgment; on the contrary, it will give us the opportunity give us the opportunity give us the opportunity to expand it in to expand it in to expand it in useful ways. useful ways. useful ways. Of course, it Of course, it Of course, it will be a narrow scope at first, will be a narrow scope at first, will be a narrow scope at first, right? Specific right? Specific right? Specific workflows where workflows where workflows where developers can developers can developers can verify the relationship verify the relationship verify the relationship between the explanation and the between the explanation and the between the explanation and the model calculation, model calculation, model calculation, right? Most of right? Most of right? Most of these tools these tools these tools start with start with start with developers. So don't developers. So don't developers. So don't expect a expect a expect a one-size-fits-all button one-size-fits-all button that fully that fully that fully explains every explains every explains every answer in any answer in any answer in any model, but expect model, but expect evidence-based explanations that allow for feedback. I wouldn't be feedback. I wouldn't be surprised at all if I surprised at all if I surprised at all if I saw that. My saw that. My saw that. My fourth prediction fourth prediction fourth prediction brings us back to the brings us back to the brings us back to the recommender recommender recommender systems I systems I systems I worked on before worked on before worked on before ChatGPT. Over time, I expect ChatGPT. Over time, I expect ChatGPT. Over time, I expect that we will all be able to that we will all be able to that we will all be able to review and review and review and adjust what the adjust what the adjust what the algorithm has learned algorithm has learned algorithm has learned about us. We about us. We about us. We talk a lot about how talk a lot about how talk a lot about how the algorithm fuels the algorithm fuels the algorithm fuels us, right? We are us, right? We are us, right? We are hostages to the algorithms hostages to the algorithms hostages to the algorithms on our social on our social on our social networks. And this is an example networks. And this is an example networks. And this is an example I keep I keep I keep coming back to. Imagine me coming back to. Imagine me coming back to. Imagine me telling a streaming telling a streaming telling a streaming service, "Hey, I want service, "Hey, I want service, "Hey, I want less "Real Housewives" in less "Real Housewives" in less "Real Housewives" in my feed." A my feed." A my feed." A future version future version future version should allow should allow should allow me to not only me to not only me to not only understand what the model understand what the model understand what the model learned from my learned from my learned from my feedback, but also feedback, but also feedback, but also fix it. Maybe fix it. Maybe she understood it as she understood it as : "I don't want : "I don't want : "I don't want any reality shows at all."

any reality shows at all." any reality shows at all." And I can say, “No, And I can say, “No, And I can say, “No, no, no, I no, no, I no, no, I like other like other like other reality shows. I just reality shows. I just reality shows. I just don't like "Real Housewives." don't like "Real Housewives." don't like "Real Housewives." Or I can say, “ Or I can say, “ Hey, I actually Hey, I actually Hey, I actually really like really like really like cooking shows. I cooking shows. I cooking shows. I mean only mean only mean only culinary ones, not all the culinary ones, not all the culinary ones, not all the others." Whatever it is, I others." Whatever it is, I others." Whatever it is, I look forward to a world that is look forward to a world that is look forward to a world that is more more more customizable customizable customizable , where we can , where we can , where we can change and change and change and adjust existing adjust existing adjust existing features, and where features, and where features, and where algorithms algorithms algorithms work for work for work for us, rather than us us, rather than us us, rather than us being their being their being their hostages. So there are hostages. So there are hostages. So there are things that we have the things that we have the things that we have the opportunity to opportunity to opportunity to advocate for, advocate for, advocate for, build, and build, and build, and develop that develop that develop that will change the daily will change the daily will change the daily interactions of billions interactions of billions interactions of billions of us with artificial of us with artificial of us with artificial intelligence. People don't intelligence. People don't intelligence. People don't consider a Netflix movie to be consider a Netflix movie to be consider a Netflix movie to be artificial intelligence. artificial intelligence. artificial intelligence. I have to disappoint you, I have to disappoint you, I have to disappoint you, this is AI. It's all AI. And this is AI. It's all AI. And this is AI. It's all AI. And interpretability matters interpretability matters interpretability matters . Just do . Just do . Just do n't get me n't get me n't get me wrong. I'm not wrong. I'm not wrong. I'm not saying everyone wants saying everyone wants saying everyone wants to read the explanation under to read the explanation under to read the explanation under every every every movie poster on Netflix. We movie poster on Netflix. We movie poster on Netflix. We want to watch a movie. want to watch a movie. want to watch a movie. But the ability to But the ability to But the ability to understand and express understand and express understand and express subjectivity is very subjectivity is very , very important. And I , very important. And I , very important. And I think that's going to start think that's going to start think that's going to start to seep into the products, to seep into the products, to seep into the products, giving us more giving us more giving us more control, because even control, because even control, because even if you're selfish and if you're selfish and if you're selfish and think that companies think that companies think that companies only care about their own only care about their own only care about their own profits, they still profits, they still profits, they still want you to want you to want you to keep keep keep watching. And if you watching. And if you watching. And if you can express can express can express more preferences more preferences more preferences about what you want about what you want about what you want to see, they will be to see, they will be to see, they will be interested in interested in interested in creating it. It's just creating it. It's just creating it. It's just logical. And now we logical. And now we logical. And now we have the tools to have the tools to have the tools to do it like do it like do it like never before. So never before. So never before. So what does this mean for us?

what does this mean for us? what does this mean for us? First, don't First, don't First, don't just give in to just give in to just give in to pessimism. When someone pessimism. When someone pessimism. When someone tells you that AI is getting tells you that AI is getting tells you that AI is getting more powerful and therefore the more powerful and therefore the more powerful and therefore the future will only get future will only get future will only get worse, ask what is worse, ask what is worse, ask what is happening to our happening to our happening to our ability to understand ability to understand ability to understand AI. Evidence that we AI. Evidence that we AI. Evidence that we can find and can find and can find and correct the causes of correct the causes of correct the causes of undesirable behavior undesirable behavior undesirable behavior is what we should is what we should is what we should celebrate on days like these celebrate on days like these . . . There is good news. And that's why I There is good news. And that's why I There is good news. And that's why I continue to continue to continue to emphasize emphasize emphasize technical technical technical literacy. Such literacy. Such literacy. Such development can development can development can significantly improve significantly improve significantly improve your outlook on your outlook on your outlook on the future, but sometimes the future, but sometimes the future, but sometimes it comes in the it comes in the it comes in the form of a 60-70 form of a 60-70 minute read minute read minute read full of terms full of terms full of terms that most of us that most of us that most of us don't understand. So if don't understand. So if don't understand. So if you're stuck on an you're stuck on an you're stuck on an incomprehensible language, I do incomprehensible language, I do incomprehensible language, I do n't want you to n't want you to n't want you to stop. And I'll give stop. And I'll give stop. And I'll give you a life hack that I you a life hack that I you a life hack that I usually recommend: usually recommend: usually recommend: use AI use AI use AI to help you to help you to help you learn it. Give AI an learn it. Give AI an learn it. Give AI an article. Ask for article. Ask for article. Ask for an explanation that a an explanation that a an explanation that a 12-year- 12-year- 12-year- old could understand. Ask old could understand. Ask old could understand. Ask what statements what statements the experiments confirmed. the experiments confirmed. Ask where Ask where Ask where the hypotheses are the hypotheses are . Go back to . Go back to . Go back to the original source and the original source and the original source and check your answers. check your answers. check your answers. It turns out that you can It turns out that you can learn a lot if you are learn a lot if you are willing to ask AI willing to ask AI willing to ask AI to teach you in such to teach you in such to teach you in such situations. And this situations. And this situations. And this helps us helps us helps us gain technical gain technical gain technical literacy. Ultimately, literacy. Ultimately, literacy. Ultimately, this is very, very good this is very, very good this is very, very good news. We have news. We have news. We have reasons to expect more reasons to expect more reasons to expect more powerful AI. I don't powerful AI. I don't powerful AI. I don't see see see scaling scaling scaling stopping stopping stopping anytime soon. And anytime soon. And anytime soon. And now we have now we have now we have encouraging reason encouraging reason encouraging reason to expect much to expect much to expect much better tools better tools better tools to understand and to understand and to understand and fix this. This fix this. This fix this. This research has yet to be research has yet to be research has yet to be scaled up into scaled up into scaled up into practice. We need to practice. We need to practice. We need to implement this into our implement this into our implement this into our products. There is a products. There is a products. There is a huge opportunity huge opportunity huge opportunity to get this right, to get this right, to get this right, but fundamentally, but fundamentally, but fundamentally, this kind of work changes the this kind of work changes the this kind of work changes the entire vector of entire vector of entire vector of what we can what we can what we can expect from AI and expect from AI and expect from AI and security in the security in the security in the future. This is a

future. This is a future. This is a good day to good day to good day to think about AI in think about AI in think about AI in 2027 and not be afraid.