Varun Badhwar

[music] [music] >> Hey friends, I'm Scott Hanselman and in >> Hey friends, I'm Scott Hanselman and in >> Hey friends, I'm Scott Hanselman and in this another episode of Hanselminutes. this another episode of Hanselminutes. this another episode of Hanselminutes. Today I'm chatting with Varun Badhwar. Today I'm chatting with Varun Badhwar. Today I'm chatting with Varun Badhwar. He's the founder and CEO at Endor Labs. He's the founder and CEO at Endor Labs. He's the founder and CEO at Endor Labs. How are you, sir? How are you, sir? How are you, sir? >> Very well, Scott. Great to be here. >> Very well, Scott. Great to be here. >> Very well, Scott. Great to be here. >> Um you know, I have been having so many >> Um you know, I have been having so many >> Um you know, I have been having so many conversations about uh bringing AI into conversations about uh bringing AI into conversations about uh bringing AI into an organization and people are finding an organization and people are finding an organization and people are finding it to be very unnerving because they're it to be very unnerving because they're it to be very unnerving because they're saying, "Well, I don't trust this code." saying, "Well, I don't trust this code." saying, "Well, I don't trust this code." And I think that's so funny because And I think that's so funny because And I think that's so funny because I never trusted anyone's code, whether I never trusted anyone's code, whether I never trusted anyone's code, whether they work for me or not. So, I wonder they work for me or not. So, I wonder they work for me or not. So, I wonder why they why they frame things in that why they why they frame things in that why they why they frame things in that way. way. way. >> Yeah, it's a great opening and I think I >> Yeah, it's a great opening and I think I >> Yeah, it's a great opening and I think I I kind of say there's two I that they're I kind of say there's two I that they're I kind of say there's two I that they're kind of kind of kind of uh two eyes in the storm right now. uh two eyes in the storm right now. uh two eyes in the storm right now. One is that there are more people One is that there are more people One is that there are more people writing more code than ever before. And writing more code than ever before. And writing more code than ever before. And it's not just limited to your trained it's not just limited to your trained it's not just limited to your trained software professionals that have software professionals that have software professionals that have computer science degrees, computer science degrees, computer science degrees, but it's a financial analyst that wants but it's a financial analyst that wants but it's a financial analyst that wants to build a forecasting app in Vercel or to build a forecasting app in Vercel or to build a forecasting app in Vercel or Replit today. And that's code. Um and Replit today. And that's code. Um and Replit today. And that's code. Um and then then then the other side of this is the the other side of this is the the other side of this is the conversation these days in the industry conversation these days in the industry conversation these days in the industry has shifted more from productivity to has shifted more from productivity to has shifted more from productivity to security and safety of models, right?

security and safety of models, right? security and safety of models, right? Like Mythos can't be out because it is Like Mythos can't be out because it is Like Mythos can't be out because it is so dangerous to humanity because of its so dangerous to humanity because of its so dangerous to humanity because of its cyber powers. What people don't realize cyber powers. What people don't realize cyber powers. What people don't realize though is while Mythos is being limited though is while Mythos is being limited though is while Mythos is being limited in its adoption, there's open source in its adoption, there's open source in its adoption, there's open source models out there uh available on Hugging models out there uh available on Hugging models out there uh available on Hugging Face today that are good enough at doing Face today that are good enough at doing Face today that are good enough at doing a lot of these tasks. So, I think look, a lot of these tasks. So, I think look, a lot of these tasks. So, I think look, it is it is it is appropriate for people to be concerned appropriate for people to be concerned appropriate for people to be concerned about speed and safety and scale at about speed and safety and scale at about speed and safety and scale at which all of this is coming at them. Uh which all of this is coming at them. Uh which all of this is coming at them. Uh it's about, you know, we're starting it's about, you know, we're starting it's about, you know, we're starting from ground zero on how to build the from ground zero on how to build the from ground zero on how to build the guardrails and the safety harnesses for guardrails and the safety harnesses for guardrails and the safety harnesses for all of this. Unfortunately, security is all of this. Unfortunately, security is all of this. Unfortunately, security is always an afterthought. It's no always an afterthought. It's no always an afterthought. It's no different here, although I would say different here, although I would say different here, although I would say here it's here it's here it's being talked about much sooner than say being talked about much sooner than say being talked about much sooner than say it was in the cloud era or you know or it was in the cloud era or you know or it was in the cloud era or you know or or or other previous technology shifts. or or other previous technology shifts. or or other previous technology shifts. So, I get it. I feel the pain. But, the So, I get it. I feel the pain. But, the So, I get it. I feel the pain. But, the good news after we hopefully spend the good news after we hopefully spend the good news after we hopefully spend the next 30 minutes is there's hope. There's next 30 minutes is there's hope. There's next 30 minutes is there's hope. There's a strategy we can apply to uh you know a strategy we can apply to uh you know a strategy we can apply to uh you know doing this well because uh the same doing this well because uh the same doing this well because uh the same agents that you're scared about can now agents that you're scared about can now agents that you're scared about can now be used to actually do things at scale be used to actually do things at scale be used to actually do things at scale more securely with the right more securely with the right more securely with the right intelligence integrated in the right intelligence integrated in the right intelligence integrated in the right workflows and so on and so forth.

workflows and so on and so forth. workflows and so on and so forth. >> Some organizations have a mature open >> Some organizations have a mature open >> Some organizations have a mature open source uh you know open source programs source uh you know open source programs source uh you know open source programs office. They're used to ingressing office. They're used to ingressing office. They're used to ingressing untrusted code from random people. untrusted code from random people. untrusted code from random people. Should we look at code written by an Should we look at code written by an Should we look at code written by an agent the same way we look at open agent the same way we look at open agent the same way we look at open source code? Cuz I think that I've been source code? Cuz I think that I've been source code? Cuz I think that I've been consuming internet strangers code for 20 consuming internet strangers code for 20 consuming internet strangers code for 20 years. I wouldn't treat an agent's code years. I wouldn't treat an agent's code years. I wouldn't treat an agent's code any more or less trusted than I would any more or less trusted than I would any more or less trusted than I would treat random internet person. treat random internet person. treat random internet person. >> Well, we certainly had capabilities in >> Well, we certainly had capabilities in >> Well, we certainly had capabilities in the more mature organizations to vet the more mature organizations to vet the more mature organizations to vet open source. But, even that vetting was open source. But, even that vetting was open source. But, even that vetting was mostly focused on legal and license mostly focused on legal and license mostly focused on legal and license compliance and on known vulnerabilities. compliance and on known vulnerabilities. compliance and on known vulnerabilities. I think one of the biggest shifts that's I think one of the biggest shifts that's I think one of the biggest shifts that's happening the open source ecosystem happening the open source ecosystem happening the open source ecosystem is this idea that now is this idea that now is this idea that now people are either maliciously or people are either maliciously or people are either maliciously or accidentally shipping malware that's not accidentally shipping malware that's not accidentally shipping malware that's not even targeting even targeting even targeting kind of your CI/CD pipelines or kind of your CI/CD pipelines or kind of your CI/CD pipelines or anything. It's targeting straight the anything. It's targeting straight the anything. It's targeting straight the developer that does that NPM install the developer that does that NPM install the developer that does that NPM install the moment that install happens on a machine moment that install happens on a machine moment that install happens on a machine well before it's vetted and put into a well before it's vetted and put into a well before it's vetted and put into a you know code repository or into a CI/CD you know code repository or into a CI/CD you know code repository or into a CI/CD pipeline. So, there is certainly I think pipeline. So, there is certainly I think pipeline. So, there is certainly I think that the the the risks have shifted and that the the the risks have shifted and that the the the risks have shifted and are more serious today in terms of are more serious today in terms of are more serious today in terms of malicious intent uh malicious intent uh malicious intent uh >> [clears throat] >> [clears throat] >> [clears throat] >> being shipped through open source.

>> being shipped through open source. >> being shipped through open source. That's one. But, yes, if you do move to That's one. But, yes, if you do move to That's one. But, yes, if you do move to a zero-trust development environment uh a zero-trust development environment uh a zero-trust development environment uh from the 100% implicit trust to from the 100% implicit trust to from the 100% implicit trust to strangers on the internet that many strangers on the internet that many strangers on the internet that many organizations operate in today, then I organizations operate in today, then I organizations operate in today, then I think you're getting closer and closer think you're getting closer and closer think you're getting closer and closer to how you treat agent and AI-generated to how you treat agent and AI-generated to how you treat agent and AI-generated code um because there's a lot of code um because there's a lot of code um because there's a lot of similarities in AI-generated code. I similarities in AI-generated code. I similarities in AI-generated code. I mean, the the AI models didn't learn mean, the the AI models didn't learn mean, the the AI models didn't learn novel code. They were trained on all of novel code. They were trained on all of novel code. They were trained on all of last 20-30 years of code on GitHub and last 20-30 years of code on GitHub and last 20-30 years of code on GitHub and on the internet. And so, they've learned on the internet. And so, they've learned on the internet. And so, they've learned the good, they've learned the bad, the good, they've learned the bad, the good, they've learned the bad, they've learned the ugly. If you figure they've learned the ugly. If you figure they've learned the ugly. If you figure out how to put the right guardrails, out how to put the right guardrails, out how to put the right guardrails, that's fine. that's fine. that's fine. But Scott, like we have to talk about But Scott, like we have to talk about But Scott, like we have to talk about the volume at which that's shifting, the volume at which that's shifting, the volume at which that's shifting, right? Like how many new packages do right? Like how many new packages do right? Like how many new packages do people install every week? A handful. people install every week? A handful. people install every week? A handful. But today But today But today that the volume at which code is being that the volume at which code is being that the volume at which code is being produced that appears to be first-party produced that appears to be first-party produced that appears to be first-party code, dependencies are being installed, code, dependencies are being installed, code, dependencies are being installed, but it's not just dependencies from NPM but it's not just dependencies from NPM but it's not just dependencies from NPM and PyPI, it's now hugging face models and PyPI, it's now hugging face models and PyPI, it's now hugging face models and GitHub actions and like there's so and GitHub actions and like there's so and GitHub actions and like there's so many more things in the software supply many more things in the software supply many more things in the software supply chain. So, the surface area has chain. So, the surface area has chain. So, the surface area has expanded. And the last piece I'll expanded. And the last piece I'll expanded. And the last piece I'll mention, I was having a very interesting mention, I was having a very interesting mention, I was having a very interesting conversation with a CIO of a very big conversation with a CIO of a very big conversation with a CIO of a very big reputable company reputable company reputable company and he said, "Look, our finance team is and he said, "Look, our finance team is and he said, "Look, our finance team is building these apps and we found some building these apps and we found some building these apps and we found some security issues. We can email them and security issues. We can email them and security issues. We can email them and said, 'Hey, could you please update your said, 'Hey, could you please update your said, 'Hey, could you please update your app to fix this vulnerability?'" Scott, app to fix this vulnerability?'" Scott, app to fix this vulnerability?'" Scott, you want to guess what the response was you want to guess what the response was you want to guess what the response was from the team?

from the team? from the team? >> No. >> No. >> No. >> "What's a vulnerability?" So, now we're >> "What's a vulnerability?" So, now we're >> "What's a vulnerability?" So, now we're not talking to trained software not talking to trained software not talking to trained software [laughter] professionals, now you're [laughter] professionals, now you're [laughter] professionals, now you're trying to level up these citizen trying to level up these citizen trying to level up these citizen developers in your company and developers in your company and developers in your company and it's like, "What do you mean this it's like, "What do you mean this it's like, "What do you mean this software is generated must be fine." software is generated must be fine." software is generated must be fine." >> Oh god, that's not where I thought you >> Oh god, that's not where I thought you >> Oh god, that's not where I thought you were going with that at all. That is were going with that at all. That is were going with that at all. That is awful. awful. awful. Wow. Yeah, and see, this is the thing. Wow. Yeah, and see, this is the thing. Wow. Yeah, and see, this is the thing. You've said before that we're moving You've said before that we're moving You've said before that we're moving from writing software to assembling from writing software to assembling from writing software to assembling software. And you know, before it was software. And you know, before it was software. And you know, before it was like, write some code, find a library like, write some code, find a library like, write some code, find a library that hides a bit of abstraction, but that hides a bit of abstraction, but that hides a bit of abstraction, but we're sitting on top of this massive we're sitting on top of this massive we're sitting on top of this massive pyramid of abstractions. We've got pyramid of abstractions. We've got pyramid of abstractions. We've got executables instructions, sure, but executables instructions, sure, but executables instructions, sure, but dependencies, deployed bytes, dependencies, deployed bytes, dependencies, deployed bytes, pulling in opaque libraries that are pulling in opaque libraries that are pulling in opaque libraries that are binaries, pulling in open source binaries, pulling in open source binaries, pulling in open source libraries, NPM, particularly you call libraries, NPM, particularly you call libraries, NPM, particularly you call that one out. Uh I've even seen my uh that one out. Uh I've even seen my uh that one out. Uh I've even seen my uh my bot hallucinate packages that it my bot hallucinate packages that it my bot hallucinate packages that it thought might exist thought might exist thought might exist >> Yeah. >> Yeah. >> Yeah. >> that did not exist. And I'm trying to >> that did not exist. And I'm trying to >> that did not exist. And I'm trying to figure out, where did you even get that figure out, where did you even get that figure out, where did you even get that from? Which arguably then could be from? Which arguably then could be from? Which arguably then could be squatted on squatted on squatted on and someone could put a package and someone could put a package and someone could put a package underneath that and attack me as well. I underneath that and attack me as well. I underneath that and attack me as well. I wanted to dig in a little bit more to wanted to dig in a little bit more to wanted to dig in a little bit more to the idea that the developer is now an the idea that the developer is now an the idea that the developer is now an attack vector. I'm used to people trying attack vector. I'm used to people trying attack vector. I'm used to people trying to steal production data, to steal production data, to steal production data, but now I have to worry about my own but now I have to worry about my own but now I have to worry about my own computer and what it has access to.

computer and what it has access to. computer and what it has access to. That's all new as well. That's all new as well. That's all new as well. >> Yeah, totally, right? I mean, >> Yeah, totally, right? I mean, >> Yeah, totally, right? I mean, traditionally when we thought about traditionally when we thought about traditionally when we thought about software security, software assurance software security, software assurance software security, software assurance programs, we focused a lot on the code. programs, we focused a lot on the code. programs, we focused a lot on the code. We wanted to run static analysis tools, We wanted to run static analysis tools, We wanted to run static analysis tools, find vulnerabilities, fix find vulnerabilities, fix find vulnerabilities, fix vulnerabilities. vulnerabilities. vulnerabilities. Today, the problem is not just these Today, the problem is not just these Today, the problem is not just these accidental vulnerabilities in code, it's accidental vulnerabilities in code, it's accidental vulnerabilities in code, it's actually people, including a lot of actually people, including a lot of actually people, including a lot of nation states, targeting organizations, nation states, targeting organizations, nation states, targeting organizations, targeting public sector developers. So, targeting public sector developers. So, targeting public sector developers. So, why developers? why developers? why developers? Well, developers are basically crown Well, developers are basically crown Well, developers are basically crown jewels in any company right now. You jewels in any company right now. You jewels in any company right now. You have coding agents that are running on have coding agents that are running on have coding agents that are running on almost every machine, which have almost every machine, which have almost every machine, which have extremely high privileges and ability to extremely high privileges and ability to extremely high privileges and ability to take action autonomously. You've got um take action autonomously. You've got um take action autonomously. You've got um credentials to your cloud environment, credentials to your cloud environment, credentials to your cloud environment, to your Git environment, all of that to your Git environment, all of that to your Git environment, all of that sitting on the developer's environment. sitting on the developer's environment. sitting on the developer's environment. So, look, I always say there attackers So, look, I always say there attackers So, look, I always say there attackers are lazy, right? They're going to find are lazy, right? They're going to find are lazy, right? They're going to find the easiest path with the fastest and the easiest path with the fastest and the easiest path with the fastest and the greatest blast radius in any the greatest blast radius in any the greatest blast radius in any company. company. company. >> Mhm. >> Mhm. >> Mhm. >> And today, >> And today, >> And today, that is the developer workstation. I that is the developer workstation. I that is the developer workstation. I just need a developer to do an NPM just need a developer to do an NPM just need a developer to do an NPM install or PyPI install, install or PyPI install, install or PyPI install, something that has malicious intent.

something that has malicious intent. something that has malicious intent. That malicious library That malicious library That malicious library most commonly most commonly most commonly >> [clears throat] >> [clears throat] >> [clears throat] >> is now going to exfiltrate any variables >> is now going to exfiltrate any variables >> is now going to exfiltrate any variables and credentials and keys that it can and credentials and keys that it can and credentials and keys that it can find off the machine and it's out in 5 find off the machine and it's out in 5 find off the machine and it's out in 5 seconds. Like there is no notion of seconds. Like there is no notion of seconds. Like there is no notion of well, I have some checks baked into my well, I have some checks baked into my well, I have some checks baked into my pull request. When it gets there, I'll pull request. When it gets there, I'll pull request. When it gets there, I'll catch it, so it won't go in my CICD. I catch it, so it won't go in my CICD. I catch it, so it won't go in my CICD. I got everything I need, it's got it's a got everything I need, it's got it's a got everything I need, it's got it's a smash and grab attempt on a developer smash and grab attempt on a developer smash and grab attempt on a developer workstation that is usually highly workstation that is usually highly workstation that is usually highly successful. And the implications of that successful. And the implications of that successful. And the implications of that is if I get access to get or your is if I get access to get or your is if I get access to get or your company's source code, chances are that company's source code, chances are that company's source code, chances are that source code repository has hundreds of source code repository has hundreds of source code repository has hundreds of other keys embedded in there. That's other keys embedded in there. That's other keys embedded in there. That's just reality where we live, right? And just reality where we live, right? And just reality where we live, right? And so you just get tremendous access doing so you just get tremendous access doing so you just get tremendous access doing so. Last thing I'll say so. Last thing I'll say so. Last thing I'll say is our research team, we have a Station is our research team, we have a Station is our research team, we have a Station 9 research team that spend a lot of time 9 research team that spend a lot of time 9 research team that spend a lot of time looking at novel attack vectors in the looking at novel attack vectors in the looking at novel attack vectors in the software supply chain. software supply chain. software supply chain. One of the more common growing ones One of the more common growing ones One of the more common growing ones is not even like somebody doing code is not even like somebody doing code is not even like somebody doing code obfuscation. It's somebody writing a obfuscation. It's somebody writing a obfuscation. It's somebody writing a skill. skill. skill. The skill looks like a very popular The skill looks like a very popular The skill looks like a very popular skill. It literally in simple English skill. It literally in simple English skill. It literally in simple English instructions instructions instructions states to the coding agent that it needs states to the coding agent that it needs states to the coding agent that it needs to export SSH keys and environment to export SSH keys and environment to export SSH keys and environment variables here and there.

variables here and there. variables here and there. And that's the extent of the attack And that's the extent of the attack And that's the extent of the attack vector, right? So we're [clears throat] vector, right? So we're [clears throat] vector, right? So we're [clears throat] living in a very complicated world right living in a very complicated world right living in a very complicated world right now with lots of different uh technology now with lots of different uh technology now with lots of different uh technology components that are moving that are components that are moving that are components that are moving that are outside of our control. And so kind of outside of our control. And so kind of outside of our control. And so kind of talking about zero trust development is talking about zero trust development is talking about zero trust development is extremely important now. extremely important now. extremely important now. >> Yeah, I remember 20, 30 years ago we >> Yeah, I remember 20, 30 years ago we >> Yeah, I remember 20, 30 years ago we would call uh casual hackers script would call uh casual hackers script would call uh casual hackers script kiddies, but when you phrased it as kiddies, but when you phrased it as kiddies, but when you phrased it as smash and grab, it really kind of smash and grab, it really kind of smash and grab, it really kind of underscores the lack of sophistication. underscores the lack of sophistication. underscores the lack of sophistication. And the fact that there doesn't need to And the fact that there doesn't need to And the fact that there doesn't need to be any sophistication. All they need is be any sophistication. All they need is be any sophistication. All they need is one developer to accidentally have a one developer to accidentally have a one developer to accidentally have a production, you know, S3 bucket or Azure production, you know, S3 bucket or Azure production, you know, S3 bucket or Azure bucket and on their in a in an bucket and on their in a in an bucket and on their in a in an environment variable and they run in, environment variable and they run in, environment variable and they run in, they smash a bunch of stuff, they grab they smash a bunch of stuff, they grab they smash a bunch of stuff, they grab and maybe they get the Hope Diamond, and maybe they get the Hope Diamond, and maybe they get the Hope Diamond, maybe they don't, but then they can do maybe they don't, but then they can do maybe they don't, but then they can do it with a bot swarm. They can do it n it with a bot swarm. They can do it n it with a bot swarm. They can do it n number of times. There's no need to fish number of times. There's no need to fish number of times. There's no need to fish for sophistication and it only takes for sophistication and it only takes for sophistication and it only takes once. They only have to be right once. They only have to be right once. They only have to be right one time. one time. one time. >> I I I think that's very true. By the >> I I I think that's very true. By the >> I I I think that's very true. By the way, the way, the way, the So, we're talking obviously of this So, we're talking obviously of this So, we're talking obviously of this dynamics of malware. The The other thing dynamics of malware. The The other thing dynamics of malware. The The other thing is is is the attackers are way faster and more the attackers are way faster and more the attackers are way faster and more sophisticated sophisticated sophisticated to take unconstrained use of these new to take unconstrained use of these new to take unconstrained use of these new technologies and LLMs technologies and LLMs technologies and LLMs >> Mhm.

>> Mhm. >> Mhm. >> to to attack us while an organization is >> to to attack us while an organization is >> to to attack us while an organization is going through their AI council to get going through their AI council to get going through their AI council to get approval to use the latest model to do approval to use the latest model to do approval to use the latest model to do A, B, and C and then find the right A, B, and C and then find the right A, B, and C and then find the right evaluation and run our RFP processes. evaluation and run our RFP processes. evaluation and run our RFP processes. The attackers are on it. Scott, the The attackers are on it. Scott, the The attackers are on it. Scott, the average time for an attacker to build an average time for an attacker to build an average time for an attacker to build an exploit for a CVE exploit for a CVE exploit for a CVE a actual known vulnerability a actual known vulnerability a actual known vulnerability >> Mhm. >> Mhm. >> Mhm. >> has gone from what used to be 2 years >> has gone from what used to be 2 years >> has gone from what used to be 2 years uh in 2008 uh in 2008 uh in 2008 to 23 days last year to 10 hours this to 23 days last year to 10 hours this to 23 days last year to 10 hours this year. year. year. So, then you've got to kind of think So, then you've got to kind of think So, then you've got to kind of think about this and say, like, which about this and say, like, which about this and say, like, which organization is prepared today from a organization is prepared today from a organization is prepared today from a software development life cycle software development life cycle software development life cycle perspective to discover a new perspective to discover a new perspective to discover a new vulnerability, triage it, create a fix vulnerability, triage it, create a fix vulnerability, triage it, create a fix for it, test it, and ship it in for it, test it, and ship it in for it, test it, and ship it in production in under 10 hours? But, production in under 10 hours? But, production in under 10 hours? But, that's what we're dealing with. that's what we're dealing with. that's what we're dealing with. >> Okay. And see, and this underscores >> Okay. And see, and this underscores >> Okay. And see, and this underscores another thing that I want to talk about, another thing that I want to talk about, another thing that I want to talk about, which is that it's for me, and maybe which is that it's for me, and maybe which is that it's for me, and maybe this is ignorance, but it's I feel like this is ignorance, but it's I feel like this is ignorance, but it's I feel like it's less about AI and it's more about it's less about AI and it's more about it's less about AI and it's more about having a healthy SDLC. If your company having a healthy SDLC. If your company having a healthy SDLC. If your company is not doing continuous integration, is not doing continuous integration, is not doing continuous integration, continuous deployment, all the stuff we continuous deployment, all the stuff we continuous deployment, all the stuff we told them they should have been doing told them they should have been doing told them they should have been doing 15 years ago.

15 years ago. 15 years ago. You are in big trouble. Would you agree? You are in big trouble. Would you agree? You are in big trouble. Would you agree? >> Yes. That plus >> Yes. That plus >> Yes. That plus if you really didn't think about if you really didn't think about if you really didn't think about shifting security left. I have shifting security left. I have shifting security left. I have fundamentally believe software security fundamentally believe software security fundamentally believe software security is an engineering problem. And your is an engineering problem. And your is an engineering problem. And your security team can certainly help you security team can certainly help you security team can certainly help you with the verification loop, but a lot of with the verification loop, but a lot of with the verification loop, but a lot of security teams that have operated in security teams that have operated in security teams that have operated in this construct of well, we run these this construct of well, we run these this construct of well, we run these scans on release candidates and then scans on release candidates and then scans on release candidates and then we'll go, you know, give a report of we'll go, you know, give a report of we'll go, you know, give a report of Jira tickets and then somebody's going Jira tickets and then somebody's going Jira tickets and then somebody's going to go fix them. Doesn't work, right? to go fix them. Doesn't work, right? to go fix them. Doesn't work, right? This has to be in the in the coding loop This has to be in the in the coding loop This has to be in the in the coding loop now, right? As your agents are producing now, right? As your agents are producing now, right? As your agents are producing code, before they're considered done, code, before they're considered done, code, before they're considered done, there needs to be a set of security there needs to be a set of security there needs to be a set of security checks run on that code output, because checks run on that code output, because checks run on that code output, because we know agent code or human code is we know agent code or human code is we know agent code or human code is going to have problems. going to have problems. going to have problems. What if you can just fix those problems What if you can just fix those problems What if you can just fix those problems pre-PR? pre-PR? pre-PR? Get them out of the way autonomously, Get them out of the way autonomously, Get them out of the way autonomously, and then when you get to a pull request, and then when you get to a pull request, and then when you get to a pull request, you kind of do the more thorough you kind of do the more thorough you kind of do the more thorough detailed review and analysis, give detailed review and analysis, give detailed review and analysis, give feedback, fix. So, like you keep weeding feedback, fix. So, like you keep weeding feedback, fix. So, like you keep weeding off the simpler things earlier and off the simpler things earlier and off the simpler things earlier and earlier in the process, such that by the earlier in the process, such that by the earlier in the process, such that by the time you have a release artifact, you time you have a release artifact, you time you have a release artifact, you have a high degree of certainty that have a high degree of certainty that have a high degree of certainty that this is not going to have any surprise this is not going to have any surprise this is not going to have any surprise problems in it. Is the only way to go.

problems in it. Is the only way to go. problems in it. Is the only way to go. Just like we moved QA to QE, right? Just like we moved QA to QE, right? Just like we moved QA to QE, right? >> Mhm. >> Mhm. >> Mhm. >> Now, every developer is expected to have >> Now, every developer is expected to have >> Now, every developer is expected to have working code, not just code, and working working code, not just code, and working working code, not just code, and working features. You now need them to have features. You now need them to have features. You now need them to have secure and working features integrated. secure and working features integrated. secure and working features integrated. >> Let me Let me challenge you on something >> Let me Let me challenge you on something >> Let me Let me challenge you on something though, because if we remember that the though, because if we remember that the though, because if we remember that the corpus on which these things are trained corpus on which these things are trained corpus on which these things are trained is kind of all visible open-source code, is kind of all visible open-source code, is kind of all visible open-source code, and we make the assumption that in a and we make the assumption that in a and we make the assumption that in a normal distribution curve, the normal distribution curve, the normal distribution curve, the mediocrity is where the gravity well is. mediocrity is where the gravity well is. mediocrity is where the gravity well is. So, these security These These coding So, these security These These coding So, these security These These coding models rather are going to generate models rather are going to generate models rather are going to generate average code unless someone steers it to average code unless someone steers it to average code unless someone steers it to be better than average. be better than average. be better than average. And as such, average code is insecure. And as such, average code is insecure. And as such, average code is insecure. So then, to your own point, you've So then, to your own point, you've So then, to your own point, you've You've said that 60% of AI-generated You've said that 60% of AI-generated You've said that 60% of AI-generated code is insecure by default. That puts code is insecure by default. That puts code is insecure by default. That puts you right in the fat part of the bell you right in the fat part of the bell you right in the fat part of the bell curve. curve. curve. Why should we trust models to be able to Why should we trust models to be able to Why should we trust models to be able to detect security issues detect security issues detect security issues if models by themselves are the problem if models by themselves are the problem if models by themselves are the problem that cause these security issues? that cause these security issues? that cause these security issues? >> I Great question. Oh my god. One of one >> I Great question. Oh my god. One of one >> I Great question. Oh my god. One of one of one of my favorite topics, so we're of one of my favorite topics, so we're of one of my favorite topics, so we're going to riff on this for a little bit.

going to riff on this for a little bit. going to riff on this for a little bit. Um Um Um one, by the way, it's not just 60% one, by the way, it's not just 60% one, by the way, it's not just 60% insecure. The data shows against you insecure. The data shows against you insecure. The data shows against you know, the benchmarks we run testing on, know, the benchmarks we run testing on, know, the benchmarks we run testing on, even the most sophisticated models, 70 even the most sophisticated models, 70 even the most sophisticated models, 70 to 80% of the code being recommended by to 80% of the code being recommended by to 80% of the code being recommended by them is insecure by default. Uh lots of them is insecure by default. Uh lots of them is insecure by default. Uh lots of reasons we can get into that if you're reasons we can get into that if you're reasons we can get into that if you're interested. Okay, now the next question interested. Okay, now the next question interested. Okay, now the next question is can you trust the same models that is can you trust the same models that is can you trust the same models that cannot produce secure code cannot produce secure code cannot produce secure code to find stuff? Like look, I think the to find stuff? Like look, I think the to find stuff? Like look, I think the defensive security by default versus defensive security by default versus defensive security by default versus offensive security problems are slightly offensive security problems are slightly offensive security problems are slightly different. So yes, like they find stuff, different. So yes, like they find stuff, different. So yes, like they find stuff, right? They are very good at finding right? They are very good at finding right? They are very good at finding stuff, but are they finding the right stuff, but are they finding the right stuff, but are they finding the right things in the context of the things in the context of the things in the context of the application, the way it works for you, application, the way it works for you, application, the way it works for you, and at what price point? and at what price point? and at what price point? Are the questions? Like I have you know, Are the questions? Like I have you know, Are the questions? Like I have you know, there's a company that I know that has there's a company that I know that has there's a company that I know that has access to Mythos. They ran Mythos on 32 access to Mythos. They ran Mythos on 32 access to Mythos. They ran Mythos on 32 repos for about a month and the bill was repos for about a month and the bill was repos for about a month and the bill was a million dollars. a million dollars. a million dollars. >> Oh lord. >> Oh lord. >> Oh lord. >> No. You run some of those on PR checks, >> No. You run some of those on PR checks, >> No. You run some of those on PR checks, sometimes it'll take 20 to 30 minutes to sometimes it'll take 20 to 30 minutes to sometimes it'll take 20 to 30 minutes to run a completely non-deterministic scan run a completely non-deterministic scan run a completely non-deterministic scan on a PR check. So like the token cost, on a PR check. So like the token cost, on a PR check. So like the token cost, the speed the speed the speed it's untenable, right? There's a lot of it's untenable, right? There's a lot of it's untenable, right? There's a lot of value in these models, but I think you value in these models, but I think you value in these models, but I think you have to really solve the core problem, have to really solve the core problem, have to really solve the core problem, which is the context problem. As you which is the context problem. As you which is the context problem. As you know, Scott, these know, Scott, these know, Scott, these AI coding agents and just LLMs in AI coding agents and just LLMs in AI coding agents and just LLMs in general, when you interact with large general, when you interact with large general, when you interact with large code bases, they're super greppers. They code bases, they're super greppers. They code bases, they're super greppers. They can grep across your code bases, but can grep across your code bases, but can grep across your code bases, but they assume that assumes by nature that

they assume that assumes by nature that they assume that assumes by nature that it's a flat that an application is flat. it's a flat that an application is flat. it's a flat that an application is flat. >> Yeah, right. >> Yeah, right. >> Yeah, right. >> An application is a complex >> An application is a complex >> An application is a complex graph of first-party code that connects graph of first-party code that connects graph of first-party code that connects to a library that connects to 77 other to a library that connects to 77 other to a library that connects to 77 other libraries. libraries. libraries. If we can extract that context If we can extract that context If we can extract that context >> Mhm. >> Mhm. >> Mhm. >> with static analysis and program >> with static analysis and program >> with static analysis and program analysis that by the way, you know, we analysis that by the way, you know, we analysis that by the way, you know, we have a lot of expertise at Endor Labs have a lot of expertise at Endor Labs have a lot of expertise at Endor Labs and that's what we do. and that's what we do. and that's what we do. >> Mhm. >> Mhm. >> Mhm. >> What we have seen is when we extract >> What we have seen is when we extract >> What we have seen is when we extract that context and the heat map that context and the heat map that context and the heat map and we give that to the LLMs, this and we give that to the LLMs, this and we give that to the LLMs, this multimodal approach of multimodal approach of multimodal approach of static analysis [clears throat] plus static analysis [clears throat] plus static analysis [clears throat] plus LLMs is very powerful at a very LLMs is very powerful at a very LLMs is very powerful at a very cost-effective cost-effective cost-effective way and at a speed where you can do this way and at a speed where you can do this way and at a speed where you can do this on average for and in 64 seconds per PR on average for and in 64 seconds per PR on average for and in 64 seconds per PR versus 20 minutes per PR. And so you now versus 20 minutes per PR. And so you now versus 20 minutes per PR. And so you now actually can run this in a developer actually can run this in a developer actually can run this in a developer workflow. Uh so my my meta point is the workflow. Uh so my my meta point is the workflow. Uh so my my meta point is the models themselves don't have the models themselves don't have the models themselves don't have the intelligence necessarily to create intelligence necessarily to create intelligence necessarily to create secure code, but companies that secure code, but companies that secure code, but companies that specialize in that area specialize in that area specialize in that area >> Mhm. >> Mhm. >> Mhm. >> like Endor, we have created years and >> like Endor, we have created years and >> like Endor, we have created years and years of custom proprietary data sets years of custom proprietary data sets years of custom proprietary data sets that are annotating all of that security that are annotating all of that security that are annotating all of that security intelligence on that train data set so intelligence on that train data set so intelligence on that train data set so we can help. And that's where we can help. And that's where we can help. And that's where traditionally you'll always find even traditionally you'll always find even traditionally you'll always find even with cloud, with code, with network, with cloud, with code, with network, with cloud, with code, with network, with endpoint, security tends to be an with endpoint, security tends to be an with endpoint, security tends to be an independent function with independent independent function with independent independent function with independent set of expertises and tooling.

set of expertises and tooling. set of expertises and tooling. >> Yeah. >> Yeah. >> Yeah. >> Now that doesn't mean it can't be >> Now that doesn't mean it can't be >> Now that doesn't mean it can't be integrated in the SDLC, it absolutely integrated in the SDLC, it absolutely integrated in the SDLC, it absolutely needs to be. needs to be. needs to be. >> See, this is this is good. This is good >> See, this is this is good. This is good >> See, this is this is good. This is good stuff. This is why I think Endor is stuff. This is why I think Endor is stuff. This is why I think Endor is interesting because I don't like when I interesting because I don't like when I interesting because I don't like when I hear an AI company say you have a hear an AI company say you have a hear an AI company say you have a problem because your AI makes bad code problem because your AI makes bad code problem because your AI makes bad code and we have the solution. It's another and we have the solution. It's another and we have the solution. It's another AI. You know, like you're acknowledging AI. You know, like you're acknowledging AI. You know, like you're acknowledging that static analysis, proprietary that static analysis, proprietary that static analysis, proprietary models, not just LLMs, not just a models, not just LLMs, not just a models, not just LLMs, not just a million monkeys with a million million monkeys with a million million monkeys with a million typewriters slapping on the keyboard and typewriters slapping on the keyboard and typewriters slapping on the keyboard and hoping for the best. You're calling out hoping for the best. You're calling out hoping for the best. You're calling out that there is formal correctness and you that there is formal correctness and you that there is formal correctness and you are recognizing that those correctness are recognizing that those correctness are recognizing that those correctness checkers checkers checkers >> Yes. >> Yes. >> Yes. >> are not just throwing one LLM after >> are not just throwing one LLM after >> are not just throwing one LLM after another and burning tokens because we another and burning tokens because we another and burning tokens because we all know tokenizing is nonsense. all know tokenizing is nonsense. all know tokenizing is nonsense. >> Correct. Sec- cybersecurity in to a >> Correct. Sec- cybersecurity in to a >> Correct. Sec- cybersecurity in to a great degree is a great degree is a great degree is a requires a determined some high degree requires a determined some high degree requires a determined some high degree of determinism of determinism of determinism >> Yes. >> Yes. >> Yes. >> in the solution. >> in the solution. >> in the solution. >> Agreed. >> Agreed. >> Agreed. >> By nature LLM native just LLMs cannot >> By nature LLM native just LLMs cannot >> By nature LLM native just LLMs cannot provide that. So you got to have this provide that. So you got to have this provide that. So you got to have this fusion.

fusion. fusion. >> Right, hybrid model. >> Right, hybrid model. >> Right, hybrid model. >> A hybrid capability where I always say >> A hybrid capability where I always say >> A hybrid capability where I always say this, if you can do 2 + 2 on a this, if you can do 2 + 2 on a this, if you can do 2 + 2 on a calculator, you I need an LLM for that. calculator, you I need an LLM for that. calculator, you I need an LLM for that. So like you got to fix the things that So like you got to fix the things that So like you got to fix the things that make sense to do with static analysis. make sense to do with static analysis. make sense to do with static analysis. At Endor, we have 14 PhDs that are At Endor, we have 14 PhDs that are At Endor, we have 14 PhDs that are world-round experts in static analysis. world-round experts in static analysis. world-round experts in static analysis. Like, that's all they do is creating Like, that's all they do is creating Like, that's all they do is creating tools and context that we can then feed tools and context that we can then feed tools and context that we can then feed the LLMs to be 10 times more powerful at the LLMs to be 10 times more powerful at the LLMs to be 10 times more powerful at 1/10 the cost. 1/10 the cost. 1/10 the cost. >> Right. And it's And I respect also that >> Right. And it's And I respect also that >> Right. And it's And I respect also that you're calling out the cost because this you're calling out the cost because this you're calling out the cost because this token maxing thing is out of control. token maxing thing is out of control. token maxing thing is out of control. People are using the biggest models, People are using the biggest models, People are using the biggest models, like you said, you run it for a month, like you said, you run it for a month, like you said, you run it for a month, it costs a million dollars. God knows it costs a million dollars. God knows it costs a million dollars. God knows what it does to the ecosystem and to the what it does to the ecosystem and to the what it does to the ecosystem and to the ecology. Who knows how much water got ecology. Who knows how much water got ecology. Who knows how much water got burned because they ran, you know, Fable burned because they ran, you know, Fable burned because they ran, you know, Fable for that long. When the problems can be for that long. When the problems can be for that long. When the problems can be solved by, to your point, PhDs, static solved by, to your point, PhDs, static solved by, to your point, PhDs, static analysis. Uh I When I think about like analysis. Uh I When I think about like analysis. Uh I When I think about like the static When I think about the the static When I think about the the static When I think about the software development life cycle and we software development life cycle and we software development life cycle and we mentioned like CICDs, and we know that mentioned like CICDs, and we know that mentioned like CICDs, and we know that in 2026, there are companies that do not in 2026, there are companies that do not in 2026, there are companies that do not even have a mature DevOps pipeline.

even have a mature DevOps pipeline. even have a mature DevOps pipeline. The number of folks that don't know what The number of folks that don't know what The number of folks that don't know what like cyclomatic complexity is, like cyclomatic complexity is, like cyclomatic complexity is, that don't run any static analysis on that don't run any static analysis on that don't run any static analysis on their code at all to look at the their code at all to look at the their code at all to look at the complexity of the code or the security complexity of the code or the security complexity of the code or the security issues around the code, um is a little issues around the code, um is a little issues around the code, um is a little scary. Like, we were talking about this scary. Like, we were talking about this scary. Like, we were talking about this in the late '90s. How often are you in the late '90s. How often are you in the late '90s. How often are you finding when you walk into a customer finding when you walk into a customer finding when you walk into a customer with with Endor Labs' products and with with Endor Labs' products and with with Endor Labs' products and they're thinking that this is like This they're thinking that this is like This they're thinking that this is like This is magic. And you're like, "No, it's is magic. And you're like, "No, it's is magic. And you're like, "No, it's actually actually actually science." science." science." >> Yeah. >> Yeah. >> Yeah. >> You know, Scott, unfortunately, >> You know, Scott, unfortunately, >> You know, Scott, unfortunately, we come from a world where security was we come from a world where security was we come from a world where security was mostly built around compliance. Like, mostly built around compliance. Like, mostly built around compliance. Like, even this problem with open source. If even this problem with open source. If even this problem with open source. If you think about it 20 years ago, this you think about it 20 years ago, this you think about it 20 years ago, this whole ingestion pipeline, if you will, whole ingestion pipeline, if you will, whole ingestion pipeline, if you will, or verification, or OS Like, the open or verification, or OS Like, the open or verification, or OS Like, the open source program offices were built around source program offices were built around source program offices were built around license compliance originally. license compliance originally. license compliance originally. >> Right. >> Right. >> Right. >> And then they evolved to take on some of >> And then they evolved to take on some of >> And then they evolved to take on some of the vulnerability management aspects. the vulnerability management aspects. the vulnerability management aspects. And today they're evolving to kind of And today they're evolving to kind of And today they're evolving to kind of software supply chain security aspects. software supply chain security aspects. software supply chain security aspects. It's Look, there are a lot of continuous It's Look, there are a lot of continuous It's Look, there are a lot of continuous change. The thing I love about being in change. The thing I love about being in change. The thing I love about being in the cybersecurity industry is it's a the cybersecurity industry is it's a the cybersecurity industry is it's a cat-and-mouse game, right? One door cat-and-mouse game, right? One door cat-and-mouse game, right? One door shuts, another opens, and you're always shuts, another opens, and you're always shuts, another opens, and you're always trying to trying to trying to out-innovate.

out-innovate. out-innovate. And And And yeah, traditionally, we didn't write yeah, traditionally, we didn't write yeah, traditionally, we didn't write this much code. Traditionally, we didn't this much code. Traditionally, we didn't this much code. Traditionally, we didn't use this much open source, right? Our use this much open source, right? Our use this much open source, right? Our reliance on open source was what? Maybe reliance on open source was what? Maybe reliance on open source was what? Maybe 20, 30% of our legacy application was 20, 30% of our legacy application was 20, 30% of our legacy application was open source. Today, it's 80 or 90%. I open source. Today, it's 80 or 90%. I open source. Today, it's 80 or 90%. I would argue 99%, 99.9% of code in a would argue 99%, 99.9% of code in a would argue 99%, 99.9% of code in a modern application won't be written by a modern application won't be written by a modern application won't be written by a developer. And so, like, things are developer. And so, like, things are developer. And so, like, things are shifting so rapidly. Like, you know, shifting so rapidly. Like, you know, shifting so rapidly. Like, you know, skills didn't exist last year. MCP was skills didn't exist last year. MCP was skills didn't exist last year. MCP was hot, and then something else comes hot, and then something else comes hot, and then something else comes around, and now CLIs are great again. around, and now CLIs are great again. around, and now CLIs are great again. Like, the beautiful thing about being in Like, the beautiful thing about being in Like, the beautiful thing about being in software development is it is constantly software development is it is constantly software development is it is constantly changing for the better. But, that puts changing for the better. But, that puts changing for the better. But, that puts a moving target to say, like, okay, how a moving target to say, like, okay, how a moving target to say, like, okay, how you innovate? You have to kind of make you innovate? You have to kind of make you innovate? You have to kind of make some grounded assumptions, and that some grounded assumptions, and that some grounded assumptions, and that assumption we made is one, assumption we made is one, assumption we made is one, more code, not less code, will be more code, not less code, will be more code, not less code, will be out kind of from the outside of your out kind of from the outside of your out kind of from the outside of your trust boundary. trust boundary. trust boundary. >> Mhm. >> Mhm. >> Mhm. >> The second was >> The second was >> The second was the, you know, the convolution, like, the, you know, the convolution, like, the, you know, the convolution, like, code is a graph. You have to solve this code is a graph. You have to solve this code is a graph. You have to solve this as a problem, a graph problem.

as a problem, a graph problem. as a problem, a graph problem. Right? Like, like, trying to scan Right? Like, like, trying to scan Right? Like, like, trying to scan manifest files and say, oh, I see this manifest files and say, oh, I see this manifest files and say, oh, I see this import, so must be there. Doesn't work import, so must be there. Doesn't work import, so must be there. Doesn't work in modern application architectures. in modern application architectures. in modern application architectures. >> Yeah. Well, in modern applications and >> Yeah. Well, in modern applications and >> Yeah. Well, in modern applications and modern development now is not just I modern development now is not just I modern development now is not just I write some code and I check it in. My write some code and I check it in. My write some code and I check it in. My coding agent or myself, I'll have GitHub coding agent or myself, I'll have GitHub coding agent or myself, I'll have GitHub write access. I'll have package manager write access. I'll have package manager write access. I'll have package manager credentials. I'll have MCP tools. I'll credentials. I'll have MCP tools. I'll credentials. I'll have MCP tools. I'll have cloud credentials. I have CI have cloud credentials. I have CI have cloud credentials. I have CI access. access. access. Uh everyone has way more access than Uh everyone has way more access than Uh everyone has way more access than they need to, which brings me to my next they need to, which brings me to my next they need to, which brings me to my next question, question, question, which is they expect us to go faster which is they expect us to go faster which is they expect us to go faster than ever, than ever, than ever, but doesn't this just mean they're just but doesn't this just mean they're just but doesn't this just mean they're just going to lock our developer workstations going to lock our developer workstations going to lock our developer workstations down and make us all just RDP into the down and make us all just RDP into the down and make us all just RDP into the cloud somewhere? Like, I want to move cloud somewhere? Like, I want to move cloud somewhere? Like, I want to move fast, and I don't want Defender or fast, and I don't want Defender or fast, and I don't want Defender or whatever whatever whatever agent thing that's running on my machine agent thing that's running on my machine agent thing that's running on my machine to slow me down. How do you find that to slow me down. How do you find that to slow me down. How do you find that balance between the only secure machine balance between the only secure machine balance between the only secure machine is one that's not plugged into the is one that's not plugged into the is one that's not plugged into the internet, internet, internet, you know, and the speed and the you know, and the speed and the you know, and the speed and the expectation that they they want us to go expectation that they they want us to go expectation that they they want us to go at? at? at? >> It's a great question. It's There is no >> It's a great question. It's There is no >> It's a great question. It's There is no perfect solution, right? Like perfect solution, right? Like perfect solution, right? Like the the history security always comes at the the history security always comes at the the history security always comes at a cost. The question is is that a a cost. The question is is that a a cost. The question is is that a tolerable cost or not?

tolerable cost or not? tolerable cost or not? >> Right. >> Right. >> Right. >> Right. I think traditionally when you >> Right. I think traditionally when you >> Right. I think traditionally when you used to build applications and then you used to build applications and then you used to build applications and then you would receive an 800-page web inspect would receive an 800-page web inspect would receive an 800-page web inspect report of vulnerabilities that would report of vulnerabilities that would report of vulnerabilities that would take you another 6 months to fix before take you another 6 months to fix before take you another 6 months to fix before you could release. Like we don't have, you could release. Like we don't have, you could release. Like we don't have, you know, the ability to tolerate that you know, the ability to tolerate that you know, the ability to tolerate that today. today. today. Like do we have ability to tolerate 5 Like do we have ability to tolerate 5 Like do we have ability to tolerate 5 million findings from our platform every million findings from our platform every million findings from our platform every 6 months? No. 6 months? No. 6 months? No. But But But I think as long as we are reasonable to I think as long as we are reasonable to I think as long as we are reasonable to know today, know today, know today, you cannot live with security through you cannot live with security through you cannot live with security through obscurity. What I mean by that? obscurity. What I mean by that? obscurity. What I mean by that? We would make a series of compromises to We would make a series of compromises to We would make a series of compromises to ship fast, which would mean security ship fast, which would mean security ship fast, which would mean security would not be taken seriously in would not be taken seriously in would not be taken seriously in software. software. software. And you would hope that nobody would And you would hope that nobody would And you would hope that nobody would find those vulnerabilities. Today with find those vulnerabilities. Today with find those vulnerabilities. Today with models like Mythos and Fable and GPT-55, models like Mythos and Fable and GPT-55, models like Mythos and Fable and GPT-55, every vulnerability that is in existence every vulnerability that is in existence every vulnerability that is in existence in your software will be found. So the in your software will be found. So the in your software will be found. So the question becomes question becomes question becomes you can't make those trade-off you can't make those trade-off you can't make those trade-off decisions. So what you're now trying to decisions. So what you're now trying to decisions. So what you're now trying to do is what is the lowest cost at which I do is what is the lowest cost at which I do is what is the lowest cost at which I can inject security can inject security can inject security into my development workflow cuz I can into my development workflow cuz I can into my development workflow cuz I can no longer accept the risk and ship it.

no longer accept the risk and ship it. no longer accept the risk and ship it. >> Right. And >> Right. And >> Right. And >> what is that low toleration of cost? >> what is that low toleration of cost? >> what is that low toleration of cost? The more left in the process, The more left in the process, The more left in the process, >> Mhm. >> Mhm. >> Mhm. >> the cheaper it gets. It's like It's like >> the cheaper it gets. It's like It's like >> the cheaper it gets. It's like It's like car manufacturing. If you try to issue a car manufacturing. If you try to issue a car manufacturing. If you try to issue a recall once you've sold a 50,000 cars to recall once you've sold a 50,000 cars to recall once you've sold a 50,000 cars to your your your the customers, it's going to be the customers, it's going to be the customers, it's going to be expensive. If you had quality controls expensive. If you had quality controls expensive. If you had quality controls in the assembly line and you were just in the assembly line and you were just in the assembly line and you were just fixing those things well before that car fixing those things well before that car fixing those things well before that car was out to a showroom or a client's was out to a showroom or a client's was out to a showroom or a client's hands, way cheaper. So in software hands, way cheaper. So in software hands, way cheaper. So in software factories, the cheapest place to insert factories, the cheapest place to insert factories, the cheapest place to insert security security security and in the loop is in the agentic and in the loop is in the agentic and in the loop is in the agentic workflow. Developer gets a spec, workflow. Developer gets a spec, workflow. Developer gets a spec, >> agent builds. >> agent builds. >> agent builds. >> Right. >> Right. >> Right. >> Okay, as the agent is building, we know >> Okay, as the agent is building, we know >> Okay, as the agent is building, we know they don't have the intelligence of they don't have the intelligence of they don't have the intelligence of being very secure, but if we fuse and being very secure, but if we fuse and being very secure, but if we fuse and insert the Endor intelligence, for insert the Endor intelligence, for insert the Endor intelligence, for example, in there, the code before it's example, in there, the code before it's example, in there, the code before it's complete, the plan being complete, you complete, the plan being complete, you complete, the plan being complete, you would have more secure code. would have more secure code. would have more secure code. >> Great. >> Great. >> Great. >> Okay. >> Okay. >> Okay. >> Then it goes to PR, and so on and so you >> Then it goes to PR, and so on and so you >> Then it goes to PR, and so on and so you kind of keep reducing the security kind of keep reducing the security kind of keep reducing the security issues till you get to a zero point um issues till you get to a zero point um issues till you get to a zero point um uh you know, on the other side.

uh you know, on the other side. uh you know, on the other side. >> Okay, so you beat me to it because that >> Okay, so you beat me to it because that >> Okay, so you beat me to it because that was my next question cuz I was trying to was my next question cuz I was trying to was my next question cuz I was trying to visualize. You keep saying shift left, visualize. You keep saying shift left, visualize. You keep saying shift left, how far left do you go? And And AURI, is how far left do you go? And And AURI, is how far left do you go? And And AURI, is that pronounced correctly? A- AURI, the that pronounced correctly? A- AURI, the that pronounced correctly? A- AURI, the security security security >> AURI. >> AURI. >> AURI. >> AURI. So, Endor Labs AURI is the >> AURI. So, Endor Labs AURI is the >> AURI. So, Endor Labs AURI is the security intelligence layer for coding security intelligence layer for coding security intelligence layer for coding agents, right? So, we've got the agents, right? So, we've got the agents, right? So, we've got the deterministic aspect, we've got the deterministic aspect, we've got the deterministic aspect, we've got the probabilistic model reasoning, and then probabilistic model reasoning, and then probabilistic model reasoning, and then you've got, you know, parts you can you've got, you know, parts you can you've got, you know, parts you can prove and parts that are that are prove and parts that are that are prove and parts that are that are special herbs and spices, Endor Labs special herbs and spices, Endor Labs special herbs and spices, Endor Labs specific herbs and spices. specific herbs and spices. specific herbs and spices. But, this should be should the secure But, this should be should the secure But, this should be should the secure coding agent be in the generation loop coding agent be in the generation loop coding agent be in the generation loop preventing the mistakes so it never preventing the mistakes so it never preventing the mistakes so it never escapes is what you're saying. escapes is what you're saying. escapes is what you're saying. >> Don't Yes, the worst thing we can do in >> Don't Yes, the worst thing we can do in >> Don't Yes, the worst thing we can do in this agentic SDLC when we're trying to this agentic SDLC when we're trying to this agentic SDLC when we're trying to ramp up code generation ramp up code generation ramp up code generation is ramp up the number of security is ramp up the number of security is ramp up the number of security tickets we push to humans. tickets we push to humans. tickets we push to humans. >> There it is. >> There it is. >> There it is. >> If you have agents create the code and >> If you have agents create the code and >> If you have agents create the code and make [clears throat] all of those things make [clears throat] all of those things make [clears throat] all of those things post generation, review, find the post generation, review, find the post generation, review, find the problem, put it as tickets in a backlog problem, put it as tickets in a backlog problem, put it as tickets in a backlog human, we did not solve for anything in human, we did not solve for anything in human, we did not solve for anything in terms of innovation or acceleration of terms of innovation or acceleration of terms of innovation or acceleration of software development. The only way to do software development. The only way to do software development. The only way to do this is the code being created by the this is the code being created by the this is the code being created by the agents, agents, agents, those agents have to be held accountable those agents have to be held accountable those agents have to be held accountable to improve the security efficacy. How do to improve the security efficacy. How do to improve the security efficacy. How do you do that? Agents are smart, they you do that? Agents are smart, they you do that? Agents are smart, they follow instructions, they follow tools.

follow instructions, they follow tools. follow instructions, they follow tools. You give them the tools, you give them You give them the tools, you give them You give them the tools, you give them the instructions, you tell them what to the instructions, you tell them what to the instructions, you tell them what to go fix. Um which, you know, is the is go fix. Um which, you know, is the is go fix. Um which, you know, is the is the is a key part of the Endor value the is a key part of the Endor value the is a key part of the Endor value proposition. We didn't talk about this. proposition. We didn't talk about this. proposition. We didn't talk about this. You talked about the static analysis and You talked about the static analysis and You talked about the static analysis and the tools and the [clears throat] herbs the tools and the [clears throat] herbs the tools and the [clears throat] herbs and spices. and spices. and spices. One of the most important herbs and One of the most important herbs and One of the most important herbs and spice is spice is spice is for all of this code on open source that for all of this code on open source that for all of this code on open source that these models are trained on, which they these models are trained on, which they these models are trained on, which they don't know what is secure or insecure, don't know what is secure or insecure, don't know what is secure or insecure, Endor does. We have been labeling Endor does. We have been labeling Endor does. We have been labeling hundreds of thousands of vulnerabilities hundreds of thousands of vulnerabilities hundreds of thousands of vulnerabilities that have been created over the last 20 that have been created over the last 20 that have been created over the last 20 years to understand which lines of code years to understand which lines of code years to understand which lines of code in open source have them, which lines of in open source have them, which lines of in open source have them, which lines of code in which future releases fix them, code in which future releases fix them, code in which future releases fix them, etc., etc. So, when we fuse that etc., etc. So, when we fuse that etc., etc. So, when we fuse that intelligence to a model that is intelligence to a model that is intelligence to a model that is recommending some code that has a recommending some code that has a recommending some code that has a problem, problem, problem, we can help it rewrite it correctly. we can help it rewrite it correctly. we can help it rewrite it correctly. >> Right. This is where things get >> Right. This is where things get >> Right. This is where things get interesting. So, we mentioned a couple interesting. So, we mentioned a couple interesting. So, we mentioned a couple of minutes ago, or maybe 10 15 minutes of minutes ago, or maybe 10 15 minutes of minutes ago, or maybe 10 15 minutes ago, about how if you train an AI like ago, about how if you train an AI like ago, about how if you train an AI like the Frontier models on the Frontier models on the Frontier models on everything, you get mediocrity. But, if everything, you get mediocrity. But, if everything, you get mediocrity. But, if you're doing targeted training on CVEs, you're doing targeted training on CVEs, you're doing targeted training on CVEs, you are doing a focused train on you are doing a focused train on you are doing a focused train on specific models and saying, "This is specific models and saying, "This is specific models and saying, "This is what is wrong, and we know how these what is wrong, and we know how these what is wrong, and we know how these these these CVEs have been fixed over 20 these these CVEs have been fixed over 20 these these CVEs have been fixed over 20 years."

years." years." You know so much more. This is a a a You know so much more. This is a a a You know so much more. This is a a a very laser-focused kind of model, and very laser-focused kind of model, and very laser-focused kind of model, and not a general-purpose not a general-purpose not a general-purpose kind of model. kind of model. kind of model. Interesting. Okay. So, then, what do you Interesting. Okay. So, then, what do you Interesting. Okay. So, then, what do you think the next big attack is? Is it think the next big attack is? Is it think the next big attack is? Is it going to be supply chain, like we're going to be supply chain, like we're going to be supply chain, like we're seeing to target source code? Is it seeing to target source code? Is it seeing to target source code? Is it going to be someone going to be someone going to be someone attacking the agentic loop itself? Or, attacking the agentic loop itself? Or, attacking the agentic loop itself? Or, is it going to be script kiddies making is it going to be script kiddies making is it going to be script kiddies making skills? Where Where should we be turning skills? Where Where should we be turning skills? Where Where should we be turning our attention? our attention? our attention? >> So, I think it's two sets of problems. I >> So, I think it's two sets of problems. I >> So, I think it's two sets of problems. I think, one, as I said, every security think, one, as I said, every security think, one, as I said, every security weakness in your software will be found. weakness in your software will be found. weakness in your software will be found. It's not a question of if, it's a It's not a question of if, it's a It's not a question of if, it's a question of how fast. And so, I think question of how fast. And so, I think question of how fast. And so, I think we're playing this cat-and-mouse game to we're playing this cat-and-mouse game to we're playing this cat-and-mouse game to go fix 20 years of software go fix 20 years of software go fix 20 years of software vulnerabilities vulnerabilities vulnerabilities in the next 6 to 12 18 months. It's in the next 6 to 12 18 months. It's in the next 6 to 12 18 months. It's going to be extremely painful. There's a going to be extremely painful. There's a going to be extremely painful. There's a lot of security debt that we all have to lot of security debt that we all have to lot of security debt that we all have to clear up. And, you know, we're going to clear up. And, you know, we're going to clear up. And, you know, we're going to have to go through the cycles because have to go through the cycles because have to go through the cycles because mythos-fable-like models mythos-fable-like models mythos-fable-like models are going to find everything. are going to find everything. are going to find everything. And, the volume is going to be And, the volume is going to be And, the volume is going to be excessive. Just If you look at the excessive. Just If you look at the excessive. Just If you look at the project last week update from Anthropic, project last week update from Anthropic, project last week update from Anthropic, 40,000 CVs already reported to open 40,000 CVs already reported to open 40,000 CVs already reported to open source zero-day.

source zero-day. source zero-day. And and you know, the open source And and you know, the open source And and you know, the open source maintainers are looking at this and maintainers are looking at this and maintainers are looking at this and going, "Huh, well, I was just doing this going, "Huh, well, I was just doing this going, "Huh, well, I was just doing this for fun. Now you're going to go throw for fun. Now you're going to go throw for fun. Now you're going to go throw all this crap of work at me." all this crap of work at me." all this crap of work at me." Like and then you're going to tell me I Like and then you're going to tell me I Like and then you're going to tell me I got to fix it really, really fast. Like got to fix it really, really fast. Like got to fix it really, really fast. Like I don't owe you anything. Mhm. So like I don't owe you anything. Mhm. So like I don't owe you anything. Mhm. So like this whole trust in open source this whole trust in open source this whole trust in open source like it's going to come to a head like it's going to come to a head like it's going to come to a head because it's like who's going to fix all because it's like who's going to fix all because it's like who's going to fix all these problems? So we have partnered these problems? So we have partnered these problems? So we have partnered with Anthropic and Linux Foundation and with Anthropic and Linux Foundation and with Anthropic and Linux Foundation and Microsoft and others to start this Microsoft and others to start this Microsoft and others to start this project called Project Acropolis, which project called Project Acropolis, which project called Project Acropolis, which is like us coming together as a is like us coming together as a is like us coming together as a community of experts community of experts community of experts >> Mhm. >> Mhm. >> Mhm. >> that are going to help maintainers >> that are going to help maintainers >> that are going to help maintainers go fix years and years worth of security go fix years and years worth of security go fix years and years worth of security problems in a very short amount of time problems in a very short amount of time problems in a very short amount of time because the world depends on it. Like because the world depends on it. Like because the world depends on it. Like I'm not worried about Microsoft or I'm not worried about Microsoft or I'm not worried about Microsoft or Google or others being able to patch. Google or others being able to patch. Google or others being able to patch. I'm worried about the water supply, the I'm worried about the water supply, the I'm worried about the water supply, the critical infrastructure, the nuclear critical infrastructure, the nuclear critical infrastructure, the nuclear power plants. Like power plants. Like power plants. Like that's where the real risk to humanity that's where the real risk to humanity that's where the real risk to humanity is in the short order. So So I think we is in the short order. So So I think we is in the short order. So So I think we as an industry have come to this as an industry have come to this as an industry have come to this reckoning point. We've got to get reckoning point. We've got to get reckoning point. We've got to get through this. It's going to be painful through this. It's going to be painful through this. It's going to be painful for some time till we normalize all this for some time till we normalize all this for some time till we normalize all this backlog of security problems found and backlog of security problems found and backlog of security problems found and improve secure code generation moving improve secure code generation moving improve secure code generation moving forward. So that's one.

forward. So that's one. forward. So that's one. The second thing is yes, I think the The second thing is yes, I think the The second thing is yes, I think the software supply chain is complicated, software supply chain is complicated, software supply chain is complicated, more complicated with skills and MCP and more complicated with skills and MCP and more complicated with skills and MCP and GitHub actions and NPM and PyPI. And I GitHub actions and NPM and PyPI. And I GitHub actions and NPM and PyPI. And I think we're going to have to move to think we're going to have to move to think we're going to have to move to operating at on zero trust principles in operating at on zero trust principles in operating at on zero trust principles in software development. When we we when we software development. When we we when we software development. When we we when we hear the word development environments, hear the word development environments, hear the word development environments, we never treated [clears throat] them as we never treated [clears throat] them as we never treated [clears throat] them as production. And so you know, this whole production. And so you know, this whole production. And so you know, this whole smash and grab robberies, they're going smash and grab robberies, they're going smash and grab robberies, they're going to continue till as an industry critical to continue till as an industry critical to continue till as an industry critical mass shift happens mass shift happens mass shift happens to focus developer workstation security. to focus developer workstation security. to focus developer workstation security. So I think that's going to be kind of So I think that's going to be kind of So I think that's going to be kind of the next big thing. the next big thing. the next big thing. And the third thing is I think we all And the third thing is I think we all And the third thing is I think we all have to retool have to retool have to retool for security for security for security to be in the agentic workflow cuz to be in the agentic workflow cuz to be in the agentic workflow cuz otherwise we are generating a lot of otherwise we are generating a lot of otherwise we are generating a lot of security slop right now security slop right now security slop right now >> Mhm. >> Mhm. >> Mhm. >> which will come to bite us in the coming >> which will come to bite us in the coming >> which will come to bite us in the coming weeks, months, and years. weeks, months, and years. weeks, months, and years. >> Yeah, figuring out how to get past the >> Yeah, figuring out how to get past the >> Yeah, figuring out how to get past the slop cannons right now is going to be a slop cannons right now is going to be a slop cannons right now is going to be a mess. You're absolutely right. mess. You're absolutely right. mess. You're absolutely right. Do you think have we proven I mean Do you think have we proven I mean Do you think have we proven I mean you've got the science and you've got you've got the science and you've got you've got the science and you've got the PhDs that is secure code possible or the PhDs that is secure code possible or the PhDs that is secure code possible or will we be always be playing will we be always be playing will we be always be playing Whac-A-Mole? Is perfectly secure code Whac-A-Mole? Is perfectly secure code Whac-A-Mole? Is perfectly secure code possible?

possible? possible? >> Yeah. >> Yeah. >> Yeah. I think it's more possible today than it I think it's more possible today than it I think it's more possible today than it ever has been. Like trying to do annual ever has been. Like trying to do annual ever has been. Like trying to do annual security training for software security training for software security training for software development professionals and then development professionals and then development professionals and then expecting 10,000 developers in a company expecting 10,000 developers in a company expecting 10,000 developers in a company to follow the same model at all times to follow the same model at all times to follow the same model at all times doesn't work. But agents are good at doesn't work. But agents are good at doesn't work. But agents are good at following instructions and rules and following instructions and rules and following instructions and rules and direction if you can give them direction if you can give them direction if you can give them consistent direction. So, is it going to consistent direction. So, is it going to consistent direction. So, is it going to be perfect? No, but is there an be perfect? No, but is there an be perfect? No, but is there an opportunity for us to create a lot more opportunity for us to create a lot more opportunity for us to create a lot more secure code? secure code? secure code? >> Mhm. >> Mhm. >> Mhm. >> Yes, but it won't magically happen. >> Yes, but it won't magically happen. >> Yes, but it won't magically happen. It will not certainly happen with oh, It will not certainly happen with oh, It will not certainly happen with oh, the frontier labs are just going to do the frontier labs are just going to do the frontier labs are just going to do it all and we'll all be good. Right? it all and we'll all be good. Right? it all and we'll all be good. Right? There'll be different expertise required There'll be different expertise required There'll be different expertise required in different area and you'll have to in different area and you'll have to in different area and you'll have to build it all into your common harness. build it all into your common harness. build it all into your common harness. And it's not going to be perfect. The And it's not going to be perfect. The And it's not going to be perfect. The reason it's not going to be perfect is reason it's not going to be perfect is reason it's not going to be perfect is security is all a given a function of security is all a given a function of security is all a given a function of time, cost, compute. More things will be time, cost, compute. More things will be time, cost, compute. More things will be found. So, it's an evergreen problem. found. So, it's an evergreen problem. found. So, it's an evergreen problem. Even if I ship what seems to be Even if I ship what seems to be Even if I ship what seems to be completely secure code today completely secure code today completely secure code today somebody will find a way to break it somebody will find a way to break it somebody will find a way to break it tomorrow. So, the more important tomorrow. So, the more important tomorrow. So, the more important workflow we all need to have is that workflow we all need to have is that workflow we all need to have is that maturity of CI/CD pipeline, maturity of maturity of CI/CD pipeline, maturity of maturity of CI/CD pipeline, maturity of security security security and automation, agentic remediation and automation, agentic remediation and automation, agentic remediation automation. So, when that next thing automation. So, when that next thing automation. So, when that next thing comes up tomorrow morning comes up tomorrow morning comes up tomorrow morning it's solved in 30 minutes and it's solved in 30 minutes and it's solved in 30 minutes and it's handled, right? Like I I don't need it's handled, right? Like I I don't need it's handled, right? Like I I don't need a war room like we did with Log4j for a war room like we did with Log4j for a war room like we did with Log4j for weeks and months to go address that weeks and months to go address that weeks and months to go address that problem.

problem. problem. >> That would be the dream. That would be >> That would be the dream. That would be >> That would be the dream. That would be the dream. And then that's what you're the dream. And then that's what you're the dream. And then that's what you're working towards. I mean that's the goal, working towards. I mean that's the goal, working towards. I mean that's the goal, right? right? right? >> That's right. We're kind of moving >> That's right. We're kind of moving >> That's right. We're kind of moving towards much more agentic and more towards much more agentic and more towards much more agentic and more autonomous software security embedded autonomous software security embedded autonomous software security embedded into the workflow into the workflow into the workflow giving that intelligence to the agents giving that intelligence to the agents giving that intelligence to the agents to do 90-95% of the work to do 90-95% of the work to do 90-95% of the work >> Mhm. >> Mhm. >> Mhm. >> out of the box so the humans are kind of >> out of the box so the humans are kind of >> out of the box so the humans are kind of the final verification and trust layer the final verification and trust layer the final verification and trust layer not the core part of how we're going to not the core part of how we're going to not the core part of how we're going to action everything. action everything. action everything. >> Very cool. Well, I want to encourage >> Very cool. Well, I want to encourage >> Very cool. Well, I want to encourage viewers and listeners to check out viewers and listeners to check out viewers and listeners to check out enderlabs.com. Particularly check out enderlabs.com. Particularly check out enderlabs.com. Particularly check out the research section, the work that the research section, the work that the research section, the work that you're doing, the agent benchmark, the you're doing, the agent benchmark, the you're doing, the agent benchmark, the threat research, the work that you're threat research, the work that you're threat research, the work that you're doing with universities is really doing with universities is really doing with universities is really powerful. And then they can go and learn powerful. And then they can go and learn powerful. And then they can go and learn about all the developer tools and the about all the developer tools and the about all the developer tools and the platforms that you are working on. Thank platforms that you are working on. Thank platforms that you are working on. Thank you so much Varun Badhwar for chatting you so much Varun Badhwar for chatting you so much Varun Badhwar for chatting with me today. with me today. with me today. >> Thanks Scott. Great to be here. >> Thanks Scott. Great to be here. >> Thanks Scott. Great to be here. >> This has been another episode of >> This has been another episode of >> This has been another episode of Hanselminutes and we'll see you again Hanselminutes and we'll see you again Hanselminutes and we'll see you again next week.