AI and the Software Supply Chain with Endor Lab's Varun Badhwar

And today, that is the developer And today, that is the developer workstation. I just need a developer to workstation. I just need a developer to workstation. I just need a developer to do an NPM install or pip install, do an NPM install or pip install, do an NPM install or pip install, something that has malicious intent. something that has malicious intent. something that has malicious intent. That malicious library That malicious library That malicious library most commonly most commonly most commonly >> [clears throat] >> [clears throat] >> [clears throat] >> is now going to exfiltrate any variables >> is now going to exfiltrate any variables >> is now going to exfiltrate any variables and credentials and keys that it can and credentials and keys that it can and credentials and keys that it can find off the machine and it's out in 5 find off the machine and it's out in 5 find off the machine and it's out in 5 seconds. Like there is no notion of, seconds. Like there is no notion of, seconds. Like there is no notion of, well, I have some checks baked into my well, I have some checks baked into my well, I have some checks baked into my pull request when it gets there I'll pull request when it gets there I'll pull request when it gets there I'll catch it so it won't go in my CI/CD. I catch it so it won't go in my CI/CD. I catch it so it won't go in my CI/CD. I got everything I needed it's got it's a got everything I needed it's got it's a got everything I needed it's got it's a smash and grab attempt. smash and grab attempt. smash and grab attempt. >> Hey friends, I'm Scott Hanselman and >> Hey friends, I'm Scott Hanselman and >> Hey friends, I'm Scott Hanselman and it's another episode of Hanselminutes. it's another episode of Hanselminutes. it's another episode of Hanselminutes. Today I'm chatting with Varun Badhwar. Today I'm chatting with Varun Badhwar. Today I'm chatting with Varun Badhwar. He's the founder and CEO at Endor Labs. He's the founder and CEO at Endor Labs. He's the founder and CEO at Endor Labs. How are you, sir? How are you, sir? How are you, sir? >> Very well, Scott. Great to be here. >> Very well, Scott. Great to be here. >> Very well, Scott. Great to be here. >> You know, I have been having so many >> You know, I have been having so many >> You know, I have been having so many conversations about bringing AI into an conversations about bringing AI into an conversations about bringing AI into an organization and people are finding it organization and people are finding it organization and people are finding it to be very unnerving because they're to be very unnerving because they're to be very unnerving because they're saying, "Well, I don't trust this code." saying, "Well, I don't trust this code." saying, "Well, I don't trust this code." And I think that's so funny because And I think that's so funny because And I think that's so funny because I never trusted anyone's code whether I never trusted anyone's code whether I never trusted anyone's code whether they work for me or not. So, I wonder they work for me or not. So, I wonder they work for me or not. So, I wonder why they why they frame things in that why they why they frame things in that why they why they frame things in that way.

way. way. >> Yeah, it's a great opening and I think >> Yeah, it's a great opening and I think >> Yeah, it's a great opening and I think I I kind of say there are two I they're I I kind of say there are two I they're I I kind of say there are two I they're kind of kind of kind of two eyes in the storm right now. two eyes in the storm right now. two eyes in the storm right now. One is One is One is that there are more people writing more that there are more people writing more that there are more people writing more code than ever before and it's not just code than ever before and it's not just code than ever before and it's not just limited to your trained software limited to your trained software limited to your trained software professionals that have computer science professionals that have computer science professionals that have computer science degrees, degrees, degrees, but it's a financial analyst that wants but it's a financial analyst that wants but it's a financial analyst that wants to build a forecasting app in Vercel or to build a forecasting app in Vercel or to build a forecasting app in Vercel or Replit today. And that's code. Replit today. And that's code. Replit today. And that's code. And then And then And then the other side of this is the the other side of this is the the other side of this is the conversation these days in the industry conversation these days in the industry conversation these days in the industry has shifted more from productivity to has shifted more from productivity to has shifted more from productivity to security and safety of models, right? security and safety of models, right? security and safety of models, right? Like Mythos can't be out because it is Like Mythos can't be out because it is Like Mythos can't be out because it is so dangerous to humanity because of its so dangerous to humanity because of its so dangerous to humanity because of its cyber powers. What people don't realize cyber powers. What people don't realize cyber powers. What people don't realize though is while Mythos is being limited though is while Mythos is being limited though is while Mythos is being limited in its adoption, there's open source in its adoption, there's open source in its adoption, there's open source models out there uh available on Hugging models out there uh available on Hugging models out there uh available on Hugging face today that are good enough at doing face today that are good enough at doing face today that are good enough at doing a lot of these tasks. So, I think look a lot of these tasks. So, I think look a lot of these tasks. So, I think look it is it is it is appropriate for people to be concerned appropriate for people to be concerned appropriate for people to be concerned about speed and safety and scale at about speed and safety and scale at about speed and safety and scale at which all of this is coming at them.

which all of this is coming at them. which all of this is coming at them. It's about, you know, we're starting It's about, you know, we're starting It's about, you know, we're starting from ground zero on how to build the from ground zero on how to build the from ground zero on how to build the guardrails and the safety harnesses for guardrails and the safety harnesses for guardrails and the safety harnesses for all of this. Unfortunately, security is all of this. Unfortunately, security is all of this. Unfortunately, security is always an afterthought. It's no always an afterthought. It's no always an afterthought. It's no different here, although I would say different here, although I would say different here, although I would say here it's here it's here it's being talked about much sooner than say being talked about much sooner than say being talked about much sooner than say it was in the cloud era or, you know, or it was in the cloud era or, you know, or it was in the cloud era or, you know, or or other previous technology shifts. So, or other previous technology shifts. So, or other previous technology shifts. So, I get it. I feel the pain. But the good I get it. I feel the pain. But the good I get it. I feel the pain. But the good news, after we hopefully spend the next news, after we hopefully spend the next news, after we hopefully spend the next 30 minutes, is there's hope. There's a 30 minutes, is there's hope. There's a 30 minutes, is there's hope. There's a strategy we can apply to strategy we can apply to strategy we can apply to you know, doing this well because you know, doing this well because you know, doing this well because the same agents that you're scared about the same agents that you're scared about the same agents that you're scared about can now be used to actually do things at can now be used to actually do things at can now be used to actually do things at scale more securely with the right scale more securely with the right scale more securely with the right intelligence integrated in the right intelligence integrated in the right intelligence integrated in the right workflows and so on and so forth. workflows and so on and so forth. workflows and so on and so forth. >> Some organizations have a mature open >> Some organizations have a mature open >> Some organizations have a mature open source source source you know, open source programs office. you know, open source programs office. you know, open source programs office. They're used to ingressing untrusted They're used to ingressing untrusted They're used to ingressing untrusted code from random people. code from random people. code from random people. Should we look at code written by an Should we look at code written by an Should we look at code written by an agent the same way we look at open agent the same way we look at open agent the same way we look at open source code? Because I think that I've source code? Because I think that I've source code? Because I think that I've been consuming internet strangers code been consuming internet strangers code been consuming internet strangers code for 20 years. I wouldn't treat an for 20 years. I wouldn't treat an for 20 years. I wouldn't treat an agent's code any more or less trusted agent's code any more or less trusted agent's code any more or less trusted than I would treat random internet than I would treat random internet than I would treat random internet person.

person. person. >> Well, we certainly had capabilities in >> Well, we certainly had capabilities in >> Well, we certainly had capabilities in the more mature organizations to vet the more mature organizations to vet the more mature organizations to vet open source. But even that vetting was open source. But even that vetting was open source. But even that vetting was mostly focused on legal and license mostly focused on legal and license mostly focused on legal and license compliance and on known vulnerabilities. compliance and on known vulnerabilities. compliance and on known vulnerabilities. I think one of the biggest shifts that's I think one of the biggest shifts that's I think one of the biggest shifts that's happening the open source ecosystem is happening the open source ecosystem is happening the open source ecosystem is this idea that now this idea that now this idea that now people are either maliciously or people are either maliciously or people are either maliciously or accidentally shipping malware that's not accidentally shipping malware that's not accidentally shipping malware that's not even targeting even targeting even targeting kind of your CICD pipelines or anything. kind of your CICD pipelines or anything. kind of your CICD pipelines or anything. It's targeting straight the developer It's targeting straight the developer It's targeting straight the developer that does that NPM install. The moment that does that NPM install. The moment that does that NPM install. The moment that install happens on a machine well that install happens on a machine well that install happens on a machine well before it's vetted and put into a, you before it's vetted and put into a, you before it's vetted and put into a, you know, code repository or into a CICD know, code repository or into a CICD know, code repository or into a CICD pipeline. So, there is certainly I think pipeline. So, there is certainly I think pipeline. So, there is certainly I think that that that the risks have shifted and are more the risks have shifted and are more the risks have shifted and are more serious today in terms of malicious serious today in terms of malicious serious today in terms of malicious intent uh intent uh intent uh being shipped [clears throat] through being shipped [clears throat] through being shipped [clears throat] through open source. That's one. But yes, if you open source. That's one. But yes, if you open source. That's one. But yes, if you do move to a zero-trust development do move to a zero-trust development do move to a zero-trust development environment uh from the 100% implicit environment uh from the 100% implicit environment uh from the 100% implicit trust to strangers on the internet that trust to strangers on the internet that trust to strangers on the internet that many organizations operate in today, many organizations operate in today, many organizations operate in today, then I think you're getting closer and then I think you're getting closer and then I think you're getting closer and closer to how you treat agent and closer to how you treat agent and closer to how you treat agent and AI-generated code AI-generated code AI-generated code because there's a lot of similarities in because there's a lot of similarities in because there's a lot of similarities in AI-generated code. I mean, the the AI AI-generated code. I mean, the the AI AI-generated code. I mean, the the AI models didn't learn novel code. They models didn't learn novel code. They models didn't learn novel code. They were trained on all of last 20-30 years were trained on all of last 20-30 years were trained on all of last 20-30 years of code on GitHub and on the internet.

of code on GitHub and on the internet. of code on GitHub and on the internet. And so, they've learned the good, And so, they've learned the good, And so, they've learned the good, they've learned the bad, they've learned they've learned the bad, they've learned they've learned the bad, they've learned the ugly. If you figure out how to put the ugly. If you figure out how to put the ugly. If you figure out how to put the right guardrails, that's fine. the right guardrails, that's fine. the right guardrails, that's fine. But Scott, like we have to talk about But Scott, like we have to talk about But Scott, like we have to talk about the volume at which that's shifting, the volume at which that's shifting, the volume at which that's shifting, right? Like how many new packages do right? Like how many new packages do right? Like how many new packages do people install every week? A handful. people install every week? A handful. people install every week? A handful. But today, But today, But today, that the volume at which code is being that the volume at which code is being that the volume at which code is being produced that appears to be first-party produced that appears to be first-party produced that appears to be first-party code, dependencies are being installed, code, dependencies are being installed, code, dependencies are being installed, but it's not just dependencies from NPM but it's not just dependencies from NPM but it's not just dependencies from NPM and PyPI, it's now hugging face models and PyPI, it's now hugging face models and PyPI, it's now hugging face models and GitHub actions and like there's so and GitHub actions and like there's so and GitHub actions and like there's so many more things in the software supply many more things in the software supply many more things in the software supply chain. So, the surface area has chain. So, the surface area has chain. So, the surface area has expanded. And the last piece I'll expanded. And the last piece I'll expanded. And the last piece I'll mention, I was having a very interesting mention, I was having a very interesting mention, I was having a very interesting conversation with a CIO of a very big conversation with a CIO of a very big conversation with a CIO of a very big reputable company reputable company reputable company and he said, "Look, our finance team is and he said, "Look, our finance team is and he said, "Look, our finance team is building these apps and we found some building these apps and we found some building these apps and we found some security issues. We kind of emailed them security issues. We kind of emailed them security issues. We kind of emailed them and said, 'Hey, could you please update and said, 'Hey, could you please update and said, 'Hey, could you please update your app to fix this vulnerability?'" your app to fix this vulnerability?'" your app to fix this vulnerability?'" Scott, you want to guess what the Scott, you want to guess what the Scott, you want to guess what the response was from the team? response was from the team? response was from the team? >> No. >> No. >> No. >> "What's a vulnerability?" So, now like >> "What's a vulnerability?" So, now like >> "What's a vulnerability?" So, now like we're not talking to trained software we're not talking to trained software we're not talking to trained software [laughter] professionals. Now you're [laughter] professionals. Now you're [laughter] professionals. Now you're trying to level up these citizen trying to level up these citizen trying to level up these citizen developers in your company. He's like, developers in your company. He's like, developers in your company. He's like, "What do you mean? It's like, what do "What do you mean? It's like, what do "What do you mean? It's like, what do you mean this software is generated must you mean this software is generated must you mean this software is generated must be fine.

be fine. be fine. >> Oh god, that's not where I thought you >> Oh god, that's not where I thought you >> Oh god, that's not where I thought you were going with that at all. That is were going with that at all. That is were going with that at all. That is awful. awful. awful. Wow. Yeah, and see this is the thing. Wow. Yeah, and see this is the thing. Wow. Yeah, and see this is the thing. You've said before that we're moving You've said before that we're moving You've said before that we're moving from writing software to assembling from writing software to assembling from writing software to assembling software. And you know, before it was software. And you know, before it was software. And you know, before it was like write some code, find a library like write some code, find a library like write some code, find a library that hides a bit of abstraction. But that hides a bit of abstraction. But that hides a bit of abstraction. But we're sitting on top of this massive we're sitting on top of this massive we're sitting on top of this massive pyramid of abstractions. We've got pyramid of abstractions. We've got pyramid of abstractions. We've got executable instructions, sure, but executable instructions, sure, but executable instructions, sure, but dependencies deployed bytes, dependencies deployed bytes, dependencies deployed bytes, pulling in opaque libraries that are pulling in opaque libraries that are pulling in opaque libraries that are binaries, pulling in open source binaries, pulling in open source binaries, pulling in open source libraries, NPM particularly you call libraries, NPM particularly you call libraries, NPM particularly you call that one out. Uh I've even seen my that one out. Uh I've even seen my that one out. Uh I've even seen my my bot hallucinate packages that it my bot hallucinate packages that it my bot hallucinate packages that it thought might exist. thought might exist. thought might exist. >> Yeah. >> Yeah. >> Yeah. >> That did not exist. And I'm trying to >> That did not exist. And I'm trying to >> That did not exist. And I'm trying to figure out where did you even get that figure out where did you even get that figure out where did you even get that from? Which arguably then could be from? Which arguably then could be from? Which arguably then could be squatted on. squatted on. squatted on. And someone could put a package And someone could put a package And someone could put a package underneath that and attack me as well. I underneath that and attack me as well. I underneath that and attack me as well. I wanted to dig in a little bit more to wanted to dig in a little bit more to wanted to dig in a little bit more to the idea that the developer is now an the idea that the developer is now an the idea that the developer is now an attack vector. I'm used to people trying attack vector. I'm used to people trying attack vector. I'm used to people trying to steal production data. to steal production data. to steal production data. But now I have to worry about my own But now I have to worry about my own But now I have to worry about my own computer and what it has access to.

computer and what it has access to. computer and what it has access to. That's all new as well. That's all new as well. That's all new as well. >> Yeah, totally, right? I mean >> Yeah, totally, right? I mean >> Yeah, totally, right? I mean traditionally when we thought about traditionally when we thought about traditionally when we thought about software security software assurance software security software assurance software security software assurance programs, we focused a lot on the code. programs, we focused a lot on the code. programs, we focused a lot on the code. We wanted to run static analysis tools, We wanted to run static analysis tools, We wanted to run static analysis tools, find vulnerabilities, fix find vulnerabilities, fix find vulnerabilities, fix vulnerabilities. vulnerabilities. vulnerabilities. Today the problem is not just these Today the problem is not just these Today the problem is not just these accidental vulnerabilities in code. It's accidental vulnerabilities in code. It's accidental vulnerabilities in code. It's actually people, including a lot of actually people, including a lot of actually people, including a lot of nation states, targeting organizations, nation states, targeting organizations, nation states, targeting organizations, targeting public sector developers. So targeting public sector developers. So targeting public sector developers. So why developers? why developers? why developers? Well, developers are basically crown Well, developers are basically crown Well, developers are basically crown jewels in any company right now. You jewels in any company right now. You jewels in any company right now. You have coding agents that are running on have coding agents that are running on have coding agents that are running on almost every machine, which have almost every machine, which have almost every machine, which have extremely high privileges and ability to extremely high privileges and ability to extremely high privileges and ability to take action autonomously. You've got take action autonomously. You've got take action autonomously. You've got credentials to your cloud environment, credentials to your cloud environment, credentials to your cloud environment, to your Git environment. All of that to your Git environment. All of that to your Git environment. All of that sitting on the developer's environment. sitting on the developer's environment. sitting on the developer's environment. So look, I always say there attackers So look, I always say there attackers So look, I always say there attackers are lazy, right? They're going to find are lazy, right? They're going to find are lazy, right? They're going to find the easiest path with the fastest and the easiest path with the fastest and the easiest path with the fastest and the greatest blast radius in any the greatest blast radius in any the greatest blast radius in any company. company. company. >> Mhm. >> Mhm. >> Mhm. >> And today, that is the developer >> And today, that is the developer >> And today, that is the developer workstation. I just need a developer to workstation. I just need a developer to workstation. I just need a developer to do an NPM install or pip install, do an NPM install or pip install, do an NPM install or pip install, something that has malicious intent.

something that has malicious intent. something that has malicious intent. That malicious library That malicious library That malicious library most commonly most commonly most commonly >> [clears throat and cough] >> [clears throat and cough] >> [clears throat and cough] >> is now going to exfiltrate any variables >> is now going to exfiltrate any variables >> is now going to exfiltrate any variables and credentials and keys that it can and credentials and keys that it can and credentials and keys that it can find off the machine and it's out in 5 find off the machine and it's out in 5 find off the machine and it's out in 5 seconds. Like, there is no notion of, seconds. Like, there is no notion of, seconds. Like, there is no notion of, well, I have some checks baked into my well, I have some checks baked into my well, I have some checks baked into my pull request when it gets there, I'll pull request when it gets there, I'll pull request when it gets there, I'll catch it, so it won't go in my CICD. I catch it, so it won't go in my CICD. I catch it, so it won't go in my CICD. I got everything I need. It's got It's a got everything I need. It's got It's a got everything I need. It's got It's a smash and grab attempt on a developer smash and grab attempt on a developer smash and grab attempt on a developer workstation that is usually highly workstation that is usually highly workstation that is usually highly successful. And the implications of that successful. And the implications of that successful. And the implications of that is, if I get access to get or your is, if I get access to get or your is, if I get access to get or your company's source code, chances are that company's source code, chances are that company's source code, chances are that source code repository has hundreds of source code repository has hundreds of source code repository has hundreds of other keys embedded in there. That's the other keys embedded in there. That's the other keys embedded in there. That's the reality where we live, right? And so, reality where we live, right? And so, reality where we live, right? And so, you just get tremendous access doing so. you just get tremendous access doing so. you just get tremendous access doing so. Last thing I'll say Last thing I'll say Last thing I'll say is our research team, we have a Station is our research team, we have a Station is our research team, we have a Station 9 research team that spends a lot of 9 research team that spends a lot of 9 research team that spends a lot of time looking at novel attack vectors in time looking at novel attack vectors in time looking at novel attack vectors in the software supply chain. the software supply chain. the software supply chain. One of the more common growing ones One of the more common growing ones One of the more common growing ones is not even like somebody doing code is not even like somebody doing code is not even like somebody doing code obfuscation. It's somebody writing a obfuscation. It's somebody writing a obfuscation. It's somebody writing a skill. skill. skill. That skill looks like a very popular That skill looks like a very popular That skill looks like a very popular skill. It literally in simple English skill. It literally in simple English skill. It literally in simple English instructions instructions instructions states to the coding agent that it needs states to the coding agent that it needs states to the coding agent that it needs to export SSH keys or environment to export SSH keys or environment to export SSH keys or environment variables here and there.

variables here and there. variables here and there. And that's the extent of the attack And that's the extent of the attack And that's the extent of the attack vector, right? So, we live in a very vector, right? So, we live in a very vector, right? So, we live in a very complicated world right now with lots of complicated world right now with lots of complicated world right now with lots of different different different technology components that are moving technology components that are moving technology components that are moving that are outside of our control. And so, that are outside of our control. And so, that are outside of our control. And so, kind of talking about zero trust kind of talking about zero trust kind of talking about zero trust development is extremely important now. development is extremely important now. development is extremely important now. >> Yeah, I remember 20, 30 years ago we >> Yeah, I remember 20, 30 years ago we >> Yeah, I remember 20, 30 years ago we would call casual hackers script would call casual hackers script would call casual hackers script kiddies. But when you phrased it as kiddies. But when you phrased it as kiddies. But when you phrased it as smash and grab, it really kind of smash and grab, it really kind of smash and grab, it really kind of underscores the lack of sophistication underscores the lack of sophistication underscores the lack of sophistication and the fact that there doesn't need to and the fact that there doesn't need to and the fact that there doesn't need to be any sophistication. All they need is be any sophistication. All they need is be any sophistication. All they need is one developer to accidentally have a one developer to accidentally have a one developer to accidentally have a production, you know, S3 bucket or Azure production, you know, S3 bucket or Azure production, you know, S3 bucket or Azure bucket and and in an environment bucket and and in an environment bucket and and in an environment variable and they run in, they smash a variable and they run in, they smash a variable and they run in, they smash a bunch of stuff, they grab and maybe they bunch of stuff, they grab and maybe they bunch of stuff, they grab and maybe they get the Hope Diamond, maybe they don't, get the Hope Diamond, maybe they don't, get the Hope Diamond, maybe they don't, but then they can do it with a bot but then they can do it with a bot but then they can do it with a bot swarm, they can do it N number of times. swarm, they can do it N number of times. swarm, they can do it N number of times. There's no need to for sophistication There's no need to for sophistication There's no need to for sophistication and it only takes once. They only have and it only takes once. They only have and it only takes once. They only have to be right to be right to be right one time. one time. one time. >> Yeah. >> Yeah. >> Yeah. I I I think that's very true. By the I I I think that's very true. By the I I I think that's very true. By the way, the way, the way, the So, we're talking obviously of this So, we're talking obviously of this So, we're talking obviously of this dynamics of malware. The the other thing dynamics of malware. The the other thing dynamics of malware. The the other thing is is is the attackers are way faster and more the attackers are way faster and more the attackers are way faster and more sophisticated to take unconstrained use sophisticated to take unconstrained use sophisticated to take unconstrained use of these new technologies and LLMs of these new technologies and LLMs of these new technologies and LLMs to to attack us. While this an to to attack us. While this an to to attack us. While this an organization is going through their AI organization is going through their AI organization is going through their AI council to get approval to use the council to get approval to use the council to get approval to use the latest model to do A, B, and C and then latest model to do A, B, and C and then latest model to do A, B, and C and then find the right evaluation and run of P find the right evaluation and run of P find the right evaluation and run of P processes, processes, processes, the attackers are on it. It's called the the attackers are on it. It's called the the attackers are on it. It's called the average time for an attacker to build an average time for an attacker to build an average time for an attacker to build an exploit for a CVE,

exploit for a CVE, exploit for a CVE, a actual known vulnerability, has gone a actual known vulnerability, has gone a actual known vulnerability, has gone from what used to be 2 years from what used to be 2 years from what used to be 2 years uh in 2008 uh in 2008 uh in 2008 to 23 days last year to 10 hours this to 23 days last year to 10 hours this to 23 days last year to 10 hours this year. year. year. So, then you got to kind of think about So, then you got to kind of think about So, then you got to kind of think about this and say, like which organization is this and say, like which organization is this and say, like which organization is prepared today from a software prepared today from a software prepared today from a software development life cycle perspective to development life cycle perspective to development life cycle perspective to discover a new vulnerability, triage it, discover a new vulnerability, triage it, discover a new vulnerability, triage it, create a fix for it, test it, and ship create a fix for it, test it, and ship create a fix for it, test it, and ship it in production in under 10 hours. But, it in production in under 10 hours. But, it in production in under 10 hours. But, that's what we're dealing with. that's what we're dealing with. that's what we're dealing with. >> Okay. And see, and this underscores >> Okay. And see, and this underscores >> Okay. And see, and this underscores another thing that I want to talk about, another thing that I want to talk about, another thing that I want to talk about, which is that it's for me, and maybe which is that it's for me, and maybe which is that it's for me, and maybe this is ignorance, but it's I feel like this is ignorance, but it's I feel like this is ignorance, but it's I feel like it's less about AI and it's more about it's less about AI and it's more about it's less about AI and it's more about having a healthy SDLC. If your company having a healthy SDLC. If your company having a healthy SDLC. If your company is not doing continuous integration, is not doing continuous integration, is not doing continuous integration, continuous deployment, and all this continuous deployment, and all this continuous deployment, and all this stuff we told them they should have been stuff we told them they should have been stuff we told them they should have been doing 15 years ago. doing 15 years ago. doing 15 years ago. You are in big trouble. Would you agree? You are in big trouble. Would you agree? You are in big trouble. Would you agree? >> Yes, that plus >> Yes, that plus >> Yes, that plus if you really didn't think about if you really didn't think about if you really didn't think about shifting security left, I have shifting security left, I have shifting security left, I have fundamentally believe software security fundamentally believe software security fundamentally believe software security is an engineering problem. And I think is an engineering problem. And I think is an engineering problem. And I think the security team can certainly help you the security team can certainly help you the security team can certainly help you with the verification loop, but a lot of with the verification loop, but a lot of with the verification loop, but a lot of security teams that have operated in security teams that have operated in security teams that have operated in this construct of well, we run these this construct of well, we run these this construct of well, we run these scans on release candidates and then scans on release candidates and then scans on release candidates and then we'll go, you know, give a report of we'll go, you know, give a report of we'll go, you know, give a report of Jira tickets and then somebody's going Jira tickets and then somebody's going Jira tickets and then somebody's going to go fix them. Doesn't work, right?

to go fix them. Doesn't work, right? to go fix them. Doesn't work, right? This has to be in the in the coding loop This has to be in the in the coding loop This has to be in the in the coding loop now, right? As your agents are producing now, right? As your agents are producing now, right? As your agents are producing code before they're considered done, code before they're considered done, code before they're considered done, there needs to be a set of security there needs to be a set of security there needs to be a set of security checks run on that code output because checks run on that code output because checks run on that code output because we know agent code or human code is we know agent code or human code is we know agent code or human code is going to have problems. going to have problems. going to have problems. What if you can just fix those problems What if you can just fix those problems What if you can just fix those problems pre-PR? pre-PR? pre-PR? Get them out of the way autonomously. Get them out of the way autonomously. Get them out of the way autonomously. And then when you get to a pull request, And then when you get to a pull request, And then when you get to a pull request, you kind of do the more thorough you kind of do the more thorough you kind of do the more thorough detailed review and analysis, give detailed review and analysis, give detailed review and analysis, give feedback, fix. So, like you keep weeding feedback, fix. So, like you keep weeding feedback, fix. So, like you keep weeding off the simpler things earlier and off the simpler things earlier and off the simpler things earlier and earlier in the process such that by the earlier in the process such that by the earlier in the process such that by the time you have a release artifact you time you have a release artifact you time you have a release artifact you have a high degree of certainty that have a high degree of certainty that have a high degree of certainty that this is not going to have any surprise this is not going to have any surprise this is not going to have any surprise problems in it. Is the only way to go. problems in it. Is the only way to go. problems in it. Is the only way to go. Just like we moved QA to QE, right? It's Just like we moved QA to QE, right? It's Just like we moved QA to QE, right? It's now every developer is expected to have now every developer is expected to have now every developer is expected to have working code, not just code and working working code, not just code and working working code, not just code and working features. You now need to have secure features. You now need to have secure features. You now need to have secure and working features integrated. and working features integrated. and working features integrated. >> Let me let me challenge you on something >> Let me let me challenge you on something >> Let me let me challenge you on something now because if we remember that the now because if we remember that the now because if we remember that the corpus on which these things are trained corpus on which these things are trained corpus on which these things are trained is kind of all visible open source code is kind of all visible open source code is kind of all visible open source code and we make the assumption that in a and we make the assumption that in a and we make the assumption that in a normal distribution curve the mediocrity normal distribution curve the mediocrity normal distribution curve the mediocrity is where the gravity well is. So, these is where the gravity well is. So, these is where the gravity well is. So, these security these these coding models security these these coding models security these these coding models rather are going to generate average rather are going to generate average rather are going to generate average code unless someone steers it to be code unless someone steers it to be code unless someone steers it to be better than average.

better than average. better than average. And as such, average code is insecure. And as such, average code is insecure. And as such, average code is insecure. So then, to your own point, you've So then, to your own point, you've So then, to your own point, you've you've said that 60% of AI-generated you've said that 60% of AI-generated you've said that 60% of AI-generated code is insecure by default. That puts code is insecure by default. That puts code is insecure by default. That puts you right in the fat part of the bell you right in the fat part of the bell you right in the fat part of the bell curve. curve. curve. Why should we trust models to be able to Why should we trust models to be able to Why should we trust models to be able to detect security issues detect security issues detect security issues if models by themselves are the problem if models by themselves are the problem if models by themselves are the problem that cause these security issues? that cause these security issues? that cause these security issues? >> I great question. Oh my god. One of one >> I great question. Oh my god. One of one >> I great question. Oh my god. One of one one of my favorite topics. So, we're one of my favorite topics. So, we're one of my favorite topics. So, we're going to riff on this for a little bit. going to riff on this for a little bit. going to riff on this for a little bit. One, by the way, it's not just 60% One, by the way, it's not just 60% One, by the way, it's not just 60% insecure. The data shows against you insecure. The data shows against you insecure. The data shows against you know, the benchmarks we run testing on know, the benchmarks we run testing on know, the benchmarks we run testing on even the most sophisticated models 70 to even the most sophisticated models 70 to even the most sophisticated models 70 to 80% of the code being recommended by 80% of the code being recommended by 80% of the code being recommended by them is insecure by default. Uh lots of them is insecure by default. Uh lots of them is insecure by default. Uh lots of reasons. We can get into that if you're reasons. We can get into that if you're reasons. We can get into that if you're interested. interested. interested. Okay, now the next question is can you Okay, now the next question is can you Okay, now the next question is can you trust the same models that cannot trust the same models that cannot trust the same models that cannot produce secure code produce secure code produce secure code to find stuff? Like look, I think the to find stuff? Like look, I think the to find stuff? Like look, I think the defensive security by default versus defensive security by default versus defensive security by default versus offensive security problems are slightly offensive security problems are slightly offensive security problems are slightly different. So, yes, like they find different. So, yes, like they find different. So, yes, like they find stuff, right? They are very good at stuff, right? They are very good at stuff, right? They are very good at finding stuff. But are they finding the finding stuff. But are they finding the finding stuff. But are they finding the right things in the context of the right things in the context of the right things in the context of the application, the way it works for you, application, the way it works for you, application, the way it works for you, and at what price point and at what price point and at what price point are the questions? Like I have you know, are the questions? Like I have you know, are the questions? Like I have you know, there's a company that I know that has there's a company that I know that has there's a company that I know that has access to Mythos. They ran Mythos on 32 access to Mythos. They ran Mythos on 32 access to Mythos. They ran Mythos on 32 repos for about a month and the bill was repos for about a month and the bill was repos for about a month and the bill was a million dollars.

a million dollars. a million dollars. >> Oh lord. >> Oh lord. >> Oh lord. >> Is it sustainable? No. You run some of >> Is it sustainable? No. You run some of >> Is it sustainable? No. You run some of those on PR checks. Sometimes it'll take those on PR checks. Sometimes it'll take those on PR checks. Sometimes it'll take 20 to 30 minutes to run a completely 20 to 30 minutes to run a completely 20 to 30 minutes to run a completely non-deterministic scan on a PR check. non-deterministic scan on a PR check. non-deterministic scan on a PR check. So, like the token cost, the speed So, like the token cost, the speed So, like the token cost, the speed it's untenable, right? There's a lot of it's untenable, right? There's a lot of it's untenable, right? There's a lot of value in these models, but I think you value in these models, but I think you value in these models, but I think you have to really solve the core problem, have to really solve the core problem, have to really solve the core problem, which is the context problem. As you which is the context problem. As you which is the context problem. As you know, Scott, these know, Scott, these know, Scott, these uh AI coding agents and just LLMs in uh AI coding agents and just LLMs in uh AI coding agents and just LLMs in general, when you interact with large general, when you interact with large general, when you interact with large code bases, they're super greppers. They code bases, they're super greppers. They code bases, they're super greppers. They can grep across your code bases, but can grep across your code bases, but can grep across your code bases, but they assume that assumes by nature that they assume that assumes by nature that they assume that assumes by nature that it's a flat that an application is flat. it's a flat that an application is flat. it's a flat that an application is flat. But an application is a complex But an application is a complex But an application is a complex graph of first-party code that connects graph of first-party code that connects graph of first-party code that connects to a library that connects to 77 other to a library that connects to 77 other to a library that connects to 77 other libraries. libraries. libraries. If we can extract that context If we can extract that context If we can extract that context with static analysis and program with static analysis and program with static analysis and program analysis that by the way, you know, we analysis that by the way, you know, we analysis that by the way, you know, we have a lot of expertise at Endor Labs have a lot of expertise at Endor Labs have a lot of expertise at Endor Labs and that's what we do. and that's what we do. and that's what we do. What we have seen is when we extract What we have seen is when we extract What we have seen is when we extract that context and the heat map that context and the heat map that context and the heat map and we give that to the LLMs, this and we give that to the LLMs, this and we give that to the LLMs, this multimodal approach of multimodal approach of multimodal approach of static analysis plus LLMs is very static analysis plus LLMs is very static analysis plus LLMs is very powerful at a very cost-effective powerful at a very cost-effective powerful at a very cost-effective way and at a speed where you can way and at a speed where you can way and at a speed where you can actually do this on average for an in 64 actually do this on average for an in 64 actually do this on average for an in 64 seconds per PR versus 20 minutes per PR.

seconds per PR versus 20 minutes per PR. seconds per PR versus 20 minutes per PR. And so you now actually can run this in And so you now actually can run this in And so you now actually can run this in any developer workflow. Uh so my my meta any developer workflow. Uh so my my meta any developer workflow. Uh so my my meta point is the models themselves don't point is the models themselves don't point is the models themselves don't have the intelligence necessarily to have the intelligence necessarily to have the intelligence necessarily to create secure code, but companies that create secure code, but companies that create secure code, but companies that specialize in that area specialize in that area specialize in that area like Endor, we have created years and like Endor, we have created years and like Endor, we have created years and years of custom proprietary data sets years of custom proprietary data sets years of custom proprietary data sets that are annotating all of that security that are annotating all of that security that are annotating all of that security intelligence on that trained data set so intelligence on that trained data set so intelligence on that trained data set so we can help. And that's where we can help. And that's where we can help. And that's where traditionally you'll always find even traditionally you'll always find even traditionally you'll always find even with cloud, with code, with network, with cloud, with code, with network, with cloud, with code, with network, with endpoint, security tends to be an with endpoint, security tends to be an with endpoint, security tends to be an independent function with independent independent function with independent independent function with independent set of expertise and tooling. set of expertise and tooling. set of expertise and tooling. Now that doesn't mean it can't be Now that doesn't mean it can't be Now that doesn't mean it can't be integrated in the SDLC, it absolutely integrated in the SDLC, it absolutely integrated in the SDLC, it absolutely needs to be. needs to be. needs to be. >> See, this is this is good this is good >> See, this is this is good this is good >> See, this is this is good this is good stuff. This is why I think Endor is stuff. This is why I think Endor is stuff. This is why I think Endor is interesting because I don't like when I interesting because I don't like when I interesting because I don't like when I hear an AI company say "You have a hear an AI company say "You have a hear an AI company say "You have a problem because your AI makes bad code problem because your AI makes bad code problem because your AI makes bad code and we have the solution. It's another and we have the solution. It's another and we have the solution. It's another AI." You know, like you're acknowledging AI." You know, like you're acknowledging AI." You know, like you're acknowledging that static analysis, proprietary that static analysis, proprietary that static analysis, proprietary models, not just LLMs, not just a models, not just LLMs, not just a models, not just LLMs, not just a million monkeys with a million million monkeys with a million million monkeys with a million typewriters slapping on the keyboard and typewriters slapping on the keyboard and typewriters slapping on the keyboard and hoping for the best. You're calling out hoping for the best. You're calling out hoping for the best. You're calling out that there is formal correctness and you that there is formal correctness and you that there is formal correctness and you are recognizing that those correctness are recognizing that those correctness are recognizing that those correctness checkers checkers checkers >> Yes. not just throwing one LLM after >> Yes. not just throwing one LLM after >> Yes. not just throwing one LLM after another and burning tokens because we another and burning tokens because we another and burning tokens because we all know token maxing is nonsense.

all know token maxing is nonsense. all know token maxing is nonsense. >> Correct. The >> Correct. The >> Correct. The cybersecurity cybersecurity cybersecurity into a great degree is a into a great degree is a into a great degree is a requires a determinant some high degree requires a determinant some high degree requires a determinant some high degree of determinism of determinism of determinism >> Yes. >> Yes. >> Yes. >> in the solution. >> in the solution. >> in the solution. >> Agreed. >> Agreed. >> Agreed. >> By nature, LLM needed just LLMs cannot >> By nature, LLM needed just LLMs cannot >> By nature, LLM needed just LLMs cannot provide that. So, you got to have this provide that. So, you got to have this provide that. So, you got to have this fusion fusion fusion >> Right. Hybrid model. >> Right. Hybrid model. >> Right. Hybrid model. >> a hybrid capability where I always say >> a hybrid capability where I always say >> a hybrid capability where I always say this, if you can do 2 + 2 on a this, if you can do 2 + 2 on a this, if you can do 2 + 2 on a calculator, you don't need an LLM for calculator, you don't need an LLM for calculator, you don't need an LLM for that. So, like you got to pick the that. So, like you got to pick the that. So, like you got to pick the things that make sense to do with static things that make sense to do with static things that make sense to do with static analysis. At Endor, we have 14 PhDs that analysis. At Endor, we have 14 PhDs that analysis. At Endor, we have 14 PhDs that are world-renowned experts in static are world-renowned experts in static are world-renowned experts in static analysis. Like that's all they do is analysis. Like that's all they do is analysis. Like that's all they do is creating tools in context that we can creating tools in context that we can creating tools in context that we can then feed the LLMs to be 10 times more then feed the LLMs to be 10 times more then feed the LLMs to be 10 times more powerful at 1/10 the cost. powerful at 1/10 the cost. powerful at 1/10 the cost. >> Right. And it's And I respect also that >> Right. And it's And I respect also that >> Right. And it's And I respect also that you're calling out the cost because you're calling out the cost because you're calling out the cost because this token maxing thing is out of this token maxing thing is out of this token maxing thing is out of control. People are using the biggest control. People are using the biggest control. People are using the biggest models, like you said, run it for a models, like you said, run it for a models, like you said, run it for a month, it costs a million dollars. God month, it costs a million dollars. God month, it costs a million dollars. God knows what it does to the ecosystem and knows what it does to the ecosystem and knows what it does to the ecosystem and to the ecology. Who knows how much water to the ecology. Who knows how much water to the ecology. Who knows how much water got burned because they ran, you know, got burned because they ran, you know, got burned because they ran, you know, Fable for that long.

Fable for that long. Fable for that long. When the problems can be solved by to When the problems can be solved by to When the problems can be solved by to your point PhDs, static analysis. your point PhDs, static analysis. your point PhDs, static analysis. Uh I When I think about like the static Uh I When I think about like the static Uh I When I think about like the static When I think about the software When I think about the software When I think about the software development life cycle and we mentioned development life cycle and we mentioned development life cycle and we mentioned like CICDs and we know that in 2026 like CICDs and we know that in 2026 like CICDs and we know that in 2026 there are companies that do not even there are companies that do not even there are companies that do not even have a mature DevOps pipeline. have a mature DevOps pipeline. have a mature DevOps pipeline. The number of folks that don't know what The number of folks that don't know what The number of folks that don't know what like cyclomatic complexity is, like cyclomatic complexity is, like cyclomatic complexity is, that don't run any static analysis on that don't run any static analysis on that don't run any static analysis on their code at all to look at the their code at all to look at the their code at all to look at the complexity of the code or the security complexity of the code or the security complexity of the code or the security issues around the code issues around the code issues around the code is a little scary. Like we were talking is a little scary. Like we were talking is a little scary. Like we were talking about this in the late '90s. How often about this in the late '90s. How often about this in the late '90s. How often are you finding when you walk into a are you finding when you walk into a are you finding when you walk into a customer with with Endor Labs products customer with with Endor Labs products customer with with Endor Labs products and they're thinking that this is like and they're thinking that this is like and they're thinking that this is like this is magic. And you're like, no, it's this is magic. And you're like, no, it's this is magic. And you're like, no, it's actually actually actually science. science. science. >> Yeah. >> Yeah. >> Yeah. You know, Scott, unfortunately, You know, Scott, unfortunately, You know, Scott, unfortunately, we come from a world where security was we come from a world where security was we come from a world where security was mostly built around compliance. Like mostly built around compliance. Like mostly built around compliance. Like even this problem with open source, if even this problem with open source, if even this problem with open source, if you think about it 20 years ago, this you think about it 20 years ago, this you think about it 20 years ago, this whole ingestion pipeline, if you will, whole ingestion pipeline, if you will, whole ingestion pipeline, if you will, or verification, or OS like the open or verification, or OS like the open or verification, or OS like the open source program offices were built around source program offices were built around source program offices were built around license compliance originally.

license compliance originally. license compliance originally. And then they evolved to take on some of And then they evolved to take on some of And then they evolved to take on some of the vulnerability management aspects. the vulnerability management aspects. the vulnerability management aspects. And today they're evolving to kind of And today they're evolving to kind of And today they're evolving to kind of solve for supply chain security aspects. solve for supply chain security aspects. solve for supply chain security aspects. It's Look, there are a lot continuous It's Look, there are a lot continuous It's Look, there are a lot continuous change. The The thing I love about being change. The The thing I love about being change. The The thing I love about being in the cybersecurity industry is it's a in the cybersecurity industry is it's a in the cybersecurity industry is it's a cat and mouse game, right? One door cat and mouse game, right? One door cat and mouse game, right? One door shuts, another opens, and you're always shuts, another opens, and you're always shuts, another opens, and you're always trying to trying to trying to out-innovate. out-innovate. out-innovate. And And And yeah, traditionally, we didn't write yeah, traditionally, we didn't write yeah, traditionally, we didn't write this much code. Traditionally, we didn't this much code. Traditionally, we didn't this much code. Traditionally, we didn't use this much open source, right? Our use this much open source, right? Our use this much open source, right? Our reliance on open source was what? Maybe reliance on open source was what? Maybe reliance on open source was what? Maybe 20, 30% of a legacy application was open 20, 30% of a legacy application was open 20, 30% of a legacy application was open source. Today, it's 80 or 90%. I would source. Today, it's 80 or 90%. I would source. Today, it's 80 or 90%. I would argue 99% argue 99% argue 99% 99.9% of code in a modern application 99.9% of code in a modern application 99.9% of code in a modern application won't be written by a developer. And so won't be written by a developer. And so won't be written by a developer. And so like things are shifting so rapidly, like things are shifting so rapidly, like things are shifting so rapidly, like, you know, skills didn't exist last like, you know, skills didn't exist last like, you know, skills didn't exist last year. MCP was hot, and then something year. MCP was hot, and then something year. MCP was hot, and then something else comes around, and now CLIs are else comes around, and now CLIs are else comes around, and now CLIs are great again. Like the the beautiful great again. Like the the beautiful great again. Like the the beautiful thing about being in software thing about being in software thing about being in software development is it is constantly changing development is it is constantly changing development is it is constantly changing for the better. But that puts a moving for the better. But that puts a moving for the better. But that puts a moving target to say like, okay, how you target to say like, okay, how you target to say like, okay, how you innovate? You have to kind of make some innovate? You have to kind of make some innovate? You have to kind of make some grounded assumptions, and that grounded assumptions, and that grounded assumptions, and that assumption we made is one assumption we made is one assumption we made is one more code, not less code, will be more code, not less code, will be more code, not less code, will be out kind of from the outside of your out kind of from the outside of your out kind of from the outside of your trust boundary.

trust boundary. trust boundary. The second was the you know, the The second was the you know, the The second was the you know, the convolution, like code is a graph. You convolution, like code is a graph. You convolution, like code is a graph. You have to solve this as a problem a graph have to solve this as a problem a graph have to solve this as a problem a graph problem. problem. problem. Right? Like trying to scan manifest Right? Like trying to scan manifest Right? Like trying to scan manifest files and say, oh, I see this import, so files and say, oh, I see this import, so files and say, oh, I see this import, so must be there. Doesn't work in modern must be there. Doesn't work in modern must be there. Doesn't work in modern application architectures. application architectures. application architectures. >> Yeah. Well, in modern applications and >> Yeah. Well, in modern applications and >> Yeah. Well, in modern applications and modern development now is not just I modern development now is not just I modern development now is not just I write some code and I check it in. My write some code and I check it in. My write some code and I check it in. My coding agent or myself I'll have GitHub coding agent or myself I'll have GitHub coding agent or myself I'll have GitHub right access. I'll have package manager right access. I'll have package manager right access. I'll have package manager credentials. I'll have MCP tools. I'll credentials. I'll have MCP tools. I'll credentials. I'll have MCP tools. I'll have cloud credentials. I'll have CI have cloud credentials. I'll have CI have cloud credentials. I'll have CI access. access. access. Everyone has way more access than they Everyone has way more access than they Everyone has way more access than they need to, which brings me to my next need to, which brings me to my next need to, which brings me to my next question. Which is they expect us to go question. Which is they expect us to go question. Which is they expect us to go faster than ever. faster than ever. faster than ever. But doesn't this just mean they're just But doesn't this just mean they're just But doesn't this just mean they're just going to lock our developer workstations going to lock our developer workstations going to lock our developer workstations down and make us all just RDP into the down and make us all just RDP into the down and make us all just RDP into the cloud somewhere? Like I want to move cloud somewhere? Like I want to move cloud somewhere? Like I want to move fast and I don't want defender or fast and I don't want defender or fast and I don't want defender or whatever whatever whatever agent thing that's running on my machine agent thing that's running on my machine agent thing that's running on my machine to slow me down. How do you find that to slow me down. How do you find that to slow me down. How do you find that balance between the only secure machine balance between the only secure machine balance between the only secure machine is one that's not plugged into the is one that's not plugged into the is one that's not plugged into the internet?

internet? internet? You know, and the speed and the You know, and the speed and the You know, and the speed and the expectation that they they want us to go expectation that they they want us to go expectation that they they want us to go at. at. at. >> Yeah. >> Yeah. >> Yeah. It's a great question. It's there is no It's a great question. It's there is no It's a great question. It's there is no perfect solution, right? Like perfect solution, right? Like perfect solution, right? Like the the history security always comes at the the history security always comes at the the history security always comes at a cost. The question is is that a a cost. The question is is that a a cost. The question is is that a tolerable cost or not? I think tolerable cost or not? I think tolerable cost or not? I think traditionally when you used to build traditionally when you used to build traditionally when you used to build applications and then you would receive applications and then you would receive applications and then you would receive an 800-page web inspect report of an 800-page web inspect report of an 800-page web inspect report of vulnerabilities that would take you vulnerabilities that would take you vulnerabilities that would take you another 6 months to fix before you could another 6 months to fix before you could another 6 months to fix before you could release. Like we don't have, you know, release. Like we don't have, you know, release. Like we don't have, you know, the ability to tolerate that today. Like the ability to tolerate that today. Like the ability to tolerate that today. Like do we have ability to tolerate 5 million do we have ability to tolerate 5 million do we have ability to tolerate 5 million findings from our platform every 6 findings from our platform every 6 findings from our platform every 6 months? No. months? No. months? No. But But But I think as long as we are reasonable to I think as long as we are reasonable to I think as long as we are reasonable to know today you cannot live with security know today you cannot live with security know today you cannot live with security through obscurity. What I mean by that? through obscurity. What I mean by that? through obscurity. What I mean by that? We would make a series of compromises to We would make a series of compromises to We would make a series of compromises to ship fast, which would mean security ship fast, which would mean security ship fast, which would mean security would not be taken seriously in would not be taken seriously in would not be taken seriously in software. software. software. And you would hope that nobody would And you would hope that nobody would And you would hope that nobody would find those vulnerabilities. Today with find those vulnerabilities. Today with find those vulnerabilities. Today with model like mythos and fable and GPT-55 model like mythos and fable and GPT-55 model like mythos and fable and GPT-55 every vulnerability that is in existence every vulnerability that is in existence every vulnerability that is in existence in your software will be found. So the in your software will be found. So the in your software will be found. So the question becomes question becomes question becomes you can't make those trade-off you can't make those trade-off you can't make those trade-off decisions. So what you're now trying to decisions. So what you're now trying to decisions. So what you're now trying to do is what is the lowest cost at which I do is what is the lowest cost at which I do is what is the lowest cost at which I can inject security can inject security can inject security into my development workflow cuz I can into my development workflow cuz I can into my development workflow cuz I can no longer accept the risk and ship it.

no longer accept the risk and ship it. no longer accept the risk and ship it. >> Right. >> Right. >> Right. >> And what is that low toleration of cost? >> And what is that low toleration of cost? >> And what is that low toleration of cost? The more left in the process, The more left in the process, The more left in the process, >> Mhm. >> Mhm. >> Mhm. >> the cheaper it gets. It's like It's like >> the cheaper it gets. It's like It's like >> the cheaper it gets. It's like It's like car manufacturing. If you try to issue a car manufacturing. If you try to issue a car manufacturing. If you try to issue a recall once you've sold the 50,000 cars recall once you've sold the 50,000 cars recall once you've sold the 50,000 cars to your to your to your like customers, it's going to be like customers, it's going to be like customers, it's going to be expensive. If you had quality controls expensive. If you had quality controls expensive. If you had quality controls in the assembly line and you were just in the assembly line and you were just in the assembly line and you were just fixing those things well before that car fixing those things well before that car fixing those things well before that car was out to a showroom or a client's was out to a showroom or a client's was out to a showroom or a client's hands, way cheaper. So, in software hands, way cheaper. So, in software hands, way cheaper. So, in software factories, the cheapest place to insert factories, the cheapest place to insert factories, the cheapest place to insert security security security and in the loop is in the agentic and in the loop is in the agentic and in the loop is in the agentic workflow. Developer gives a spec, agent workflow. Developer gives a spec, agent workflow. Developer gives a spec, agent builds. Okay, as the agent is building, builds. Okay, as the agent is building, builds. Okay, as the agent is building, we know they don't have the intelligence we know they don't have the intelligence we know they don't have the intelligence of being very secure, but if we fuse and of being very secure, but if we fuse and of being very secure, but if we fuse and insert the Andor intelligence, for insert the Andor intelligence, for insert the Andor intelligence, for example, in there, the code before it's example, in there, the code before it's example, in there, the code before it's complete, the plan being complete, you complete, the plan being complete, you complete, the plan being complete, you would have more secure code. Great. would have more secure code. Great. would have more secure code. Great. >> Okay. >> Okay. >> Okay. >> Then it goes to PR and so on and so you >> Then it goes to PR and so on and so you >> Then it goes to PR and so on and so you kind of keep reducing the security kind of keep reducing the security kind of keep reducing the security issues till you get to a zero point issues till you get to a zero point issues till you get to a zero point uh you know, on the other side.

uh you know, on the other side. uh you know, on the other side. >> Okay, so you beat me to it because that >> Okay, so you beat me to it because that >> Okay, so you beat me to it because that was my next question cuz I was trying to was my next question cuz I was trying to was my next question cuz I was trying to visualize. You keep saying shift left, visualize. You keep saying shift left, visualize. You keep saying shift left, how far left do you go? And Andor AI, is how far left do you go? And Andor AI, is how far left do you go? And Andor AI, is that pronounced correctly? A A U R I, that pronounced correctly? A A U R I, that pronounced correctly? A A U R I, the security the security the security >> AURI. >> AURI. >> AURI. >> AURI. So, Andor Labs AURI is the >> AURI. So, Andor Labs AURI is the >> AURI. So, Andor Labs AURI is the security intelligence layer for coding security intelligence layer for coding security intelligence layer for coding agents, right? So, we've got the agents, right? So, we've got the agents, right? So, we've got the deterministic aspect, we've got the deterministic aspect, we've got the deterministic aspect, we've got the probabilistic model reasoning, and then probabilistic model reasoning, and then probabilistic model reasoning, and then you've got you know, parts you can prove you've got you know, parts you can prove you've got you know, parts you can prove and parts that are that are special and parts that are that are special and parts that are that are special herbs and spices, Andor Labs specific herbs and spices, Andor Labs specific herbs and spices, Andor Labs specific herbs and spices. But this should be herbs and spices. But this should be herbs and spices. But this should be Should the secure coding agent be in the Should the secure coding agent be in the Should the secure coding agent be in the generation loop preventing the mistake generation loop preventing the mistake generation loop preventing the mistake so it never escapes is what you're so it never escapes is what you're so it never escapes is what you're saying. saying. saying. >> Do Yes, the worst thing we can do in >> Do Yes, the worst thing we can do in >> Do Yes, the worst thing we can do in this agentic SDLC when we're trying to this agentic SDLC when we're trying to this agentic SDLC when we're trying to ramp up code generation is ramp up the ramp up code generation is ramp up the ramp up code generation is ramp up the number of security tickets we push to number of security tickets we push to number of security tickets we push to humans. If we humans. If we humans. If we Agents create the code and make all of Agents create the code and make all of Agents create the code and make all of those things post generation, review, those things post generation, review, those things post generation, review, find the problem, put it as tickets in find the problem, put it as tickets in find the problem, put it as tickets in the backlog human, we did not solve for the backlog human, we did not solve for the backlog human, we did not solve for anything in terms of innovation or anything in terms of innovation or anything in terms of innovation or acceleration of software development.

acceleration of software development. acceleration of software development. The only way to do this is the code The only way to do this is the code The only way to do this is the code being created by the agents. being created by the agents. being created by the agents. Those agents have to be held accountable Those agents have to be held accountable Those agents have to be held accountable to improve the security efficacy. How do to improve the security efficacy. How do to improve the security efficacy. How do you do that? Agents are smart, they you do that? Agents are smart, they you do that? Agents are smart, they follow instructions, they follow tools. follow instructions, they follow tools. follow instructions, they follow tools. You give them the tools, you give them You give them the tools, you give them You give them the tools, you give them the instructions, you tell them what to the instructions, you tell them what to the instructions, you tell them what to go fix. Um, which, you know, is the is go fix. Um, which, you know, is the is go fix. Um, which, you know, is the is the is a key part of the Endor value the is a key part of the Endor value the is a key part of the Endor value proposition. We didn't talk about this, proposition. We didn't talk about this, proposition. We didn't talk about this, you talked about the static analysis and you talked about the static analysis and you talked about the static analysis and the tools and the herbs and spices. the tools and the herbs and spices. the tools and the herbs and spices. One of the most important herbs and One of the most important herbs and One of the most important herbs and spice is spice is spice is for all of this code on open source that for all of this code on open source that for all of this code on open source that these models are trained on, which they these models are trained on, which they these models are trained on, which they don't know what is secure or insecure, don't know what is secure or insecure, don't know what is secure or insecure, Endor does. We have been labeling Endor does. We have been labeling Endor does. We have been labeling hundreds of thousands of vulnerabilities hundreds of thousands of vulnerabilities hundreds of thousands of vulnerabilities that have been created over the last 20 that have been created over the last 20 that have been created over the last 20 years to understand which lines of code years to understand which lines of code years to understand which lines of code in open source have them, which lines of in open source have them, which lines of in open source have them, which lines of code and which future releases fix them, code and which future releases fix them, code and which future releases fix them, etc., etc. So, when we fuse that etc., etc. So, when we fuse that etc., etc. So, when we fuse that intelligence to a model that is intelligence to a model that is intelligence to a model that is recommending some code that has a recommending some code that has a recommending some code that has a problem, we can help it rewrite it problem, we can help it rewrite it problem, we can help it rewrite it correctly. correctly. correctly. >> Right. This is where things get >> Right. This is where things get >> Right. This is where things get interesting. So, we had mentioned a interesting. So, we had mentioned a interesting. So, we had mentioned a couple of minutes ago, or maybe 10 15 couple of minutes ago, or maybe 10 15 couple of minutes ago, or maybe 10 15 minutes ago, about how if you train an minutes ago, about how if you train an minutes ago, about how if you train an AI like the Frontier models on AI like the Frontier models on AI like the Frontier models on everything, you get mediocrity. But, if everything, you get mediocrity. But, if everything, you get mediocrity. But, if you're doing targeted training on CVEs, you're doing targeted training on CVEs, you're doing targeted training on CVEs, you are doing a focused train on you are doing a focused train on you are doing a focused train on specific models and saying, "This is specific models and saying, "This is specific models and saying, "This is what is wrong, and we know how these what is wrong, and we know how these what is wrong, and we know how these these these CVEs have been fixed over 20 these these CVEs have been fixed over 20 these these CVEs have been fixed over 20 years."

years." years." You know so much more. This is a You know so much more. This is a You know so much more. This is a a very laser-focused kind of model and a very laser-focused kind of model and a very laser-focused kind of model and not a general-purpose not a general-purpose not a general-purpose >> Correct. >> Correct. >> Correct. >> kind of model. >> kind of model. >> kind of model. Yes. Interesting. Okay. So, then, what Yes. Interesting. Okay. So, then, what Yes. Interesting. Okay. So, then, what do you think the next big attack is? Is do you think the next big attack is? Is do you think the next big attack is? Is it going to be supply chain, like we're it going to be supply chain, like we're it going to be supply chain, like we're seeing to target source code? Is it seeing to target source code? Is it seeing to target source code? Is it going to be someone going to be someone going to be someone attacking the the loop itself? Or is it attacking the the loop itself? Or is it attacking the the loop itself? Or is it going to be script kiddies making going to be script kiddies making going to be script kiddies making skills? Where where should we be skills? Where where should we be skills? Where where should we be training our attention? training our attention? training our attention? >> So, I think there's two sets of >> So, I think there's two sets of >> So, I think there's two sets of problems. I think one as I said, every problems. I think one as I said, every problems. I think one as I said, every security weakness in your software will security weakness in your software will security weakness in your software will be found. It's not a question of if, be found. It's not a question of if, be found. It's not a question of if, it's a question of how fast. And so, I it's a question of how fast. And so, I it's a question of how fast. And so, I think we're playing this cat and mouse think we're playing this cat and mouse think we're playing this cat and mouse game to go fix 20 years of software game to go fix 20 years of software game to go fix 20 years of software vulnerabilities vulnerabilities vulnerabilities in the next 6 to 12 18 months. It's in the next 6 to 12 18 months. It's in the next 6 to 12 18 months. It's going to be extremely painful. There's a going to be extremely painful. There's a going to be extremely painful. There's a lot of security debt that we all have to lot of security debt that we all have to lot of security debt that we all have to clear up. And you know, we're going to clear up. And you know, we're going to clear up. And you know, we're going to have to go through the cycles because have to go through the cycles because have to go through the cycles because mythos fable-like models are going to mythos fable-like models are going to mythos fable-like models are going to find everything. find everything. find everything. And the volume is going to be And the volume is going to be And the volume is going to be excessive. Just if you look at the excessive. Just if you look at the excessive. Just if you look at the project glass wing update from project glass wing update from project glass wing update from Anthropic, Anthropic, Anthropic, >> Mhm.

>> Mhm. >> Mhm. >> 40,000 CVs already reported to open >> 40,000 CVs already reported to open >> 40,000 CVs already reported to open source zero-day. source zero-day. source zero-day. And and you know, the open source And and you know, the open source And and you know, the open source maintainers are looking at this and maintainers are looking at this and maintainers are looking at this and going, "Huh, well, I was just doing this going, "Huh, well, I was just doing this going, "Huh, well, I was just doing this for fun. Now you're going to go throw for fun. Now you're going to go throw for fun. Now you're going to go throw all this crap of work at me. all this crap of work at me. all this crap of work at me. Like and then you're going to tell me I Like and then you're going to tell me I Like and then you're going to tell me I got to fix it really really fast. Like I got to fix it really really fast. Like I got to fix it really really fast. Like I don't owe you anything." don't owe you anything." don't owe you anything." >> Mhm. >> Mhm. >> Mhm. >> So, like this whole trust in open >> So, like this whole trust in open >> So, like this whole trust in open source, source, source, like it's going to come to a head like it's going to come to a head like it's going to come to a head because it's like who's going to fix all because it's like who's going to fix all because it's like who's going to fix all these problems? So, we have partnered these problems? So, we have partnered these problems? So, we have partnered with Anthropic and Linux Foundation and with Anthropic and Linux Foundation and with Anthropic and Linux Foundation and Microsoft and others to start this Microsoft and others to start this Microsoft and others to start this project called Project Acropolis, which project called Project Acropolis, which project called Project Acropolis, which is like us coming together as a is like us coming together as a is like us coming together as a community of experts community of experts community of experts >> Mhm. >> Mhm. >> Mhm. >> that are going to help maintainers >> that are going to help maintainers >> that are going to help maintainers go fix years and years worth of security go fix years and years worth of security go fix years and years worth of security problems in a very short amount of time problems in a very short amount of time problems in a very short amount of time because the world depends on it. Like because the world depends on it. Like because the world depends on it. Like I'm not worried about Microsoft or I'm not worried about Microsoft or I'm not worried about Microsoft or Google or others being able to patch. Google or others being able to patch. Google or others being able to patch. I'm worried about the water supply, the I'm worried about the water supply, the I'm worried about the water supply, the critical infrastructure, the nuclear critical infrastructure, the nuclear critical infrastructure, the nuclear power plants. Like power plants. Like power plants. Like that's where the real risk to humanity that's where the real risk to humanity that's where the real risk to humanity is in the short order. So so, I think we is in the short order. So so, I think we is in the short order. So so, I think we as an industry have come to this as an industry have come to this as an industry have come to this reckoning point. We've got to get reckoning point. We've got to get reckoning point. We've got to get through this. It's going to be painful through this. It's going to be painful through this. It's going to be painful for some time till we normalize all this for some time till we normalize all this for some time till we normalize all this backlog of security problems found and backlog of security problems found and backlog of security problems found and improve secure code generation moving improve secure code generation moving improve secure code generation moving forward. So, that's one.

forward. So, that's one. forward. So, that's one. The second thing is yes, I think the The second thing is yes, I think the The second thing is yes, I think the software supply chain is complicated, software supply chain is complicated, software supply chain is complicated, more complicated with skills and MCP and more complicated with skills and MCP and more complicated with skills and MCP and get up actions and NPM and PiPI and I get up actions and NPM and PiPI and I get up actions and NPM and PiPI and I think we're going to have to move to think we're going to have to move to think we're going to have to move to operating at on zero trust principles on operating at on zero trust principles on operating at on zero trust principles on software development. When we we when we software development. When we we when we software development. When we we when we hear the word development environments, hear the word development environments, hear the word development environments, we never treated [clears throat] them as we never treated [clears throat] them as we never treated [clears throat] them as production. And so, you know, this whole production. And so, you know, this whole production. And so, you know, this whole smash and grab robberies, they're going smash and grab robberies, they're going smash and grab robberies, they're going to continue till as an industry critical to continue till as an industry critical to continue till as an industry critical mass shift happens mass shift happens mass shift happens to focus developer workstation security. to focus developer workstation security. to focus developer workstation security. So, I think that's going to be kind of So, I think that's going to be kind of So, I think that's going to be kind of the next big thing. the next big thing. the next big thing. And the third thing is I think we all And the third thing is I think we all And the third thing is I think we all have to retool have to retool have to retool for security for security for security to be in the agenda workflow cuz to be in the agenda workflow cuz to be in the agenda workflow cuz otherwise we are generating a lot of otherwise we are generating a lot of otherwise we are generating a lot of security slop right now security slop right now security slop right now >> Mhm. >> Mhm. >> Mhm. >> which will come to bite us in the coming >> which will come to bite us in the coming >> which will come to bite us in the coming weeks, months, and years. weeks, months, and years. weeks, months, and years. >> Yeah, figuring out how to get past the >> Yeah, figuring out how to get past the >> Yeah, figuring out how to get past the slop cannons right now is going to be a slop cannons right now is going to be a slop cannons right now is going to be a mess. You're absolutely right. mess. You're absolutely right. mess. You're absolutely right. Do you think Have we proven? I mean, Do you think Have we proven? I mean, Do you think Have we proven? I mean, you've got the science and you've got you've got the science and you've got you've got the science and you've got the PhDs. Like, is secure code possible the PhDs. Like, is secure code possible the PhDs. Like, is secure code possible or will we be always be playing or will we be always be playing or will we be always be playing whack-a-mole? Is perfectly secure code whack-a-mole? Is perfectly secure code whack-a-mole? Is perfectly secure code possible?

possible? possible? >> Yeah. >> Yeah. >> Yeah. I think it's more possible today than it I think it's more possible today than it I think it's more possible today than it ever has been. Like, trying to do annual ever has been. Like, trying to do annual ever has been. Like, trying to do annual security training for software security training for software security training for software development professionals and then development professionals and then development professionals and then expecting 10,000 developers in a company expecting 10,000 developers in a company expecting 10,000 developers in a company to follow the same model at all times to follow the same model at all times to follow the same model at all times doesn't work. But agents are good at doesn't work. But agents are good at doesn't work. But agents are good at following instructions and rules and following instructions and rules and following instructions and rules and direction if you can give them direction if you can give them direction if you can give them consistent direction. So, is it going to consistent direction. So, is it going to consistent direction. So, is it going to be perfect? No, but is there an be perfect? No, but is there an be perfect? No, but is there an opportunity for us to create a lot more opportunity for us to create a lot more opportunity for us to create a lot more secure code? secure code? secure code? >> Mhm. >> Mhm. >> Mhm. >> Yes, but it won't magically happen. It >> Yes, but it won't magically happen. It >> Yes, but it won't magically happen. It will not certainly happen with oh, the will not certainly happen with oh, the will not certainly happen with oh, the frontier labs are just going to do it frontier labs are just going to do it frontier labs are just going to do it all and we'll all be good, right? all and we'll all be good, right? all and we'll all be good, right? There'll be different expertise required There'll be different expertise required There'll be different expertise required in different area and you'll have to in different area and you'll have to in different area and you'll have to build it all into your common harness. build it all into your common harness. build it all into your common harness. And it's not going to be perfect. The And it's not going to be perfect. The And it's not going to be perfect. The reason it's not going to be perfect is reason it's not going to be perfect is reason it's not going to be perfect is security is all a given a function of security is all a given a function of security is all a given a function of time, cost, compute. More things will be time, cost, compute. More things will be time, cost, compute. More things will be found. So, it's an evergreen problem. found. So, it's an evergreen problem. found. So, it's an evergreen problem. Even if I ship what seems to be Even if I ship what seems to be Even if I ship what seems to be completely secure code today, completely secure code today, completely secure code today, somebody will find a way to break it somebody will find a way to break it somebody will find a way to break it tomorrow. So, the more [clears throat] tomorrow. So, the more [clears throat] tomorrow. So, the more [clears throat] important workflow we all need to have important workflow we all need to have important workflow we all need to have is that maturity of CI/CD pipeline, is that maturity of CI/CD pipeline, is that maturity of CI/CD pipeline, maturity of security, maturity of security, maturity of security, and automation, agentic remediation and automation, agentic remediation and automation, agentic remediation automation. So, when that next thing automation. So, when that next thing automation. So, when that next thing comes up tomorrow morning, comes up tomorrow morning, comes up tomorrow morning, it's solved in 30 minutes, and it's solved in 30 minutes, and it's solved in 30 minutes, and it's handled, right? Like I I don't need it's handled, right? Like I I don't need it's handled, right? Like I I don't need a war room like we did with Log4j from a war room like we did with Log4j from a war room like we did with Log4j from weeks and months to go address that weeks and months to go address that weeks and months to go address that problem.

problem. problem. >> That would be the dream. That would be >> That would be the dream. That would be >> That would be the dream. That would be the dream. And then that's what you're the dream. And then that's what you're the dream. And then that's what you're working towards. I mean, that's the working towards. I mean, that's the working towards. I mean, that's the goal, right? goal, right? goal, right? >> That's right. We're kind of moving >> That's right. We're kind of moving >> That's right. We're kind of moving towards much more agentic and more towards much more agentic and more towards much more agentic and more autonomous software security embedded autonomous software security embedded autonomous software security embedded into the workflow, into the workflow, into the workflow, giving that intelligence to the agents giving that intelligence to the agents giving that intelligence to the agents to do 90-95% of the work out of the box, to do 90-95% of the work out of the box, to do 90-95% of the work out of the box, so the humans are kind of that final so the humans are kind of that final so the humans are kind of that final verification and trust layer, not the verification and trust layer, not the verification and trust layer, not the core part of how we're going to action core part of how we're going to action core part of how we're going to action everything. everything. everything. >> Very cool. Well, I want to encourage >> Very cool. Well, I want to encourage >> Very cool. Well, I want to encourage viewers and listeners to check out viewers and listeners to check out viewers and listeners to check out enderlabs.com. Particularly, check out enderlabs.com. Particularly, check out enderlabs.com. Particularly, check out the research section, the work that the research section, the work that the research section, the work that you're doing, the agent benchmark, the you're doing, the agent benchmark, the you're doing, the agent benchmark, the threat research, the work you're doing threat research, the work you're doing threat research, the work you're doing with universities is really powerful. with universities is really powerful. with universities is really powerful. And then they can go and learn about all And then they can go and learn about all And then they can go and learn about all the developer tools and the platforms the developer tools and the platforms the developer tools and the platforms that you are working on. Thank you so that you are working on. Thank you so that you are working on. Thank you so much, Varun Badhwar, for chatting with much, Varun Badhwar, for chatting with much, Varun Badhwar, for chatting with me today. me today. me today. >> Thanks, Scott. Great to be here. >> Thanks, Scott. Great to be here. >> Thanks, Scott. Great to be here. >> This has been another episode of >> This has been another episode of >> This has been another episode of Hanselminutes, and we'll see you again Hanselminutes, and we'll see you again Hanselminutes, and we'll see you again next week.