Your LLM Stack Is a 2008 Database With Better Marketing — Lovina Dmello, NVIDIA

>> Hi everyone. So, my name is Lavina >> Hi everyone. So, my name is Lavina D'Mello and I'm a senior software D'Mello and I'm a senior software D'Mello and I'm a senior software developer at Nvidia and I work on the developer at Nvidia and I work on the developer at Nvidia and I work on the deep learning infrastructure team. deep learning infrastructure team. deep learning infrastructure team. So, the title here is a bit of So, the title here is a bit of So, the title here is a bit of provocation. What I really mean and what provocation. What I really mean and what provocation. What I really mean and what I want you to leave with is uh for the I want you to leave with is uh for the I want you to leave with is uh for the next 15 minutes, I'm going to make one next 15 minutes, I'm going to make one next 15 minutes, I'm going to make one uncomfortable argument, which is almost uncomfortable argument, which is almost uncomfortable argument, which is almost everything that is breaking in the everything that is breaking in the everything that is breaking in the production ML security isn't some exotic production ML security isn't some exotic production ML security isn't some exotic AI attack. AI attack. AI attack. It's the same boring infrastructure It's the same boring infrastructure It's the same boring infrastructure mistakes that we supposedly fixed years mistakes that we supposedly fixed years mistakes that we supposedly fixed years ago. ago. ago. So, now once uh you look across the So, now once uh you look across the So, now once uh you look across the research from the whole field, the research from the whole field, the research from the whole field, the pattern is very pattern is very pattern is very easy, like it's very difficult to unsee. easy, like it's very difficult to unsee. easy, like it's very difficult to unsee. Let me start with a story. So, into in Let me start with a story. So, into in Let me start with a story. So, into in 2023, security researchers went looking 2023, security researchers went looking 2023, security researchers went looking at Ray clusters. at Ray clusters. at Ray clusters. And Ray is one of the most popular And Ray is one of the most popular And Ray is one of the most popular framework for distributed ML.

framework for distributed ML. framework for distributed ML. So, what they found out over there was So, what they found out over there was So, what they found out over there was there were thousands of clusters that there were thousands of clusters that there were thousands of clusters that were sitting open on the internet. What were sitting open on the internet. What were sitting open on the internet. What that means is the dashboards were open that means is the dashboards were open that means is the dashboards were open and there were job APIs that were open. and there were job APIs that were open. and there were job APIs that were open. And why that happened? Because And why that happened? Because And why that happened? Because authentication was off by default and authentication was off by default and authentication was off by default and somebody just forgot to turn it on while somebody just forgot to turn it on while somebody just forgot to turn it on while putting them ML in them uh putting them ML in them uh putting them ML in them uh in production. So, I'm a little bit short, you know. So, I'm a little bit short, you know. >> [laughter] >> [laughter] >> [laughter] >> So, I need this. >> So, I need this. >> So, I need this. Thank you. Thank you. Thank you. Okay. So, uh the exposure at that time Okay. So, uh the exposure at that time Okay. So, uh the exposure at that time was a lot. Like, it was over billion was a lot. Like, it was over billion was a lot. Like, it was over billion dollars. So, now let's sit with this dollars. So, now let's sit with this dollars. So, now let's sit with this part. This was not a zero-day attack, part. This was not a zero-day attack, part. This was not a zero-day attack, and it was not a clever new attack and it was not a clever new attack and it was not a clever new attack uh on a neural network. So, what uh on a neural network. So, what uh on a neural network. So, what happened was somebody just forgot to happened was somebody just forgot to happened was somebody just forgot to turn the default setting on while turn the default setting on while turn the default setting on while putting them into the production putting them into the production putting them into the production environment. So, it just depends upon environment. So, it just depends upon environment. So, it just depends upon what the configurations are on different what the configurations are on different what the configurations are on different deployment environments in the deployment environments in the deployment environments in the infrastructure.

infrastructure. infrastructure. And when we look at the breaches that And when we look at the breaches that And when we look at the breaches that actually make the news, actually make the news, actually make the news, what we see, this is not a rule, it's what we see, this is not a rule, it's what we see, this is not a rule, it's not an exception, but it's almost never not an exception, but it's almost never not an exception, but it's almost never the fancy adversarial attack. What is the fancy adversarial attack. What is the fancy adversarial attack. What is happening is it's maybe like somebody happening is it's maybe like somebody happening is it's maybe like somebody exposed an API key. What happened is exposed an API key. What happened is exposed an API key. What happened is maybe some account was just maybe some account was just maybe some account was just over-privileged. Maybe some more model over-privileged. Maybe some more model over-privileged. Maybe some more model weight was weight was weight was uh in a public bucket. And the whole uh in a public bucket. And the whole uh in a public bucket. And the whole breach, breach, breach, it's it's it's it was just like somebody forgot to it was just like somebody forgot to it was just like somebody forgot to flip. flip. flip. So, how did we get here? Let's rewind a So, how did we get here? Let's rewind a So, how did we get here? Let's rewind a little bit. So, what used to happen is a little bit. So, what used to happen is a little bit. So, what used to happen is a classic 2008 application was classic 2008 application was classic 2008 application was deterministic. So, we could reason what deterministic. So, we could reason what deterministic. So, we could reason what the code would exactly do. You know, it the code would exactly do. You know, it the code would exactly do. You know, it had a fixed parameter, and it served one had a fixed parameter, and it served one had a fixed parameter, and it served one tenant. Security researchers or security tenant. Security researchers or security tenant. Security researchers or security team had a clear mental model. They knew team had a clear mental model. They knew team had a clear mental model. They knew like what they need to secure. And like like what they need to secure. And like like what they need to secure. And like great tools were built around that time.

great tools were built around that time. great tools were built around that time. So, later what happened is we put this So, later what happened is we put this So, later what happened is we put this machine learning models into production. machine learning models into production. machine learning models into production. And we quietly broke all of the three And we quietly broke all of the three And we quietly broke all of the three assumptions. assumptions. assumptions. Because behaviors became probabilistic. Because behaviors became probabilistic. Because behaviors became probabilistic. Correct is a distribution now. It is not Correct is a distribution now. It is not Correct is a distribution now. It is not one branch like we say yes or no. one branch like we say yes or no. one branch like we say yes or no. And then we are protecting the weights And then we are protecting the weights And then we are protecting the weights which the weights itself are copyable. which the weights itself are copyable. which the weights itself are copyable. And the leaks are straight through the And the leaks are straight through the And the leaks are straight through the API that we are trying to serve. And to API that we are trying to serve. And to API that we are trying to serve. And to make it more affordable, we pack make it more affordable, we pack make it more affordable, we pack everything onto a shared multi-tenant everything onto a shared multi-tenant everything onto a shared multi-tenant GPU where GPU where GPU where other security risk come into picture. other security risk come into picture. other security risk come into picture. And now the LLM stack itself blows up And now the LLM stack itself blows up And now the LLM stack itself blows up wide open. And prompts become control wide open. And prompts become control wide open. And prompts become control flow and answers come from retrieval flow and answers come from retrieval flow and answers come from retrieval corpora. And agents are going to take corpora. And agents are going to take corpora. And agents are going to take the actions for us. the actions for us. the actions for us. So here is the thing. We changed So here is the thing. We changed So here is the thing. We changed everything about the stack when we moved everything about the stack when we moved everything about the stack when we moved to machine learning, but we forgot to to machine learning, but we forgot to to machine learning, but we forgot to change the security assumptions. So we change the security assumptions. So we change the security assumptions. So we are running a 2028 playbook on a 2026 are running a 2028 playbook on a 2026 are running a 2028 playbook on a 2026 system. That's what the whole talk is system. That's what the whole talk is system. That's what the whole talk is about.

about. about. So few things, five things stand out So few things, five things stand out So few things, five things stand out across all of these and I have tried to across all of these and I have tried to across all of these and I have tried to boil them down into three core ideas. boil them down into three core ideas. boil them down into three core ideas. Uh and here they are. Uh and here they are. Uh and here they are. So first one is misconfigurations. So So first one is misconfigurations. So So first one is misconfigurations. So lot of the times misconfigurations lot of the times misconfigurations lot of the times misconfigurations happen. It is not the missing features happen. It is not the missing features happen. It is not the missing features that are number one failure reason. that are number one failure reason. that are number one failure reason. Second is the defenses that we are Second is the defenses that we are Second is the defenses that we are actually reading, they might not actually reading, they might not actually reading, they might not actually survive into production. actually survive into production. actually survive into production. And third one is we have to secure ML And third one is we have to secure ML And third one is we have to secure ML like infrastructure and it's not like like infrastructure and it's not like like infrastructure and it's not like securing just a model, you know? securing just a model, you know? securing just a model, you know? And if we if you forget everything about And if we if you forget everything about And if we if you forget everything about this talk, I want you to remember these this talk, I want you to remember these this talk, I want you to remember these three points over here. three points over here. three points over here. Uh and I'll share a few couple of things Uh and I'll share a few couple of things Uh and I'll share a few couple of things at the end. at the end. at the end. Uh Uh Uh Okay. So over here is a map I want you Okay. So over here is a map I want you Okay. So over here is a map I want you to carry with you. It's four pillars to carry with you. It's four pillars to carry with you. It's four pillars about the defenses in depth. So at the about the defenses in depth. So at the about the defenses in depth. So at the bottom we can see the at the bottom in bottom we can see the at the bottom in bottom we can see the at the bottom in the black we can see the infrastructure the black we can see the infrastructure the black we can see the infrastructure security. Over here comes into picture security. Over here comes into picture security. Over here comes into picture the containers, the networks, the GPU the containers, the networks, the GPU the containers, the networks, the GPU virtualization and encryption. On top virtualization and encryption. On top virtualization and encryption. On top So, this infrastructure security is the So, this infrastructure security is the So, this infrastructure security is the foundation.

foundation. foundation. And on top of it sits the access control And on top of it sits the access control And on top of it sits the access control like who is allowed to do what and then like who is allowed to do what and then like who is allowed to do what and then is the runtime security which is is the runtime security which is is the runtime security which is checking checking checking uh and watching the traffic while the uh and watching the traffic while the uh and watching the traffic while the system is life. So, they feed each other system is life. So, they feed each other system is life. So, they feed each other and wrapping all of it and wrapping all of it and wrapping all of it uh to day-to-day practice like um how uh to day-to-day practice like um how uh to day-to-day practice like um how the team works plus compliance, incident the team works plus compliance, incident the team works plus compliance, incident response, and honestly the culture. So, response, and honestly the culture. So, response, and honestly the culture. So, one thing to remember over here is if uh one thing to remember over here is if uh one thing to remember over here is if uh infrastructure is the foundation and infrastructure is the foundation and infrastructure is the foundation and failures down there can cascade up. So, failures down there can cascade up. So, failures down there can cascade up. So, if we have a perfect access control if we have a perfect access control if we have a perfect access control policy, it's just a decoration if the policy, it's just a decoration if the policy, it's just a decoration if the cluster underneath is wide open. cluster underneath is wide open. cluster underneath is wide open. So, So, So, yeah. So, those are the four levels. yeah. So, those are the four levels. yeah. So, those are the four levels. And this is a quick map of what we are And this is a quick map of what we are And this is a quick map of what we are defending against. So, there are six defending against. So, there are six defending against. So, there are six categories. categories. categories. The two are about the model itself. So, The two are about the model itself. So, The two are about the model itself. So, where we have the adversarial inputs and where we have the adversarial inputs and where we have the adversarial inputs and we have the model extraction. Over here we have the model extraction. Over here we have the model extraction. Over here the primary defenses should be uh about the primary defenses should be uh about the primary defenses should be uh about how we do the input validation, how we how we do the input validation, how we how we do the input validation, how we do the adversarial detection, how we do do the adversarial detection, how we do do the adversarial detection, how we do the rate limiting, API auth, and query the rate limiting, API auth, and query the rate limiting, API auth, and query analysis.

analysis. analysis. The next two are about the data and The next two are about the data and The next two are about the data and integrity. Over there comes the integrity. Over there comes the integrity. Over there comes the poisoning and supply chain and privacy poisoning and supply chain and privacy poisoning and supply chain and privacy breaches. So, we should make sure that breaches. So, we should make sure that breaches. So, we should make sure that over there we make sure to secure the over there we make sure to secure the over there we make sure to secure the CI/CD, model signing, and provenance, CI/CD, model signing, and provenance, CI/CD, model signing, and provenance, access control, encryption, all of those access control, encryption, all of those access control, encryption, all of those comes into that picture. And the last comes into that picture. And the last comes into that picture. And the last one, our classic infrastructure, where one, our classic infrastructure, where one, our classic infrastructure, where we take care of we take care of we take care of um um um basically container isolation, network basically container isolation, network basically container isolation, network segmentation, segmentation, segmentation, uh R back, and behavior analysis. uh R back, and behavior analysis. uh R back, and behavior analysis. So, if we notice one thing over here, uh So, if we notice one thing over here, uh So, if we notice one thing over here, uh the green ones, infrastructure the green ones, infrastructure the green ones, infrastructure compromise and insiders, those are compromise and insiders, those are compromise and insiders, those are exactly where the real breaches keep exactly where the real breaches keep exactly where the real breaches keep landing. And they are the cheapest to landing. And they are the cheapest to landing. And they are the cheapest to get wrong by accident, which brings me get wrong by accident, which brings me get wrong by accident, which brings me to the core idea number one, which is to the core idea number one, which is to the core idea number one, which is mis- misconfiguration, not missing mis- misconfiguration, not missing mis- misconfiguration, not missing feature. So, those that is one of the feature. So, those that is one of the feature. So, those that is one of the number one failure mode. number one failure mode. number one failure mode. Um okay. So, let me walk you through one Um okay. So, let me walk you through one Um okay. So, let me walk you through one of the research. What happens is of the research. What happens is of the research. What happens is researchers audited 50 real production researchers audited 50 real production researchers audited 50 real production setups running machine learning. In 78% setups running machine learning. In 78% setups running machine learning. In 78% of them, what the researchers found out of them, what the researchers found out of them, what the researchers found out was at least one critical security was at least one critical security was at least one critical security mistake. And mistake. And mistake. And and and and And here's why that mattered beyond And here's why that mattered beyond And here's why that mattered beyond engineering. So, that was a engineering. So, that was a engineering. So, that was a misconfiguration, misconfiguration, misconfiguration, and the difference between a normal

and the difference between a normal and the difference between a normal Tuesday and something Tuesday and something Tuesday and something having a headline about our company in having a headline about our company in having a headline about our company in the news is this kind of incident, where the news is this kind of incident, where the news is this kind of incident, where there could be like a critical there could be like a critical there could be like a critical misconfiguration mistake. misconfiguration mistake. misconfiguration mistake. And And And And the same three problems kept showing And the same three problems kept showing And the same three problems kept showing up in this research study. First was up in this research study. First was up in this research study. First was access controls were left wide open. So, access controls were left wide open. So, access controls were left wide open. So, what happens when an access controls are what happens when an access controls are what happens when an access controls are left wide open? Almost any account left wide open? Almost any account left wide open? Almost any account can do almost anything. So, that is one can do almost anything. So, that is one can do almost anything. So, that is one of the problem. Second is nothing sep- of the problem. Second is nothing sep- of the problem. Second is nothing sep- nothing was separated. For example, nothing was separated. For example, nothing was separated. For example, different parts of the system different parts of the system different parts of the system So, once an attacker would get access to So, once an attacker would get access to So, once an attacker would get access to one part of the system, he could do one part of the system, he could do one part of the system, he could do anything in the whole system. So, there anything in the whole system. So, there anything in the whole system. So, there is the thing like separating different is the thing like separating different is the thing like separating different parts of the system. parts of the system. parts of the system. And And And the third one were the passwords and the third one were the passwords and the third one were the passwords and trained models. So, themselves they were trained models. So, themselves they were trained models. So, themselves they were sitting into a storage that anybody sitting into a storage that anybody sitting into a storage that anybody could reach.

could reach. could reach. So, why does this keep happening? So, So, why does this keep happening? So, So, why does this keep happening? So, because this production protections are because this production protections are because this production protections are built by security experts for security built by security experts for security built by security experts for security experts and then they get handed to the experts and then they get handed to the experts and then they get handed to the ML teams. And the responsibility of ML ML teams. And the responsibility of ML ML teams. And the responsibility of ML team is um basically making the model team is um basically making the model team is um basically making the model accurate, not configuring the accurate, not configuring the accurate, not configuring the infrastructure. So, the protection is infrastructure. So, the protection is infrastructure. So, the protection is right there in the platform. It just right there in the platform. It just right there in the platform. It just sometimes get switched off by some sometimes get switched off by some sometimes get switched off by some configurations. Okay. So, now moving to the core idea Okay. So, now moving to the core idea two, which is defenses you read about two, which is defenses you read about two, which is defenses you read about don't survive into the production. don't survive into the production. don't survive into the production. So, here is So, here is So, here is here is the thing. Every control costs here is the thing. Every control costs here is the thing. Every control costs us something. And in ML that something us something. And in ML that something us something. And in ML that something is latency and throughput. The two is latency and throughput. The two is latency and throughput. The two things that things that things that SLA is made up of. Let's watch the SLA is made up of. Let's watch the SLA is made up of. Let's watch the colors. So, the green tier over here, colors. So, the green tier over here, colors. So, the green tier over here, the basics like logins, input checking, the basics like logins, input checking, the basics like logins, input checking, costs costs costs do those cost under about 8% and that do those cost under about 8% and that do those cost under about 8% and that should always be there. No debate over should always be there. No debate over should always be there. No debate over there. The second one is the orange there. The second one is the orange there. The second one is the orange tier. So, this is like the heavier tier. So, this is like the heavier tier. So, this is like the heavier isolation that walls around that isolation that walls around that isolation that walls around that that walls workloads of each other. So, that walls workloads of each other. So, that walls workloads of each other. So, here it runs 10 to 20%. So, we can use here it runs 10 to 20%. So, we can use here it runs 10 to 20%. So, we can use it selectively like for sensitive and it selectively like for sensitive and it selectively like for sensitive and untrusted workloads, what we can do is

untrusted workloads, what we can do is untrusted workloads, what we can do is we can decide if we want it or not. And we can decide if we want it or not. And we can decide if we want it or not. And the purple ones is catching malicious the purple ones is catching malicious the purple ones is catching malicious input in real time. So, this is the most input in real time. So, this is the most input in real time. So, this is the most expensive one. Over here, it can cost expensive one. Over here, it can cost expensive one. Over here, it can cost like like like I don't know, 15 to 30% and it's one of I don't know, 15 to 30% and it's one of I don't know, 15 to 30% and it's one of the thing that researchers love the the thing that researchers love the the thing that researchers love the most. But, because we cannot slow every most. But, because we cannot slow every most. But, because we cannot slow every request down, request down, request down, so it's a non-starter. so it's a non-starter. so it's a non-starter. But, what really matters is how much it But, what really matters is how much it But, what really matters is how much it cost depends almost entirely on how well cost depends almost entirely on how well cost depends almost entirely on how well we build it. So, the same security check we build it. So, the same security check we build it. So, the same security check check, it can be done sloppily check, it can be done sloppily check, it can be done sloppily and it can double our response time. and it can double our response time. and it can double our response time. But, if it is done well, like But, if it is done well, like But, if it is done well, like considering all of these things, then considering all of these things, then considering all of these things, then it's fair, like the balance is fair. it's fair, like the balance is fair. it's fair, like the balance is fair. So, the guidance team actually need So, the guidance team actually need So, the guidance team actually need isn't should I do this, it's like how do isn't should I do this, it's like how do isn't should I do this, it's like how do I do it without slowing everything down. I do it without slowing everything down. I do it without slowing everything down. And then we can run the cheap controls And then we can run the cheap controls And then we can run the cheap controls everywhere and we can save the expensive everywhere and we can save the expensive everywhere and we can save the expensive ones for the heavier risk systems.

So, here is the the next slide. So, here is the the next slide. The ecosystem isn't production ready The ecosystem isn't production ready The ecosystem isn't production ready yet. So, what I meant is um yet. So, what I meant is um yet. So, what I meant is um So, there is the first gap that we will So, there is the first gap that we will So, there is the first gap that we will see. The security tooling for ML is see. The security tooling for ML is see. The security tooling for ML is years behind the rest of the software. years behind the rest of the software. years behind the rest of the software. So, regular software teams, they solved So, regular software teams, they solved So, regular software teams, they solved a lot of this decades ago with automated a lot of this decades ago with automated a lot of this decades ago with automated security scanning, proper passport security scanning, proper passport security scanning, proper passport password management, and then we haven't password management, and then we haven't password management, and then we haven't carried it all over to ML yet. Even carried it all over to ML yet. Even carried it all over to ML yet. Even today, there's a good share of early today, there's a good share of early today, there's a good share of early warning systems for ML specific threats. warning systems for ML specific threats. warning systems for ML specific threats. So, the second gap is in the expertise So, the second gap is in the expertise So, the second gap is in the expertise silos. So, security team don't speak ML, silos. So, security team don't speak ML, silos. So, security team don't speak ML, ML teams don't speak security, and the ML teams don't speak security, and the ML teams don't speak security, and the operation teams don't know how the model operation teams don't know how the model operation teams don't know how the model behaves. So, there is a gap over here behaves. So, there is a gap over here behaves. So, there is a gap over here and the problem leaves in the gaps. So, and the problem leaves in the gaps. So, and the problem leaves in the gaps. So, between those gaps where nobody has a between those gaps where nobody has a between those gaps where nobody has a full picture. So, there's a little bit full picture. So, there's a little bit full picture. So, there's a little bit of mismatch over here. And underneath of mismatch over here. And underneath of mismatch over here. And underneath both is a gap between the research and both is a gap between the research and both is a gap between the research and reality. So, research is busy defending reality. So, research is busy defending reality. So, research is busy defending against tiny invisible tweets.

against tiny invisible tweets. against tiny invisible tweets. But But But to an image that to an image that to an image that while the real world, we are fighting while the real world, we are fighting while the real world, we are fighting against the stolen password and then against the stolen password and then against the stolen password and then people are misusing access that they people are misusing access that they people are misusing access that they already have and it studies one model on already have and it studies one model on already have and it studies one model on its own while we are running dozens of its own while we are running dozens of its own while we are running dozens of them them them on shared system. on shared system. on shared system. So, what it lacks are one team can So, what it lacks are one team can So, what it lacks are one team can actually deploy everything. actually deploy everything. actually deploy everything. So, there is this core idea number So, there is this core idea number So, there is this core idea number three. It's the constructive one. So, if three. It's the constructive one. So, if three. It's the constructive one. So, if the problem is infrastructure, we have the problem is infrastructure, we have the problem is infrastructure, we have to secure it like infrastructure, not to secure it like infrastructure, not to secure it like infrastructure, not like a model. And two practical tools like a model. And two practical tools like a model. And two practical tools for that are a maturity model to know for that are a maturity model to know for that are a maturity model to know where we stand and a short list of how where we stand and a short list of how where we stand and a short list of how to fix it first. to fix it first. to fix it first. So, here is So, here is So, here is the next slide. What we have to do is the next slide. What we have to do is the next slide. What we have to do is how to figure out where we actually how to figure out where we actually how to figure out where we actually stand. It's a maturity model that will stand. It's a maturity model that will stand. It's a maturity model that will help us map onto the NIST AI risk help us map onto the NIST AI risk help us map onto the NIST AI risk management framework. So, there will be management framework. So, there will be management framework. So, there will be four levels, each tied four levels, each tied four levels, each tied to an overhead budget and the context.

to an overhead budget and the context. to an overhead budget and the context. So, on the level one, we can see that So, on the level one, we can see that So, on the level one, we can see that under 5% overhead is the bare basics. under 5% overhead is the bare basics. under 5% overhead is the bare basics. Fine for test environment, not for Fine for test environment, not for Fine for test environment, not for production. So, the level two production. So, the level two production. So, the level two 5 to 10% is a real baseline like proper 5 to 10% is a real baseline like proper 5 to 10% is a real baseline like proper logins, encryptions, separating our logins, encryptions, separating our logins, encryptions, separating our networks, basic monitoring. This is the networks, basic monitoring. This is the networks, basic monitoring. This is the minimum bar for production. And the minimum bar for production. And the minimum bar for production. And the level three adds more advanced controls level three adds more advanced controls level three adds more advanced controls that regulated industries like health that regulated industries like health that regulated industries like health care and finance needs. care and finance needs. care and finance needs. So, on this part, most of teams teams So, on this part, most of teams teams So, on this part, most of teams teams believe that they are at level three, believe that they are at level three, believe that they are at level three, but uh they are actually either at level but uh they are actually either at level but uh they are actually either at level one or two, like it really depends how one or two, like it really depends how one or two, like it really depends how we have set up everything. So, on this slide, I want to discuss So, on this slide, I want to discuss about the three misconfiguration behind about the three misconfiguration behind about the three misconfiguration behind most of the risk. So, most of the risk. So, most of the risk. So, the default setup often gives every the default setup often gives every the default setup often gives every account a full admin rights. So, that is account a full admin rights. So, that is account a full admin rights. So, that is what have been observed in most of the what have been observed in most of the what have been observed in most of the research studies. With credentials that research studies. With credentials that research studies. With credentials that never expire, so there are there have never expire, so there are there have never expire, so there are there have been credentials that do not have any been credentials that do not have any been credentials that do not have any expiry. And that can leak the key on its expiry. And that can leak the key on its expiry. And that can leak the key on its own.

own. own. And like each account And like each account And like each account like should do only what it needs to do. like should do only what it needs to do. like should do only what it needs to do. And then the credentials to should And then the credentials to should And then the credentials to should expire very quickly. Now, nothing stops one part of the Now, nothing stops one part of the system from reaching another part of the system from reaching another part of the system from reaching another part of the system. So, if that can be tackled in a system. So, if that can be tackled in a system. So, if that can be tackled in a way way way uh then that can be solved. Like uh then that can be solved. Like uh then that can be solved. Like attacker gets in one part of the system attacker gets in one part of the system attacker gets in one part of the system and then they get it can do everything. and then they get it can do everything. and then they get it can do everything. So, that's uh like a flat network kind So, that's uh like a flat network kind So, that's uh like a flat network kind of issue. of issue. of issue. Then the fix. So, what we can do to fix Then the fix. So, what we can do to fix Then the fix. So, what we can do to fix this is we can wall the parts this is we can wall the parts this is we can wall the parts from each other. And we can require a from each other. And we can require a from each other. And we can require a verified identity between services, not verified identity between services, not verified identity between services, not just network access. just network access. just network access. So, So, So, there is an another issue called the there is an another issue called the there is an another issue called the exposed secrets and models. So, over exposed secrets and models. So, over exposed secrets and models. So, over there passwords could be hardcoded in there passwords could be hardcoded in there passwords could be hardcoded in the code, model files sitting in the the code, model files sitting in the the code, model files sitting in the open storage. So, the fix could be like open storage. So, the fix could be like open storage. So, the fix could be like a proper secret manager, encryption, and a proper secret manager, encryption, and a proper secret manager, encryption, and automatic scanning before before all of automatic scanning before before all of automatic scanning before before all of this ships. So, this is nothing but like this ships. So, this is nothing but like this ships. So, this is nothing but like a basic hygiene. And this kind of a basic hygiene. And this kind of a basic hygiene. And this kind of hygiene can prevent breaches that hygiene can prevent breaches that hygiene can prevent breaches that actually make the news.

So, uh here's the practical stuff. Uh So, uh here's the practical stuff. Uh so, now a couple of personal takes on so, now a couple of personal takes on so, now a couple of personal takes on where this is heading because the ground where this is heading because the ground where this is heading because the ground is moving very faster under us. So, this is moving very faster under us. So, this is moving very faster under us. So, this is the part uh that's least settled. And is the part uh that's least settled. And is the part uh that's least settled. And four things are on my mind right now. four things are on my mind right now. four things are on my mind right now. So, one is prompt injection. So, the So, one is prompt injection. So, the So, one is prompt injection. So, the model can't reliably tell the difference model can't reliably tell the difference model can't reliably tell the difference between our instructions and someone between our instructions and someone between our instructions and someone else's input. else's input. else's input. So, a cleverly worded message can So, a cleverly worded message can So, a cleverly worded message can quickly hijack the whole system. quickly hijack the whole system. quickly hijack the whole system. So, rag poisoning, if our model pulls So, rag poisoning, if our model pulls So, rag poisoning, if our model pulls answers from a document store, and if answers from a document store, and if answers from a document store, and if the document store is poisoned by the document store is poisoned by the document store is poisoned by somebody or somehow, then what happens somebody or somehow, then what happens somebody or somehow, then what happens is it can poison like everything. is it can poison like everything. is it can poison like everything. And nobody has to retain anything, And nobody has to retain anything, And nobody has to retain anything, right? So, GPUs So, next is the GPU side right? So, GPUs So, next is the GPU side right? So, GPUs So, next is the GPU side channel. So, where we can pack multiple channel. So, where we can pack multiple channel. So, where we can pack multiple customers on one GPU in order to save customers on one GPU in order to save customers on one GPU in order to save money. That time information can leak money. That time information can leak money. That time information can leak from one customer to the other. And from one customer to the other. And from one customer to the other. And another is the supply chain. So, what another is the supply chain. So, what another is the supply chain. So, what happens over there is we download models happens over there is we download models happens over there is we download models and add-ons from public sources without and add-ons from public sources without and add-ons from public sources without uh really checking where they come from.

uh really checking where they come from. uh really checking where they come from. So, that's another issue. And the honest So, that's another issue. And the honest So, that's another issue. And the honest caveat over here is uh caveat over here is uh caveat over here is uh all of this moves very fast. Every week all of this moves very fast. Every week all of this moves very fast. Every week something or the else comes up. So, we something or the else comes up. So, we something or the else comes up. So, we treat it as a direction, not a gospel, treat it as a direction, not a gospel, treat it as a direction, not a gospel, and the defenses are immature right now and the defenses are immature right now and the defenses are immature right now because uh the threats keep evolving. Okay. So, three uh So, let's bring it Okay. So, three uh So, let's bring it together. If you forget everything I together. If you forget everything I together. If you forget everything I said today, here are the three said today, here are the three said today, here are the three takeaways. So, one is securing ML is an takeaways. So, one is securing ML is an takeaways. So, one is securing ML is an infrastructural problem, not a model infrastructural problem, not a model infrastructural problem, not a model problem. problem. problem. So, we have to spend So, we have to spend So, we have to spend our budget where the breaches actually our budget where the breaches actually our budget where the breaches actually are happening. Second, the field has are happening. Second, the field has are happening. Second, the field has enough defenses. It just need deployable enough defenses. It just need deployable enough defenses. It just need deployable ones. The frontier isn't a new attack ones. The frontier isn't a new attack ones. The frontier isn't a new attack defense pair. It's making the existing defense pair. It's making the existing defense pair. It's making the existing controls run with proper production controls run with proper production controls run with proper production overhead. overhead. overhead. And that's the engineering problem, And that's the engineering problem, And that's the engineering problem, which is a good news for a room full of which is a good news for a room full of which is a good news for a room full of engineers. And the third one is we have engineers. And the third one is we have engineers. And the third one is we have to fix the misconfigurations, to fix the misconfigurations, to fix the misconfigurations, like over-privileged accesses, flat like over-privileged accesses, flat like over-privileged accesses, flat networks, and exposed secrets. So, then networks, and exposed secrets. So, then networks, and exposed secrets. So, then climb the maturity model on purpose.

climb the maturity model on purpose. climb the maturity model on purpose. And to close the loop And to close the loop And to close the loop on the title, your LLM stack really is a on the title, your LLM stack really is a on the title, your LLM stack really is a 2008 database with better marketing. 2008 database with better marketing. 2008 database with better marketing. So, secure it like a database, lock down So, secure it like a database, lock down So, secure it like a database, lock down accesses, segment the network, and accesses, segment the network, and accesses, segment the network, and protect the data at rest. protect the data at rest. protect the data at rest. So, we already know how to do that. We So, we already know how to do that. We So, we already know how to do that. We just have to do it actually in a better just have to do it actually in a better just have to do it actually in a better way. That's all. So, thank you.