Privacy-Preserving Intelligence — Steve Korshakov, Bee (acq. Amazon)

Hello everyone. I hope this talk will be Hello everyone. I hope this talk will be shorter. Uh shorter. Uh shorter. Uh I'm from Amazon. Uh our company was I'm from Amazon. Uh our company was I'm from Amazon. Uh our company was acquired about 8 months ago and we built acquired about 8 months ago and we built acquired about 8 months ago and we built the the the uh AI wearable which is on my hand. Uh uh AI wearable which is on my hand. Uh uh AI wearable which is on my hand. Uh which is essentially a microphone that which is essentially a microphone that which is essentially a microphone that records everything and builds your records everything and builds your records everything and builds your personal agent, personal AI and uh on personal agent, personal AI and uh on personal agent, personal AI and uh on top of that you can extract all the data top of that you can extract all the data top of that you can extract all the data that you record and plug it to your that you record and plug it to your that you record and plug it to your systems or agents uh and do whatever you systems or agents uh and do whatever you systems or agents uh and do whatever you want. want. want. Uh just to be to get in perspective how Uh just to be to get in perspective how Uh just to be to get in perspective how confidential how like private data we confidential how like private data we confidential how like private data we capturing capturing capturing uh a single person usually like captures uh a single person usually like captures uh a single person usually like captures about 10 million tokens per year. So about 10 million tokens per year. So about 10 million tokens per year. So this um and even within like a first this um and even within like a first this um and even within like a first week of recording uh uh people usually week of recording uh uh people usually week of recording uh uh people usually tell tell tell uh extremely sensitive stuff to their uh extremely sensitive stuff to their uh extremely sensitive stuff to their friends to their family. uh you can friends to their family. uh you can friends to their family. uh you can learn virtually everything about the learn virtually everything about the learn virtually everything about the person within the just like one week of person within the just like one week of person within the just like one week of wearing the B device which is extremely wearing the B device which is extremely wearing the B device which is extremely sensitive. I think we one of the most sensitive. I think we one of the most sensitive. I think we one of the most sensitive uh capture device on the sensitive uh capture device on the sensitive uh capture device on the market now. So and uh because of this we market now. So and uh because of this we market now. So and uh because of this we had to encrypt everything and our had to encrypt everything and our had to encrypt everything and our mission was to not have access to any of mission was to not have access to any of mission was to not have access to any of this data and not being able to look at this data and not being able to look at this data and not being able to look at it anyone at Amazon and it became a it anyone at Amazon and it became a it anyone at Amazon and it became a little bit challenging for us at Amazon little bit challenging for us at Amazon little bit challenging for us at Amazon because uh Amazon itself provides strong

because uh Amazon itself provides strong because uh Amazon itself provides strong security and privacy guarantees but if security and privacy guarantees but if security and privacy guarantees but if you Amazon and you using Amazon stuff you Amazon and you using Amazon stuff you Amazon and you using Amazon stuff there is like much more uh serious there is like much more uh serious there is like much more uh serious security stuff you need to do. security stuff you need to do. security stuff you need to do. Uh Uh Uh first of all we defined like few core first of all we defined like few core first of all we defined like few core principles what we needed to do for our principles what we needed to do for our principles what we needed to do for our specific agent. First of all we believe specific agent. First of all we believe specific agent. First of all we believe the agent should be like working all the the agent should be like working all the the agent should be like working all the time non-stop for for your good. Um it time non-stop for for your good. Um it time non-stop for for your good. Um it should be doing stuff on your behalf. Uh should be doing stuff on your behalf. Uh should be doing stuff on your behalf. Uh and um we should not consume uh customer and um we should not consume uh customer and um we should not consume uh customer resources such as batteries and stuff. resources such as batteries and stuff. resources such as batteries and stuff. So this way uh we uh so this leads us to So this way uh we uh so this leads us to So this way uh we uh so this leads us to one specific design uh of the um of the one specific design uh of the um of the one specific design uh of the um of the our system. Current system usually our system. Current system usually our system. Current system usually builds the uh on state uh on request builds the uh on state uh on request builds the uh on state uh on request response uh system where you send uh response uh system where you send uh response uh system where you send uh request to a l from your say iPhone uh request to a l from your say iPhone uh request to a l from your say iPhone uh calculate something and the back end calculate something and the back end calculate something and the back end that gives you back. Unfortunately, we that gives you back. Unfortunately, we that gives you back. Unfortunately, we already see that this is not enough already see that this is not enough already see that this is not enough uh that we need to uh that we need to run stuff uh that we need to run stuff continuously and sometimes for days we continuously and sometimes for days we continuously and sometimes for days we can see uh we can see this like as a can see uh we can see this like as a can see uh we can see this like as a like glimpse into the future how cloud like glimpse into the future how cloud like glimpse into the future how cloud code works. So like just few months ago code works. So like just few months ago code works. So like just few months ago it was like more like request response it was like more like request response it was like more like request response stuff like change this, change that and

stuff like change this, change that and stuff like change this, change that and now it can works for like hours for us. now it can works for like hours for us. now it can works for like hours for us. We think the same will happen to all We think the same will happen to all We think the same will happen to all your like personal agents anyway. So uh your like personal agents anyway. So uh your like personal agents anyway. So uh because of this we built uh a stateful because of this we built uh a stateful because of this we built uh a stateful runtime with persistent memory um that runtime with persistent memory um that runtime with persistent memory um that we still don't have access to. Uh it can we still don't have access to. Uh it can we still don't have access to. Uh it can connect to different tools. It can connect to different tools. It can connect to different tools. It can connect to any like third party services connect to any like third party services connect to any like third party services if we if we program it to um and we if we if we program it to um and we if we if we program it to um and we don't require the user device to be don't require the user device to be don't require the user device to be online. So it's fully autonomous but at online. So it's fully autonomous but at online. So it's fully autonomous but at the same time it's fully controlled by the same time it's fully controlled by the same time it's fully controlled by the user. the user. the user. Um Um Um so encryption system is built um uh on so encryption system is built um uh on so encryption system is built um uh on few on four like core ideas that we need few on four like core ideas that we need few on four like core ideas that we need to follow. First of all the key leaves to follow. First of all the key leaves to follow. First of all the key leaves and manage it only on customer device. and manage it only on customer device. and manage it only on customer device. So it's users iPhone or Android device So it's users iPhone or Android device So it's users iPhone or Android device itself. We don't have the key ourselves. itself. We don't have the key ourselves. itself. We don't have the key ourselves. We don't persist it anywhere. So keys is We don't persist it anywhere. So keys is We don't persist it anywhere. So keys is is um is stored only on the customer is um is stored only on the customer is um is stored only on the customer phone.

phone. phone. Everything is encrypted. We don't have Everything is encrypted. We don't have Everything is encrypted. We don't have any opt out there. No way to disable it. any opt out there. No way to disable it. any opt out there. No way to disable it. There no way to bypass it. Uh at the There no way to bypass it. Uh at the There no way to bypass it. Uh at the same time we uh to protect ourselves same time we uh to protect ourselves same time we uh to protect ourselves from um like internal threats uh we do from um like internal threats uh we do from um like internal threats uh we do fully transparent and audit uh of our fully transparent and audit uh of our fully transparent and audit uh of our all our workloads and we on top of that all our workloads and we on top of that all our workloads and we on top of that we try to minimize the dependencies on we try to minimize the dependencies on we try to minimize the dependencies on the uh the uh the uh on what we can trust really. Um on what we can trust really. Um on what we can trust really. Um so uh any security system if you do end so uh any security system if you do end so uh any security system if you do end to end encryption or any kind of to end encryption or any kind of to end encryption or any kind of encryption there is a huge problem is encryption there is a huge problem is encryption there is a huge problem is key management. So the first step is key management. So the first step is key management. So the first step is like I want to tell you how we manage like I want to tell you how we manage like I want to tell you how we manage the key. So we start with the uh the key the key. So we start with the uh the key the key. So we start with the uh the key as I mentioned before starts on the as I mentioned before starts on the as I mentioned before starts on the phone and it leaves persisted on the phone and it leaves persisted on the phone and it leaves persisted on the phone. Then the phone connects to our phone. Then the phone connects to our phone. Then the phone connects to our back end and runs very sophisticated back end and runs very sophisticated back end and runs very sophisticated attestation uh pipeline uh that verifies attestation uh pipeline uh that verifies attestation uh pipeline uh that verifies both integrity and that the specific both integrity and that the specific both integrity and that the specific workload is inside of public workload is inside of public workload is inside of public uh transparency log. We use six store uh transparency log. We use six store uh transparency log. We use six store for our transparency log and anyone can for our transparency log and anyone can for our transparency log and anyone can go there and try to look and verify that go there and try to look and verify that go there and try to look and verify that this workload is genuine.

this workload is genuine. this workload is genuine. uh the the method is too complicated to uh the the method is too complicated to uh the the method is too complicated to include in this uh talk but we will include in this uh talk but we will include in this uh talk but we will publish details uh at some point. Uh publish details uh at some point. Uh publish details uh at some point. Uh once the uh once uh at the station was once the uh once uh at the station was once the uh once uh at the station was finished we the the client shares with finished we the the client shares with finished we the the client shares with the our main front end back end and back the our main front end back end and back the our main front end back end and back end then replicates this key with uh the end then replicates this key with uh the end then replicates this key with uh the similar nodes uh that runs within our similar nodes uh that runs within our similar nodes uh that runs within our confidation compute because we can't confidation compute because we can't confidation compute because we can't leave the unencrypted data out of our leave the unencrypted data out of our leave the unencrypted data out of our perimeter uh all our we run our own perimeter uh all our we run our own perimeter uh all our we run our own inference too. So this puts us a little inference too. So this puts us a little inference too. So this puts us a little bit uh more complicated task uh than bit uh more complicated task uh than bit uh more complicated task uh than typical AI company. We run like all kind typical AI company. We run like all kind typical AI company. We run like all kind of models, all kind of inference uh uh of models, all kind of inference uh uh of models, all kind of inference uh uh software and uh so software and uh so software and uh so um yeah and we don't replicate code to um yeah and we don't replicate code to um yeah and we don't replicate code to this inference node. We like replicate this inference node. We like replicate this inference node. We like replicate only on specific ones limiting the scope only on specific ones limiting the scope only on specific ones limiting the scope of what we can do. And on top of that we of what we can do. And on top of that we of what we can do. And on top of that we introduced everywhere where we have the introduced everywhere where we have the introduced everywhere where we have the keys in the memory uh uh the forced keys in the memory uh uh the forced keys in the memory uh uh the forced expiration of seven days. Uh we picked expiration of seven days. Uh we picked expiration of seven days. Uh we picked the seven days because we think it's the seven days because we think it's the seven days because we think it's like how much realistically the time like how much realistically the time like how much realistically the time horizon horizon for the like something horizon horizon for the like something horizon horizon for the like something useful can be done for the user. uh 24 useful can be done for the user. uh 24 useful can be done for the user. uh 24 hours will be too low because you can hours will be too low because you can hours will be too low because you can like not open your phone for like 24 like not open your phone for like 24 like not open your phone for like 24 hours something will be missed and like hours something will be missed and like hours something will be missed and like so we pick like about seven days.

so we pick like about seven days. so we pick like about seven days. Um Um Um then we need you know we need to ship then we need you know we need to ship then we need you know we need to ship some something to the production and some something to the production and some something to the production and then the the biggest question like how then the the biggest question like how then the the biggest question like how we can uh well not ship something that we can uh well not ship something that we can uh well not ship something that will compromise anything. Our goal was will compromise anything. Our goal was will compromise anything. Our goal was to uh build a system that no one inside to uh build a system that no one inside to uh build a system that no one inside of uh Amazon will be able to ship of uh Amazon will be able to ship of uh Amazon will be able to ship anything unnoticed. Obviously the anything unnoticed. Obviously the anything unnoticed. Obviously the software has bugs have problems but we software has bugs have problems but we software has bugs have problems but we shouldn't be well being able to ship shouldn't be well being able to ship shouldn't be well being able to ship anything. So we solve this by two uh two anything. So we solve this by two uh two anything. So we solve this by two uh two tire system essentially. So there is a tire system essentially. So there is a tire system essentially. So there is a dedicated team inside of organization dedicated team inside of organization dedicated team inside of organization inside of uh Amazon and maybe maybe not inside of uh Amazon and maybe maybe not inside of uh Amazon and maybe maybe not even one I would say uh that um manages even one I would say uh that um manages even one I would say uh that um manages the privacy part of this the the privacy part of this the the privacy part of this the transparency log our team when we ship transparency log our team when we ship transparency log our team when we ship the software we can influence them we the software we can influence them we the software we can influence them we can't control them and we hardcode their can't control them and we hardcode their can't control them and we hardcode their in their signing keys inside of our in their signing keys inside of our in their signing keys inside of our client apps and our back ends so we client apps and our back ends so we client apps and our back ends so we can't really uh um uh we as the team we can't really uh um uh we as the team we can't really uh um uh we as the team we can do this and like it's very very hard can do this and like it's very very hard can do this and like it's very very hard and very high level uh um employees need and very high level uh um employees need and very high level uh um employees need to sign off to any kind of change so to sign off to any kind of change so to sign off to any kind of change so it's like at the big company like Amazon it's like at the big company like Amazon it's like at the big company like Amazon it's virtually impossible really um and it's virtually impossible really um and it's virtually impossible really um and uh and we do this in two parts because uh and we do this in two parts because uh and we do this in two parts because it's this process too slow so we split it's this process too slow so we split it's this process too slow so we split it in two parts so like the first one is it in two parts so like the first one is it in two parts so like the first one is to build the base image which we put

to build the base image which we put to build the base image which we put some kind of base software for uh that some kind of base software for uh that some kind of base software for uh that is needed for our own team like the is needed for our own team like the is needed for our own team like the tools that measure the boot, measure the tools that measure the boot, measure the tools that measure the boot, measure the manifest, measure workloads and data manifest, measure workloads and data manifest, measure workloads and data that we need to uh put to the node. Um that we need to uh put to the node. Um that we need to uh put to the node. Um and um and then when we want to deploy and um and then when we want to deploy and um and then when we want to deploy we do the process the same similar time. we do the process the same similar time. we do the process the same similar time. uh we got the base image and then we uh we got the base image and then we uh we got the base image and then we deploy the uh to transparency log deploy the uh to transparency log deploy the uh to transparency log specific manifest that anyone and uh specific manifest that anyone and uh specific manifest that anyone and uh this setup helps us to be able to this setup helps us to be able to this setup helps us to be able to security audit companies and inside security audit companies and inside security audit companies and inside outside to anyone to well we're not outside to anyone to well we're not outside to anyone to well we're not doing this public but like we two like doing this public but like we two like doing this public but like we two like very um high-profile audit companies we very um high-profile audit companies we very um high-profile audit companies we work with them all the time um we can work with them all the time um we can work with them all the time um we can provide any image any any data that was provide any image any any data that was provide any image any any data that was deployed ever so we can like trace any deployed ever so we can like trace any deployed ever so we can like trace any possible uh weak spots uh if we like possible uh weak spots uh if we like possible uh weak spots uh if we like deploy something wrong uh which we do deploy something wrong uh which we do deploy something wrong uh which we do not um then after selfverification of VM not um then after selfverification of VM not um then after selfverification of VM it's uh issues a certificate that embeds it's uh issues a certificate that embeds it's uh issues a certificate that embeds all encryption pro uh all transparency all encryption pro uh all transparency all encryption pro uh all transparency proofs attestation documents into proofs attestation documents into proofs attestation documents into certificate itself we are using private certificate itself we are using private certificate itself we are using private CA because you can't do this in public CA because you can't do this in public CA because you can't do this in public certificates because it will populate certificates because it will populate certificates because it will populate the public uh transparency log so we had the public uh transparency log so we had the public uh transparency log so we had to use the private on um we probably to use the private on um we probably to use the private on um we probably will introduce the uh extra proxy that will introduce the uh extra proxy that will introduce the uh extra proxy that will do a normal TLS with attestation will do a normal TLS with attestation will do a normal TLS with attestation with like lighter mode. Uh but we don't

with like lighter mode. Uh but we don't with like lighter mode. Uh but we don't have this yet. have this yet. have this yet. Um yeah, that's essentially what we Um yeah, that's essentially what we Um yeah, that's essentially what we built. Thank you. built. Thank you. built. Thank you. Any questions? [applause] Can you repeat? Can you repeat? >> Yeah. Sorry. Were there things that had to change Were there things that had to change your own your own your own existing?

existing? existing? >> Oh, what changed with like when we >> Oh, what changed with like when we >> Oh, what changed with like when we joined Amazon? joined Amazon? joined Amazon? >> Yeah. >> Yeah. >> Yeah. >> Well, the the big change is that we uh >> Well, the the big change is that we uh >> Well, the the big change is that we uh before like you run on Amazon and Amazon before like you run on Amazon and Amazon before like you run on Amazon and Amazon gives you pre like guarantees as a gives you pre like guarantees as a gives you pre like guarantees as a customer that they can see your data. customer that they can see your data. customer that they can see your data. But once you inside this changes a lot But once you inside this changes a lot But once you inside this changes a lot because you Amazon like so that's why because you Amazon like so that's why because you Amazon like so that's why you need like to provide more protection you need like to provide more protection you need like to provide more protection on top of this. So we need to protect on top of this. So we need to protect on top of this. So we need to protect from our internal threats too. Uh so from our internal threats too. Uh so from our internal threats too. Uh so that's was a big change. So um before that's was a big change. So um before that's was a big change. So um before that it was um just kind of easier I that it was um just kind of easier I that it was um just kind of easier I would say to configure everything. Um would say to configure everything. Um would say to configure everything. Um not sure I can tell much honestly. Yes, it's just normal EC2 instances. Yes, it's just normal EC2 instances. Yeah. Yeah. Yeah. >> Yeah. Yeah. We almost we not using like >> Yeah. Yeah. We almost we not using like >> Yeah. Yeah. We almost we not using like Yeah. Almost everything we build from Yeah. Almost everything we build from Yeah. Almost everything we build from scratch. Well, um we tried to use like scratch. Well, um we tried to use like scratch. Well, um we tried to use like the existing stuff like that is more the existing stuff like that is more the existing stuff like that is more like common like popular software. We like common like popular software. We like common like popular software. We built uh try we try to minimize amount built uh try we try to minimize amount built uh try we try to minimize amount of code that we produce. So it's I I of code that we produce. So it's I I of code that we produce. So it's I I calculated before this talk it's just calculated before this talk it's just calculated before this talk it's just like about 20k lines on memory safe like about 20k lines on memory safe like about 20k lines on memory safe language. So it was very small scope language. So it was very small scope language. So it was very small scope that we were able to audit and verify that we were able to audit and verify that we were able to audit and verify that all this kind of stuff and most of that all this kind of stuff and most of that all this kind of stuff and most of this code is just verifying at the this code is just verifying at the this code is just verifying at the station really and everything else can station really and everything else can station really and everything else can be like reused and like very we like you

be like reused and like very we like you be like reused and like very we like you know it's very trustworthy I would say know it's very trustworthy I would say know it's very trustworthy I would say software so we didn't try to don't software so we didn't try to don't software so we didn't try to don't invent invent invent uh um yeah we don't try to invent like uh um yeah we don't try to invent like uh um yeah we don't try to invent like when when I was a telegram like we when when I was a telegram like we when when I was a telegram like we reintroduced like build our own crypto reintroduced like build our own crypto reintroduced like build our own crypto and that was like questionable way of and that was like questionable way of and that was like questionable way of doing stuff. So, I try not to do the doing stuff. So, I try not to do the doing stuff. So, I try not to do the same at Amazon, obviously. Yeah, that's same at Amazon, obviously. Yeah, that's same at Amazon, obviously. Yeah, that's what we do. what we do. what we do. >> Any other questions for Steve >> Any other questions for Steve >> Any other questions for Steve >> up in the back? Well, I I just prefer them not to put to Well, I I just prefer them not to put to the computers uh to personal one. Uh so the computers uh to personal one. Uh so the computers uh to personal one. Uh so we we did several experiments how to we we did several experiments how to we we did several experiments how to tame them, not to do bad things. H say I

tame them, not to do bad things. H say I tame them, not to do bad things. H say I think nothing works except like think nothing works except like think nothing works except like sandboxing and just not giving them a sandboxing and just not giving them a sandboxing and just not giving them a way to hurt themselves. It's like you way to hurt themselves. It's like you way to hurt themselves. It's like you know our brains they can't stop the know our brains they can't stop the know our brains they can't stop the heart at will right so otherwise you heart at will right so otherwise you heart at will right so otherwise you know they will be we have much more know they will be we have much more know they will be we have much more problems so I think the same we problems so I think the same we problems so I think the same we shouldn't give them away to do any harm shouldn't give them away to do any harm shouldn't give them away to do any harm that's the only way honestly and um yeah that's the only way honestly and um yeah that's the only way honestly and um yeah and put something in between if they and put something in between if they and put something in between if they want to change something unfort want to change something unfort want to change something unfort yeah >> uh >> uh I surprised they're not representing I surprised they're not representing I surprised they're not representing this uh right like they're screaming so this uh right like they're screaming so this uh right like they're screaming so much about security but they didn't came much about security but they didn't came much about security but they didn't came to this one. Uh to this one. Uh to this one. Uh uh I'm not I tried open claw. It's like uh I'm not I tried open claw. It's like uh I'm not I tried open claw. It's like it was once they started to try to it was once they started to try to it was once they started to try to tighten this down it became much less tighten this down it became much less tighten this down it became much less use useful. use useful. use useful. Um so I think their approach is not Um so I think their approach is not Um so I think their approach is not really that good. So I would love to really that good. So I would love to really that good. So I would love to have wild agent that's that's our goal have wild agent that's that's our goal have wild agent that's that's our goal too but we try to just deploy the too but we try to just deploy the too but we try to just deploy the sandboxer for specific agent and it will sandboxer for specific agent and it will sandboxer for specific agent and it will just they just can't do much of the just they just can't do much of the just they just can't do much of the stuff. Everything else fails stuff. Everything else fails stuff. Everything else fails unfortunately.

unfortunately. unfortunately. >> Cool. All right. Well, thank you so >> Cool. All right. Well, thank you so >> Cool. All right. Well, thank you so much, Steve, for the presentation. That much, Steve, for the presentation. That much, Steve, for the presentation. That was amazing.