Build-Time vs. Run-Time: Why Dev Tools Fail in Production — Averi Kitsch & Prerna Kakkar, Google

Hey everyone, how all of you are doing Hey everyone, how all of you are doing today? today? today? Yeah. Uh so nice to meet you everyone. Yeah. Uh so nice to meet you everyone. Yeah. Uh so nice to meet you everyone. Uh today uh I and my friend Avery are Uh today uh I and my friend Avery are Uh today uh I and my friend Avery are going to talk about build time versus going to talk about build time versus going to talk about build time versus runtime. Why your developer tools fail runtime. Why your developer tools fail runtime. Why your developer tools fail in production. in production. in production. So firstly, know about us. So firstly, know about us. So firstly, know about us. >> Hi everybody. I'm Avery Kit and I'm a >> Hi everybody. I'm Avery Kit and I'm a >> Hi everybody. I'm Avery Kit and I'm a staff software engineer working on staff software engineer working on staff software engineer working on Google Cloud databases. I'm currently Google Cloud databases. I'm currently Google Cloud databases. I'm currently the technical lead for MCP toolbox for the technical lead for MCP toolbox for the technical lead for MCP toolbox for databases, our open-source uh database databases, our open-source uh database databases, our open-source uh database MCP server and our Google Cloud MCP MCP server and our Google Cloud MCP MCP server and our Google Cloud MCP server um maintainer. server um maintainer. server um maintainer. Hi, I'm Pna and I am currently working Hi, I'm Pna and I am currently working Hi, I'm Pna and I am currently working as senior software engineer at Google as senior software engineer at Google as senior software engineer at Google and I am currently tech lead for Eval and I am currently tech lead for Eval and I am currently tech lead for Eval bench which is the evaluation framework bench which is the evaluation framework bench which is the evaluation framework for all your agent tech MCP and skills for all your agent tech MCP and skills for all your agent tech MCP and skills need and I'm also an active contributor need and I'm also an active contributor need and I'm also an active contributor to MCP toolbox. to MCP toolbox. to MCP toolbox. So today we are going to cover three So today we are going to cover three So today we are going to cover three areas broadly. We will firstly start areas broadly. We will firstly start areas broadly. We will firstly start with the history of MCP at Google. Then with the history of MCP at Google. Then with the history of MCP at Google. Then we will cover on the common tool we will cover on the common tool we will cover on the common tool patterns that we have found from our own patterns that we have found from our own patterns that we have found from our own work and practices and how did we use work and practices and how did we use work and practices and how did we use all those practices to build some tools all those practices to build some tools all those practices to build some tools for database access and how you can use for database access and how you can use for database access and how you can use them and then lastly we will talk about them and then lastly we will talk about them and then lastly we will talk about security guard rails how you can stop security guard rails how you can stop security guard rails how you can stop data leaks using identity aware data leaks using identity aware data leaks using identity aware guardrails.

guardrails. guardrails. So let's get to know the background So let's get to know the background So let's get to know the background quickly. Um I'll talk about MCB toolbox quickly. Um I'll talk about MCB toolbox quickly. Um I'll talk about MCB toolbox for database. It's an open-source for database. It's an open-source for database. It's an open-source self-managed uh serving that we provide. self-managed uh serving that we provide. self-managed uh serving that we provide. Uh it has currently about 15.7K GitHub Uh it has currently about 15.7K GitHub Uh it has currently about 15.7K GitHub stars. We have 132 plus active stars. We have 132 plus active stars. We have 132 plus active contributors across 40 plus different contributors across 40 plus different contributors across 40 plus different databases. It's highly customizable databases. It's highly customizable databases. It's highly customizable framework and basically we provide you framework and basically we provide you framework and basically we provide you with connection pooling integrated O and with connection pooling integrated O and with connection pooling integrated O and you don't even need to care about the you don't even need to care about the you don't even need to care about the observability. You will get all of them observability. You will get all of them observability. You will get all of them out of the box. out of the box. out of the box. Then if you don't want to do a Then if you don't want to do a Then if you don't want to do a self-managed one but you want to have a self-managed one but you want to have a self-managed one but you want to have a hosted scaled version, we provide hosted scaled version, we provide hosted scaled version, we provide something as Google managed MCP. It's something as Google managed MCP. It's something as Google managed MCP. It's fully managed. Uh you can plug it across fully managed. Uh you can plug it across fully managed. Uh you can plug it across various agents and ids or harnesses like various agents and ids or harnesses like various agents and ids or harnesses like Gemini CLI, anti-gravity CLI, cloud Gemini CLI, anti-gravity CLI, cloud Gemini CLI, anti-gravity CLI, cloud code, you name any. uh it's co uh it's code, you name any. uh it's co uh it's code, you name any. uh it's co uh it's governed and the discovery is simple and governed and the discovery is simple and governed and the discovery is simple and we also provide model armor which we also provide model armor which we also provide model armor which provides secure access management and provides secure access management and provides secure access management and identity control. So combined with uh identity control. So combined with uh identity control. So combined with uh the managed version of MCP and the MCP the managed version of MCP and the MCP the managed version of MCP and the MCP toolbox last month we had 20 million toolbox last month we had 20 million toolbox last month we had 20 million tool calls.

tool calls. tool calls. Um some of the common tool patterns that Um some of the common tool patterns that Um some of the common tool patterns that we have observed specifically for we have observed specifically for we have observed specifically for databases. So I'm going to quickly talk databases. So I'm going to quickly talk databases. So I'm going to quickly talk about them. about them. about them. Firstly uh is the control plane tools. Firstly uh is the control plane tools. Firstly uh is the control plane tools. What we like to call them is admin tools What we like to call them is admin tools What we like to call them is admin tools or manage tools. It is basically in or manage tools. It is basically in or manage tools. It is basically in developer assistance space. So it will developer assistance space. So it will developer assistance space. So it will help you create like instance, manage help you create like instance, manage help you create like instance, manage your instance, create your databases, your instance, create your databases, your instance, create your databases, manage your databases. It will help you manage your databases. It will help you manage your databases. It will help you with all your DBA needs. But you need to with all your DBA needs. But you need to with all your DBA needs. But you need to be very careful. You need to be you need be very careful. You need to be you need be very careful. You need to be you need to have a human in the loop because we to have a human in the loop because we to have a human in the loop because we don't want to carry out any dangerous don't want to carry out any dangerous don't want to carry out any dangerous activities. activities. activities. Um so these tools are built on already Um so these tools are built on already Um so these tools are built on already provisioned public API so you get provisioned public API so you get provisioned public API so you get monitoring and other things out of the monitoring and other things out of the monitoring and other things out of the box. box. box. Next one is natural language to SQL or Next one is natural language to SQL or Next one is natural language to SQL or NL2SQL tools. So basically we are NL2SQL tools. So basically we are NL2SQL tools. So basically we are relying on a tool called as execute SQL relying on a tool called as execute SQL relying on a tool called as execute SQL and with the help of agent we generate and with the help of agent we generate and with the help of agent we generate raw SQL queries. So you can use this raw SQL queries. So you can use this raw SQL queries. So you can use this cases where you don't know uh what cases where you don't know uh what cases where you don't know uh what queries you would require beforehand. So queries you would require beforehand. So queries you would require beforehand. So you will get all these queries out of you will get all these queries out of you will get all these queries out of the out of the box. So it it focuses on the out of the box. So it it focuses on the out of the box. So it it focuses on the developer assistance and analytical the developer assistance and analytical the developer assistance and analytical agents and uh you can use it for agents and uh you can use it for agents and uh you can use it for flexible explorations. So for example, flexible explorations. So for example, flexible explorations. So for example, we have one of the examples like find we have one of the examples like find we have one of the examples like find all customers in California who bought a all customers in California who bought a all customers in California who bought a winter coat in July and returned it winter coat in July and returned it winter coat in July and returned it within 14 days and group them by the within 14 days and group them by the within 14 days and group them by the marketing campaign that originally

marketing campaign that originally marketing campaign that originally acquired them. So this is one of the acquired them. So this is one of the acquired them. So this is one of the queries where uh you can use this tool queries where uh you can use this tool queries where uh you can use this tool uh to get your answers. But then we have something called a But then we have something called a structure SQL tools which is getting structure SQL tools which is getting structure SQL tools which is getting quite popular and this targets mainly quite popular and this targets mainly quite popular and this targets mainly the production use cases where you know the production use cases where you know the production use cases where you know like what SQL query you want to use and like what SQL query you want to use and like what SQL query you want to use and you want to have security built in and you want to have security built in and you want to have security built in and uh you the parameters are already uh you the parameters are already uh you the parameters are already configured so uh you prevent SQL configured so uh you prevent SQL configured so uh you prevent SQL injection injection injection and ensure highly controlled access by and ensure highly controlled access by and ensure highly controlled access by restricting agent to predefined logic. restricting agent to predefined logic. restricting agent to predefined logic. It also helps you with your latency It also helps you with your latency It also helps you with your latency needs and reduce the hallucination on needs and reduce the hallucination on needs and reduce the hallucination on the agent side. the agent side. the agent side. Now we come to the main topic I guess Now we come to the main topic I guess Now we come to the main topic I guess for which you guys are here for for which you guys are here for for which you guys are here for buildtime versus runtime. So buildtime buildtime versus runtime. So buildtime buildtime versus runtime. So buildtime are the developer assistant use cases. are the developer assistant use cases. are the developer assistant use cases. Um you can think about the initial two Um you can think about the initial two Um you can think about the initial two cases that we presented to you like the cases that we presented to you like the cases that we presented to you like the NL2SQL tools and the control plane NL2SQL tools and the control plane NL2SQL tools and the control plane tools. They come into the category of tools. They come into the category of tools. They come into the category of buildtime tools. uh it's atomic and f buildtime tools. uh it's atomic and f buildtime tools. uh it's atomic and f flexible but again you don't want to flexible but again you don't want to flexible but again you don't want to delete your databases so it requires to delete your databases so it requires to delete your databases so it requires to be a human in the loop case and you be a human in the loop case and you be a human in the loop case and you can't run them on the on production use can't run them on the on production use can't run them on the on production use cases but let's say I'm interested in cases but let's say I'm interested in cases but let's say I'm interested in building some chat B and I want to do building some chat B and I want to do building some chat B and I want to do production use cases there you rely on production use cases there you rely on production use cases there you rely on runtime or end user applications you can runtime or end user applications you can runtime or end user applications you can build those using patenting AI or lchain build those using patenting AI or lchain build those using patenting AI or lchain um so you can see one of the examples um so you can see one of the examples um so you can see one of the examples like we have a cancel order a like we have a cancel order a like we have a cancel order a deterministic structure SQL query that

deterministic structure SQL query that deterministic structure SQL query that we have given and you can use it as a we have given and you can use it as a we have given and you can use it as a tool. tool. tool. This is one of the examples uh or demo This is one of the examples uh or demo This is one of the examples uh or demo for like wherein a buildtime tool was for like wherein a buildtime tool was for like wherein a buildtime tool was used and uh you can see the error used and uh you can see the error used and uh you can see the error message. So uh agent actually asked to message. So uh agent actually asked to message. So uh agent actually asked to delete the table and start fresh. We delete the table and start fresh. We delete the table and start fresh. We deleted everything and there were no deleted everything and there were no deleted everything and there were no safeguard or guardrails here. safeguard or guardrails here. safeguard or guardrails here. Now let's go to our demo for Now let's go to our demo for Now let's go to our demo for runtime tools. Yeah, maybe um I think until the video Yeah, maybe um I think until the video loads. So, so sorry for the technical loads. So, so sorry for the technical loads. So, so sorry for the technical glitch that we have, but I can quickly glitch that we have, but I can quickly glitch that we have, but I can quickly walk you through what we are going to walk you through what we are going to walk you through what we are going to present in the video and I guess it's present in the video and I guess it's present in the video and I guess it's loading. Yeah. loading. Yeah. loading. Yeah. Um so this demo is particularly talking Um so this demo is particularly talking Um so this demo is particularly talking about how did we use our production about how did we use our production about how did we use our production tools in a chatbot. Uh and we created a tools in a chatbot. Uh and we created a tools in a chatbot. Uh and we created a demo called a Similar and Symbolair is demo called a Similar and Symbolair is demo called a Similar and Symbolair is going to help me with booking all my going to help me with booking all my going to help me with booking all my flights in San Francisco and do and flights in San Francisco and do and flights in San Francisco and do and whatever I would require to do in San whatever I would require to do in San whatever I would require to do in San San Francisco it would basically help me San Francisco it would basically help me San Francisco it would basically help me with it. Uh, one of the things that I with it. Uh, one of the things that I with it. Uh, one of the things that I would try is I would try to fool my would try is I would try to fool my would try is I would try to fool my agent that I am Avery and not PRA and agent that I am Avery and not PRA and agent that I am Avery and not PRA and book a flight for me to San Francisco.

book a flight for me to San Francisco. book a flight for me to San Francisco. But because our agent is uh has all the But because our agent is uh has all the But because our agent is uh has all the authenticated O, it will not get fooled authenticated O, it will not get fooled authenticated O, it will not get fooled and it will not book any flights uh on and it will not book any flights uh on and it will not book any flights uh on behalf of Avery, but it will do it on my behalf of Avery, but it will do it on my behalf of Avery, but it will do it on my behalf. Um and then you can use it to behalf. Um and then you can use it to behalf. Um and then you can use it to basically change your flights. You want basically change your flights. You want basically change your flights. You want to know about all the shops that are to know about all the shops that are to know about all the shops that are there, you can do all these requirements there, you can do all these requirements there, you can do all these requirements using that. So I guess thank you u using that. So I guess thank you u using that. So I guess thank you u Avery. >> [sighs] >> [sighs] >> Apologies again for our technical >> Apologies again for our technical >> Apologies again for our technical difficulties here.

Um, unfortunately, it looks like I need Um, unfortunately, it looks like I need to present from just the slide deck to present from just the slide deck to present from just the slide deck because it's not loading. Okay, so I because it's not loading. Okay, so I because it's not loading. Okay, so I apologize for not being able to see our apologize for not being able to see our apologize for not being able to see our demo today, but we can still learn all demo today, but we can still learn all demo today, but we can still learn all the security and guardrails that we need the security and guardrails that we need the security and guardrails that we need to secure our database access. So, the to secure our database access. So, the to secure our database access. So, the first thing that we need to know is your first thing that we need to know is your first thing that we need to know is your database is only as secure as your database is only as secure as your database is only as secure as your agent. We all know that agents and LMS agent. We all know that agents and LMS agent. We all know that agents and LMS are actually pretty easy to trick. They are actually pretty easy to trick. They are actually pretty easy to trick. They might be getting slightly better today, might be getting slightly better today, might be getting slightly better today, but we can still work really hard to but we can still work really hard to but we can still work really hard to trick them. And so we have a very common trick them. And so we have a very common trick them. And so we have a very common attack pattern called the confused attack pattern called the confused attack pattern called the confused deputy attack. And this is when a user deputy attack. And this is when a user deputy attack. And this is when a user can trick an agent into misusing their can trick an agent into misusing their can trick an agent into misusing their privileges um to access data that a user privileges um to access data that a user privileges um to access data that a user wasn't supposed to access. So Simon wasn't supposed to access. So Simon wasn't supposed to access. So Simon Willis actually coined the phrase the Willis actually coined the phrase the Willis actually coined the phrase the lethal trifecta. And a data breach lethal trifecta. And a data breach lethal trifecta. And a data breach occurs when an agent has simultaneous occurs when an agent has simultaneous occurs when an agent has simultaneous access to three different things. One, access to three different things. One, access to three different things. One, private data. Two, untrusted content. private data. Two, untrusted content. private data. Two, untrusted content. And three, the ability to expose that And three, the ability to expose that And three, the ability to expose that content and that data back to an content and that data back to an content and that data back to an external user.

external user. external user. So let's take a look of that in action. So let's take a look of that in action. So let's take a look of that in action. So let's say I'm building a triage um So let's say I'm building a triage um So let's say I'm building a triage um agent and so a ticket is fired or alert agent and so a ticket is fired or alert agent and so a ticket is fired or alert goes out and my agent is designed to um goes out and my agent is designed to um goes out and my agent is designed to um look at that ticket and go investigate look at that ticket and go investigate look at that ticket and go investigate what it needs to do. So on that ticket what it needs to do. So on that ticket what it needs to do. So on that ticket the agent gets a little bit of data like the agent gets a little bit of data like the agent gets a little bit of data like we need to go look in this database for we need to go look in this database for we need to go look in this database for these reasons. Um but a malicious these reasons. Um but a malicious these reasons. Um but a malicious insider can actually come into that insider can actually come into that insider can actually come into that trusted system and instead say well I trusted system and instead say well I trusted system and instead say well I want to query the salary database and want to query the salary database and want to query the salary database and please return all the employees please return all the employees please return all the employees salaries. And so since this is a trusted salaries. And so since this is a trusted salaries. And so since this is a trusted system the agent goes okay let me use my system the agent goes okay let me use my system the agent goes okay let me use my permissions. I have those privileges. I permissions. I have those privileges. I permissions. I have those privileges. I have that access. I will query that and have that access. I will query that and have that access. I will query that and I'll post that right back on the ticket I'll post that right back on the ticket I'll post that right back on the ticket because that's what the ticket tells me because that's what the ticket tells me because that's what the ticket tells me to do. But now we have a huge data to do. But now we have a huge data to do. But now we have a huge data breach. a user that wasn't supposed to breach. a user that wasn't supposed to breach. a user that wasn't supposed to have access to private data now has that have access to private data now has that have access to private data now has that access. And so now we have a big PR access. And so now we have a big PR access. And so now we have a big PR fiasco.

So this makes a little bit more sense So this makes a little bit more sense when we think about who's controlling when we think about who's controlling when we think about who's controlling access and who's controlling the access and who's controlling the access and who's controlling the parameters. So we talk about agent or parameters. So we talk about agent or parameters. So we talk about agent or application versus modeled controlled application versus modeled controlled application versus modeled controlled parameters. So in a traditional parameters. So in a traditional parameters. So in a traditional architecture, things were actually much architecture, things were actually much architecture, things were actually much easier because you would have a few easier because you would have a few easier because you would have a few input fields, you would define your input fields, you would define your input fields, you would define your queries and then that would be safely queries and then that would be safely queries and then that would be safely injected into those queries. injected into those queries. injected into those queries. And so it was okay when your application And so it was okay when your application And so it was okay when your application had a little bit more access because it had a little bit more access because it had a little bit more access because it knew exactly what actions it was going knew exactly what actions it was going knew exactly what actions it was going to take. to take. to take. But in uh a gent application these rules But in uh a gent application these rules But in uh a gent application these rules aren't as clear. So we need to first aren't as clear. So we need to first aren't as clear. So we need to first think about um separating the three think about um separating the three think about um separating the three different identities. We have the user different identities. We have the user different identities. We have the user identity, we have the application identity, we have the application identity, we have the application identity and the agent identity. So first um we need to think about what So first um we need to think about what the user has access to. So the user just the user has access to. So the user just the user has access to. So the user just needs to have access to the application.

needs to have access to the application. needs to have access to the application. that application's workload identity can that application's workload identity can that application's workload identity can have a little bit more broader access um have a little bit more broader access um have a little bit more broader access um because it needs to probably talk to because it needs to probably talk to because it needs to probably talk to different services but the agent running different services but the agent running different services but the agent running in that application only needs to have in that application only needs to have in that application only needs to have access to the data that that end user access to the data that that end user access to the data that that end user initially needs to have. initially needs to have. initially needs to have. So then next we need to think about So then next we need to think about So then next we need to think about who's controlling the tool inputs. So we who's controlling the tool inputs. So we who's controlling the tool inputs. So we have um agent parameters have um agent parameters have um agent parameters um and a application parameters. So um and a application parameters. So um and a application parameters. So agent parameters are the untrusted agent parameters are the untrusted agent parameters are the untrusted inputs that the agent is deriving inputs that the agent is deriving inputs that the agent is deriving dynamically. And then we also have dynamically. And then we also have dynamically. And then we also have application parameters. These are the application parameters. These are the application parameters. These are the factual constraints that we need to keep factual constraints that we need to keep factual constraints that we need to keep outside of the agents uh control. Okay. So now let's look at the evolution Okay. So now let's look at the evolution of a secure tool. Here we have a fully of a secure tool. Here we have a fully of a secure tool. Here we have a fully modeled control tool. And so essentially modeled control tool. And so essentially modeled control tool. And so essentially the agent here is a super user. It has the agent here is a super user. It has the agent here is a super user. It has access to database credentials, the access to database credentials, the access to database credentials, the host, the port, the connection details, host, the port, the connection details, host, the port, the connection details, and even the raw SQL query.

and even the raw SQL query. and even the raw SQL query. And so we're only secure as um the agent And so we're only secure as um the agent And so we're only secure as um the agent here. And we can really easily again here. And we can really easily again here. And we can really easily again trick the agent into exposing all of trick the agent into exposing all of trick the agent into exposing all of this data. And now we have access to this data. And now we have access to this data. And now we have access to essentially any database in the system. essentially any database in the system. essentially any database in the system. So Toolbox solves for this um by So Toolbox solves for this um by So Toolbox solves for this um by introducing a source primitive. introducing a source primitive. introducing a source primitive. So we move the connection details out of So we move the connection details out of So we move the connection details out of the agents control and in toolbox um a the agents control and in toolbox um a the agents control and in toolbox um a user will preconfigure the connection user will preconfigure the connection user will preconfigure the connection details in a YAML file and then when we details in a YAML file and then when we details in a YAML file and then when we start our MCP server those are safely start our MCP server those are safely start our MCP server those are safely injected and so we do not have to have injected and so we do not have to have injected and so we do not have to have the agent um to have access to that. So we can add a little bit more control So we can add a little bit more control to our um source security as well. Our to our um source security as well. Our to our um source security as well. Our number one request that we get from number one request that we get from number one request that we get from customers is read only restrictions. We customers is read only restrictions. We customers is read only restrictions. We want to be able to remove all right want to be able to remove all right want to be able to remove all right ability from agents if we need that ability from agents if we need that ability from agents if we need that specific uh user journey. So this means specific uh user journey. So this means specific uh user journey. So this means removing right tools but also down to removing right tools but also down to removing right tools but also down to the database driver ensuring that we can the database driver ensuring that we can the database driver ensuring that we can only do read only queries.

only do read only queries. only do read only queries. If we're also concerned about again If we're also concerned about again If we're also concerned about again blast radius um and securing all of our blast radius um and securing all of our blast radius um and securing all of our tables and our databases um some of our tables and our databases um some of our tables and our databases um some of our cloudnative databases have this concept cloudnative databases have this concept cloudnative databases have this concept of allowed data sets. So again we can of allowed data sets. So again we can of allowed data sets. So again we can add that like enum to our source in add that like enum to our source in add that like enum to our source in order to continue to restrict um the order to continue to restrict um the order to continue to restrict um the blast radius of um the agents control blast radius of um the agents control blast radius of um the agents control and lastly is output size. You might not and lastly is output size. You might not and lastly is output size. You might not actually think that this is a security actually think that this is a security actually think that this is a security layer, but if again the agent gets into layer, but if again the agent gets into layer, but if again the agent gets into the wrong hands, we can reduce that the wrong hands, we can reduce that the wrong hands, we can reduce that blast radius by saying uh the agent can blast radius by saying uh the agent can blast radius by saying uh the agent can only uh grab this much data. So we're only uh grab this much data. So we're only uh grab this much data. So we're not overwhelming both our agent or our not overwhelming both our agent or our not overwhelming both our agent or our database. So sweet, we have our configurable So sweet, we have our configurable sources tool. So you can see here that sources tool. So you can see here that sources tool. So you can see here that actually now our tool input, our tool actually now our tool input, our tool actually now our tool input, our tool signature is very minimalized. we only signature is very minimalized. we only signature is very minimalized. we only have the SQL string that's um being have the SQL string that's um being have the SQL string that's um being generated by the agent. But this comes to our actual our next But this comes to our actual our next pro problem. We want to be able to pro problem. We want to be able to pro problem. We want to be able to control what the agent is running. We control what the agent is running. We control what the agent is running. We don't want the agent to have the ability don't want the agent to have the ability don't want the agent to have the ability to generate any SQL um that it can think to generate any SQL um that it can think to generate any SQL um that it can think of. So toolbox introduces custom tools of. So toolbox introduces custom tools of. So toolbox introduces custom tools and again in our YAML file we can define and again in our YAML file we can define and again in our YAML file we can define the exact SQL uh statement that will run the exact SQL uh statement that will run the exact SQL uh statement that will run very reliable.

very reliable. very reliable. It's a reliable and secure uh SQL query. It's a reliable and secure uh SQL query. It's a reliable and secure uh SQL query. Um this also allows us to customize the Um this also allows us to customize the Um this also allows us to customize the tool name and the tool description. tool name and the tool description. tool name and the tool description. These are really important for the agent These are really important for the agent These are really important for the agent to have the context on how to use this to have the context on how to use this to have the context on how to use this tool um accurately. tool um accurately. tool um accurately. And in the system we use prepared And in the system we use prepared And in the system we use prepared statements with type parameters in order statements with type parameters in order statements with type parameters in order to reduce um SQL injection attacks. So to reduce um SQL injection attacks. So to reduce um SQL injection attacks. So we make sure that everything is um we make sure that everything is um we make sure that everything is um we validate all the input types um when we validate all the input types um when we validate all the input types um when we inject that into the SQL for the we inject that into the SQL for the we inject that into the SQL for the user. Okay, let's dive into a little bit more Okay, let's dive into a little bit more of best practices for tool quality. So of best practices for tool quality. So of best practices for tool quality. So we really highly recommend that tools we really highly recommend that tools we really highly recommend that tools focus on outcomes. We really shouldn't focus on outcomes. We really shouldn't focus on outcomes. We really shouldn't be thinking in atomic rest APIs. we be thinking in atomic rest APIs. we be thinking in atomic rest APIs. we should think about what the action should think about what the action should think about what the action actually needs to do. This also reduces actually needs to do. This also reduces actually needs to do. This also reduces the round trip of needing to make the round trip of needing to make the round trip of needing to make multiple tool calls. And again, the multiple tool calls. And again, the multiple tool calls. And again, the descriptions are guidance. We shouldn't descriptions are guidance. We shouldn't descriptions are guidance. We shouldn't um duplicate information like input um duplicate information like input um duplicate information like input parameters because the agent already has parameters because the agent already has parameters because the agent already has access to that. So, writing really good access to that. So, writing really good access to that. So, writing really good um tool descriptions is very important um tool descriptions is very important um tool descriptions is very important for accurate tool usage.

for accurate tool usage. for accurate tool usage. We also recommend that you separate read We also recommend that you separate read We also recommend that you separate read versus write tools. Um by doing this you versus write tools. Um by doing this you versus write tools. Um by doing this you can automatically approve read tools and can automatically approve read tools and can automatically approve read tools and but you can also then send write tools but you can also then send write tools but you can also then send write tools uh to the user for um confirmation and uh to the user for um confirmation and uh to the user for um confirmation and this just makes it very much more clear this just makes it very much more clear this just makes it very much more clear for the agent to use these for the agent to use these for the agent to use these and this is actually uh the next is and this is actually uh the next is and this is actually uh the next is actionable errors. This is the number actionable errors. This is the number actionable errors. This is the number one thing that I think we can all do one thing that I think we can all do one thing that I think we can all do better. So usually we just return like a better. So usually we just return like a better. So usually we just return like a generic HTTP error four or four but we generic HTTP error four or four but we generic HTTP error four or four but we all know agents are actually really all know agents are actually really all know agents are actually really smart now and so if you give the ability smart now and so if you give the ability smart now and so if you give the ability to have an error of that can be to have an error of that can be to have an error of that can be retrieded the agent can actually take retrieded the agent can actually take retrieded the agent can actually take that action. So being able to return a that action. So being able to return a that action. So being able to return a error is really important and lastly is error is really important and lastly is error is really important and lastly is simple inputs. We see that people try to simple inputs. We see that people try to simple inputs. We see that people try to use these complex maps uh complex use these complex maps uh complex use these complex maps uh complex primitives to um that an agent needs to primitives to um that an agent needs to primitives to um that an agent needs to be able to build and that is not be able to build and that is not be able to build and that is not reliable. Using flat structure with um reliable. Using flat structure with um reliable. Using flat structure with um with uh simple inputs will really with uh simple inputs will really with uh simple inputs will really increase your reliability.

So sweet. Now we're at custom semantic So sweet. Now we're at custom semantic tools. You can see that we now have our tools. You can see that we now have our tools. You can see that we now have our lookup flights tool that takes in the lookup flights tool that takes in the lookup flights tool that takes in the dynamic parameters such as user ID and dynamic parameters such as user ID and dynamic parameters such as user ID and date. And so now our we're very much date. And so now our we're very much date. And so now our we're very much more secure because the agent isn't more secure because the agent isn't more secure because the agent isn't generating that SQL query. It doesn't generating that SQL query. It doesn't generating that SQL query. It doesn't have the ability to kind of go off the have the ability to kind of go off the have the ability to kind of go off the rails. It only is looking at these very rails. It only is looking at these very rails. It only is looking at these very specific inputs. But user ID is actually a very sensitive But user ID is actually a very sensitive piece of information. It is PII. we need piece of information. It is PII. we need piece of information. It is PII. we need to also remove that from the ability of to also remove that from the ability of to also remove that from the ability of the agent's control. So we can do this the agent's control. So we can do this the agent's control. So we can do this in two different ways. We have bounded in two different ways. We have bounded in two different ways. We have bounded parameters. This is when the application parameters. This is when the application parameters. This is when the application first um authenticates the user and then first um authenticates the user and then first um authenticates the user and then we can bind that parameter um directly we can bind that parameter um directly we can bind that parameter um directly to our tool. And so that restricts the to our tool. And so that restricts the to our tool. And so that restricts the agents control of it. It actually never agents control of it. It actually never agents control of it. It actually never sees that user identity. sees that user identity. sees that user identity. But toolbox also solves for this in But toolbox also solves for this in But toolbox also solves for this in another way called authenticated another way called authenticated another way called authenticated parameters. This is when we tell the parameters. This is when we tell the parameters. This is when we tell the tool that you're going to receive a tool that you're going to receive a tool that you're going to receive a identity token, an open ID, a signed jot identity token, an open ID, a signed jot identity token, an open ID, a signed jot token, and when we call that tool that token, and when we call that tool that token, and when we call that tool that we want it first to validate that token.

we want it first to validate that token. we want it first to validate that token. Is that token real? Is that token Is that token real? Is that token Is that token real? Is that token correct? And then we'll extract the user correct? And then we'll extract the user correct? And then we'll extract the user claims from that token for the user. And claims from that token for the user. And claims from that token for the user. And so the claims usually include like a so the claims usually include like a so the claims usually include like a user ID, an email, um an issuer. user ID, an email, um an issuer. user ID, an email, um an issuer. And so it's secured because we're again And so it's secured because we're again And so it's secured because we're again extracting that user identity out of the extracting that user identity out of the extracting that user identity out of the agents control and binding that to the agents control and binding that to the agents control and binding that to the tool. So now um we're have a much more secure So now um we're have a much more secure tool. We have our lookup flights tool tool. We have our lookup flights tool tool. We have our lookup flights tool that only takes in a very easy parameter that only takes in a very easy parameter that only takes in a very easy parameter such as date. It doesn't have to handle such as date. It doesn't have to handle such as date. It doesn't have to handle any sensitive information such as PII, any sensitive information such as PII, any sensitive information such as PII, user identity. And so we're really here user identity. And so we're really here user identity. And so we're really here now at um our zero trust architecture now at um our zero trust architecture now at um our zero trust architecture where we're in full control of where we're in full control of where we're in full control of everything that we need to be in control everything that we need to be in control everything that we need to be in control of. So thank you all for coming to listen to So thank you all for coming to listen to our talk today. Again, I apologize for our talk today. Again, I apologize for our talk today. Again, I apologize for our technical difficulties. Uh we highly our technical difficulties. Uh we highly our technical difficulties. Uh we highly recommend if you want to learn more recommend if you want to learn more recommend if you want to learn more about our technologies um that you look about our technologies um that you look about our technologies um that you look at our documentation and our uh GitHub at our documentation and our uh GitHub at our documentation and our uh GitHub repository. I also really want to repository. I also really want to repository. I also really want to highlight our eval bench repository highlight our eval bench repository highlight our eval bench repository because this is how we know that our because this is how we know that our because this is how we know that our tools are working well and eval.

So thank you all for joining us today. So thank you all for joining us today. [applause]