Building GTM AI Agents: Lessons from Deploying to 6,000 Users — Sait Izmit, Snowflake

>> Hi everyone. So, I think I'm one of the >> Hi everyone. So, I think I'm one of the last speakers that is standing between last speakers that is standing between last speakers that is standing between you and the long weekend. you and the long weekend. you and the long weekend. So, I hope I can get your energy levels So, I hope I can get your energy levels So, I hope I can get your energy levels up. up. up. Um so, I'm responsible for our internal Um so, I'm responsible for our internal Um so, I'm responsible for our internal AI tools for our sales team. AI tools for our sales team. AI tools for our sales team. And the reason I'm here today is indeed And the reason I'm here today is indeed And the reason I'm here today is indeed like we launched our internal like we launched our internal like we launched our internal go-to-market assistant go-to-market assistant go-to-market assistant uh in September last year. uh in September last year. uh in September last year. It answered more than 1 million It answered more than 1 million It answered more than 1 million questions so far. We have roughly questions so far. We have roughly questions so far. We have roughly answered 40,000 questions a week. answered 40,000 questions a week. answered 40,000 questions a week. Um and we are the customer zero for a Um and we are the customer zero for a Um and we are the customer zero for a lot of Snowflake products. So, this is lot of Snowflake products. So, this is lot of Snowflake products. So, this is built on Snowflake co-work. built on Snowflake co-work. built on Snowflake co-work. Um and I meet a lot of customers every Um and I meet a lot of customers every Um and I meet a lot of customers every week. Okay? So, I meet a lot of week. Okay? So, I meet a lot of week. Okay? So, I meet a lot of enterprises, Fortune 500 companies, and enterprises, Fortune 500 companies, and enterprises, Fortune 500 companies, and then they're all trying to build similar then they're all trying to build similar then they're all trying to build similar things, and they all struggle, right? things, and they all struggle, right? things, and they all struggle, right? So, and then I end up like having this So, and then I end up like having this So, and then I end up like having this discussion with them all the time. Like discussion with them all the time. Like discussion with them all the time. Like they ask like how did you guys do it? they ask like how did you guys do it? they ask like how did you guys do it? And then we share our best practices. And then we share our best practices. And then we share our best practices. So, I will try to share some of those So, I will try to share some of those So, I will try to share some of those things with you. things with you. things with you. Uh I'm told that I need to have some Uh I'm told that I need to have some Uh I'm told that I need to have some code in my presentation. I don't, but I code in my presentation. I don't, but I code in my presentation. I don't, but I will try to show you at least some will try to show you at least some will try to show you at least some architectural diagrams just to make it architectural diagrams just to make it architectural diagrams just to make it more interesting for the engineering more interesting for the engineering more interesting for the engineering audience. audience. audience. Uh but let's jump into it.

Uh but let's jump into it. Uh but let's jump into it. Um Um Um I think before we start like I think I I think before we start like I think I I think before we start like I think I already I was watching the other already I was watching the other already I was watching the other presentations like I think everyone presentations like I think everyone presentations like I think everyone tries to give their interpretation of tries to give their interpretation of tries to give their interpretation of like you know, why are we even building like you know, why are we even building like you know, why are we even building things for go-to-market. Okay? So, this things for go-to-market. Okay? So, this things for go-to-market. Okay? So, this is how I explain it to family and is how I explain it to family and is how I explain it to family and friends. friends. friends. So, let's take Snowflake. Okay? So, we So, let's take Snowflake. Okay? So, we So, let's take Snowflake. Okay? So, we are a company of like, you know, close are a company of like, you know, close are a company of like, you know, close to 10,000 people. to 10,000 people. to 10,000 people. So, if you look at that or our So, if you look at that or our So, if you look at that or our organization, almost half of our organization, almost half of our organization, almost half of our basically workforce is sales, right? And basically workforce is sales, right? And basically workforce is sales, right? And what are they responsible for? They're what are they responsible for? They're what are they responsible for? They're responsible for revenue generation. responsible for revenue generation. responsible for revenue generation. What do they struggle with? And I into I What do they struggle with? And I into I What do they struggle with? And I into I talked to a lot of customers. It's very talked to a lot of customers. It's very talked to a lot of customers. It's very common. You know, everyone's data is common. You know, everyone's data is common. You know, everyone's data is siloed. We work with a lot of siloed. We work with a lot of siloed. We work with a lot of first-party data, a lot of third-party first-party data, a lot of third-party first-party data, a lot of third-party data. It's all locked down in these like data. It's all locked down in these like data. It's all locked down in these like SaaS tools and things like that. And SaaS tools and things like that. And SaaS tools and things like that. And literally we have for example like reps literally we have for example like reps literally we have for example like reps who are using 15 different tools, not who are using 15 different tools, not who are using 15 different tools, not because they love the UI of those tools, because they love the UI of those tools, because they love the UI of those tools, because every tool has a different data because every tool has a different data because every tool has a different data point, and then they end up stitching point, and then they end up stitching point, and then they end up stitching all of that together in spreadsheets and all of that together in spreadsheets and all of that together in spreadsheets and running it there, right? running it there, right? running it there, right? And the data is endless. Like we have And the data is endless. Like we have And the data is endless. Like we have reps who have 1,000 accounts assigned to reps who have 1,000 accounts assigned to reps who have 1,000 accounts assigned to them, 1,000 customers.

them, 1,000 customers. them, 1,000 customers. They have to stay on top of their recent They have to stay on top of their recent They have to stay on top of their recent news, what's happening with their news, what's happening with their news, what's happening with their consumption, did they get in support consumption, did they get in support consumption, did they get in support tickets recently, what was their latest tickets recently, what was their latest tickets recently, what was their latest earning results, everything. There's no earning results, everything. There's no earning results, everything. There's no It's not a single human on this planet It's not a single human on this planet It's not a single human on this planet that can stay on top of that much data. that can stay on top of that much data. that can stay on top of that much data. And then they need to do that 30 times, And then they need to do that 30 times, And then they need to do that 30 times, 40 times a day, right? 40 times a day, right? 40 times a day, right? So, what does AI offer for them? So, what does AI offer for them? So, what does AI offer for them? It offers that data democratization. No It offers that data democratization. No It offers that data democratization. No more like 1,000 dashboards, right? No more like 1,000 dashboards, right? No more like 1,000 dashboards, right? No more access to analysts. Like you know, more access to analysts. Like you know, more access to analysts. Like you know, um it offers automation possibilities um it offers automation possibilities um it offers automation possibilities for them, right? It frees up their for them, right? It frees up their for them, right? It frees up their inbox. It offers tool consolidation. No inbox. It offers tool consolidation. No inbox. It offers tool consolidation. No longer 15 different tools that I need to longer 15 different tools that I need to longer 15 different tools that I need to work for. work for. work for. And that brings productivity savings. It And that brings productivity savings. It And that brings productivity savings. It frees up your time, right? You can use frees up your time, right? You can use frees up your time, right? You can use that time on other things. It helps you that time on other things. It helps you that time on other things. It helps you become a better seller. You're more become a better seller. You're more become a better seller. You're more effective with your customers. And that effective with your customers. And that effective with your customers. And that translates to business results. You can translates to business results. You can translates to business results. You can cover more of your book, you know, you cover more of your book, you know, you cover more of your book, you know, you can have better win rates, uh you can can have better win rates, uh you can can have better win rates, uh you can have shorter deal cycles, and ultimately have shorter deal cycles, and ultimately have shorter deal cycles, and ultimately what everyone cares about, you can get what everyone cares about, you can get what everyone cares about, you can get incremental revenue. Okay, so that's the incremental revenue. Okay, so that's the incremental revenue. Okay, so that's the reason why I'm I'm working for, you reason why I'm I'm working for, you reason why I'm I'm working for, you know, making the go-to-market know, making the go-to-market know, making the go-to-market organizations more effective.

But, there's a catch. These are But, there's a catch. These are non-deterministic systems, non-deterministic systems, non-deterministic systems, right? right? right? And I run into this problem every time And I run into this problem every time And I run into this problem every time with users. with users. with users. I see many, many, many AI projects I see many, many, many AI projects I see many, many, many AI projects failed, failed, failed, and then it fails on this principle. and then it fails on this principle. and then it fails on this principle. User trust is earned extremely hard and User trust is earned extremely hard and User trust is earned extremely hard and is lost overnight, is lost overnight, is lost overnight, right? right? right? So, at the end what you're doing is So, at the end what you're doing is So, at the end what you're doing is you're putting a free-form chatbot you're putting a free-form chatbot you're putting a free-form chatbot there, there, there, right? And people will come in and they right? And people will come in and they right? And people will come in and they will ask any question they can think of. will ask any question they can think of. will ask any question they can think of. If they like what they see in the first If they like what they see in the first If they like what they see in the first five questions, they come back. five questions, they come back. five questions, they come back. If they don't like what they see, it's If they don't like what they see, it's If they don't like what they see, it's 10 times more effort for you to win them 10 times more effort for you to win them 10 times more effort for you to win them back, if you can ever win them back. back, if you can ever win them back. back, if you can ever win them back. Right? Right? Right? So, we have a saying in our team, we say So, we have a saying in our team, we say So, we have a saying in our team, we say quality is P minus one. quality is P minus one. quality is P minus one. Right? Right? Right? And that's basically we take that very, And that's basically we take that very, And that's basically we take that very, very seriously. very seriously. very seriously. So, one of the things that we really So, one of the things that we really So, one of the things that we really cared about is when I first joined the cared about is when I first joined the cared about is when I first joined the team, team, team, you know, the team had all these like you know, the team had all these like you know, the team had all these like data sources connected from our top data sources connected from our top data sources connected from our top dashboards. We they had a knowledge dashboards. We they had a knowledge dashboards. We they had a knowledge assistant built into it and so on. We assistant built into it and so on. We assistant built into it and so on. We had three lines of agent instructions.

had three lines of agent instructions. had three lines of agent instructions. And then before I even tried the agent, And then before I even tried the agent, And then before I even tried the agent, I opened a spreadsheet, I took the sales I opened a spreadsheet, I took the sales I opened a spreadsheet, I took the sales process, I wrote down 150 questions. process, I wrote down 150 questions. process, I wrote down 150 questions. And then the sales our engineering team And then the sales our engineering team And then the sales our engineering team was like, "What are you doing? We don't was like, "What are you doing? We don't was like, "What are you doing? We don't have that data in the agent." have that data in the agent." have that data in the agent." I was like, "It doesn't matter. These I was like, "It doesn't matter. These I was like, "It doesn't matter. These are the questions your sellers are going are the questions your sellers are going are the questions your sellers are going to ask." to ask." to ask." Right? And then we run our test, 50% Right? And then we run our test, 50% Right? And then we run our test, 50% accuracy, you know, like everyone's accuracy, you know, like everyone's accuracy, you know, like everyone's depressed and so on. So, we said, "Okay, depressed and so on. So, we said, "Okay, depressed and so on. So, we said, "Okay, let's make sure that we don't go for let's make sure that we don't go for let's make sure that we don't go for coverage, but we go for quality." Right? coverage, but we go for quality." Right? coverage, but we go for quality." Right? We don't want to try to answer 100 We don't want to try to answer 100 We don't want to try to answer 100 questions and get them 70% right. We questions and get them 70% right. We questions and get them 70% right. We want to answer 50 questions, but get want to answer 50 questions, but get want to answer 50 questions, but get them 95% right. Right? Because with that them 95% right. Right? Because with that them 95% right. Right? Because with that you get a first impression, good first you get a first impression, good first you get a first impression, good first impression, you build a trust with them. impression, you build a trust with them. impression, you build a trust with them. And then rather than being in that boat And then rather than being in that boat And then rather than being in that boat of, "Oh, this thing doesn't work." of, "Oh, this thing doesn't work." of, "Oh, this thing doesn't work." people are like, "Oh, this thing is people are like, "Oh, this thing is people are like, "Oh, this thing is awesome. Can I get more of that?" awesome. Can I get more of that?" awesome. Can I get more of that?" Right? Right? Right? So, we started small and 60% of the data So, we started small and 60% of the data So, we started small and 60% of the data we actually added after the launch, we actually added after the launch, we actually added after the launch, after the 6-7 months post launch. Today, after the 6-7 months post launch. Today, after the 6-7 months post launch. Today, if you look into our agent, I mean, it's if you look into our agent, I mean, it's if you look into our agent, I mean, it's not a small agent. We have 15 semantic not a small agent. We have 15 semantic not a small agent. We have 15 semantic views, 85 tables, 3,000 columns of data. views, 85 tables, 3,000 columns of data. views, 85 tables, 3,000 columns of data. We have like five to six different MCP We have like five to six different MCP We have like five to six different MCP connections on it. You know, close to 20 connections on it. You know, close to 20 connections on it. You know, close to 20 skills connected to that and so on and skills connected to that and so on and skills connected to that and so on and so on.

so on. so on. Right? So, it's a huge system that we Right? So, it's a huge system that we Right? So, it's a huge system that we are managing in here. are managing in here. are managing in here. And then you cannot just launch these And then you cannot just launch these And then you cannot just launch these things to everyone, right? So, we said things to everyone, right? So, we said things to everyone, right? So, we said that, "Look, we need to do this in a that, "Look, we need to do this in a that, "Look, we need to do this in a controlled way because we want to make controlled way because we want to make controlled way because we want to make sure that we earn that first five sure that we earn that first five sure that we earn that first five questions. We don't want to burn our questions. We don't want to burn our questions. We don't want to burn our bridges in that first five questions." bridges in that first five questions." bridges in that first five questions." Right? So, that's why with every product Right? So, that's why with every product Right? So, that's why with every product we do, we do a face launch. we do, we do a face launch. we do, we do a face launch. The first one is a pilot. The goal of The first one is a pilot. The goal of The first one is a pilot. The goal of the pilot is to prove the accuracy, the pilot is to prove the accuracy, the pilot is to prove the accuracy, prove the quality, right? You get your prove the quality, right? You get your prove the quality, right? You get your top, you know, AI native folks in the top, you know, AI native folks in the top, you know, AI native folks in the organization who are eager to work with organization who are eager to work with organization who are eager to work with you, give you feedback, improve the you, give you feedback, improve the you, give you feedback, improve the product, make sure that you got the product, make sure that you got the product, make sure that you got the rough edges through that, right? And rough edges through that, right? And rough edges through that, right? And then after a couple of weeks, you come then after a couple of weeks, you come then after a couple of weeks, you come to a point where it looks like, okay, to a point where it looks like, okay, to a point where it looks like, okay, those rough edges are more smoother now. those rough edges are more smoother now. those rough edges are more smoother now. Okay? Then you go into your better Okay? Then you go into your better Okay? Then you go into your better launch. We do 10% better, right? With launch. We do 10% better, right? With launch. We do 10% better, right? With 600 people. 600 people. 600 people. There you are looking at do I truly have There you are looking at do I truly have There you are looking at do I truly have an basically a minimum viable product? an basically a minimum viable product? an basically a minimum viable product? Is the MVP really there, right? And what Is the MVP really there, right? And what Is the MVP really there, right? And what will happen is that you will start will happen is that you will start will happen is that you will start getting tons of requests. Can you getting tons of requests. Can you getting tons of requests. Can you connect this data? Can you connect that connect this data? Can you connect that connect this data? Can you connect that data and everything? And then you are data and everything? And then you are data and everything? And then you are looking at like where are the actually looking at like where are the actually looking at like where are the actually the concentrations happening? Because the concentrations happening? Because the concentrations happening? Because that means that if you don't get those that means that if you don't get those that means that if you don't get those things in, you don't truly have an MVP, things in, you don't truly have an MVP, things in, you don't truly have an MVP, right? Then it's not going to work for right? Then it's not going to work for right? Then it's not going to work for their daily workflows.

their daily workflows. their daily workflows. And then at this stage, you're also And then at this stage, you're also And then at this stage, you're also trying to prove are they coming back? trying to prove are they coming back? trying to prove are they coming back? Right? So, the things that we really Right? So, the things that we really Right? So, the things that we really track there track there track there is basically like, okay, how many is basically like, okay, how many is basically like, okay, how many questions they're asking and everything, questions they're asking and everything, questions they're asking and everything, but what is the retention rate? So, we but what is the retention rate? So, we but what is the retention rate? So, we exited for example that at like more exited for example that at like more exited for example that at like more than 70% retention rate that the weekly than 70% retention rate that the weekly than 70% retention rate that the weekly active users were coming back. Okay, now active users were coming back. Okay, now active users were coming back. Okay, now we're in a good place, right? We have we're in a good place, right? We have we're in a good place, right? We have confidence on the accuracy, we have on confidence on the accuracy, we have on confidence on the accuracy, we have on the confidence of the basically the the confidence of the basically the the confidence of the basically the coverage of the product we have, and coverage of the product we have, and coverage of the product we have, and people are coming back. Okay, now let's people are coming back. Okay, now let's people are coming back. Okay, now let's go to GA, and then you launch through go to GA, and then you launch through go to GA, and then you launch through the GA. And then you have your next the GA. And then you have your next the GA. And then you have your next problem. problem. problem. So, I know that this is a technical So, I know that this is a technical So, I know that this is a technical conference, but this is also where a lot conference, but this is also where a lot conference, but this is also where a lot of these products fail. of these products fail. of these products fail. It's basically how do you drive change It's basically how do you drive change It's basically how do you drive change management? So, you launch your product, management? So, you launch your product, management? So, you launch your product, you are 2 weeks into the launch, and you are 2 weeks into the launch, and you are 2 weeks into the launch, and then you are here. And all your then you are here. And all your then you are here. And all your management is like disappointed or management is like disappointed or management is like disappointed or frustrated. Why aren't people using frustrated. Why aren't people using frustrated. Why aren't people using this? Why are numbers are real low? this? Why are numbers are real low? this? Why are numbers are real low? Right? And I show them this graph. Right? And I show them this graph. Right? And I show them this graph. I say that only 20% of your basically I say that only 20% of your basically I say that only 20% of your basically organization actually tried the product. organization actually tried the product. organization actually tried the product. I cannot do anything. This is not the I cannot do anything. This is not the I cannot do anything. This is not the product's fault if people are not even product's fault if people are not even product's fault if people are not even taking 5 minutes to try try the product.

taking 5 minutes to try try the product. taking 5 minutes to try try the product. Right? If they try it and if they don't Right? If they try it and if they don't Right? If they try it and if they don't come back, okay, that's my problem. come back, okay, that's my problem. come back, okay, that's my problem. Right? But if they don't try it, then we Right? But if they don't try it, then we Right? But if they don't try it, then we have another problem. have another problem. have another problem. So, the first and I've been, you know, So, the first and I've been, you know, So, the first and I've been, you know, I've seen this with many many sales I've seen this with many many sales I've seen this with many many sales organization in my past life as well and organization in my past life as well and organization in my past life as well and so on. Usually this is a couple of month so on. Usually this is a couple of month so on. Usually this is a couple of month process. And then you significantly process. And then you significantly process. And then you significantly invest in basically change management, invest in basically change management, invest in basically change management, in activation. I will spend 60 70% of my in activation. I will spend 60 70% of my in activation. I will spend 60 70% of my time in sales meetings, giving demos, time in sales meetings, giving demos, time in sales meetings, giving demos, building dashboards, which teams building dashboards, which teams building dashboards, which teams adopted, you know, shaming the like the adopted, you know, shaming the like the adopted, you know, shaming the like the managers whose team is actually doing managers whose team is actually doing managers whose team is actually doing good, getting sponsorship from sales good, getting sponsorship from sales good, getting sponsorship from sales leaders to basically like make sure that leaders to basically like make sure that leaders to basically like make sure that they you know, they push their people to they you know, they push their people to they you know, they push their people to try these things and so on. And then try these things and so on. And then try these things and so on. And then ultimately that gets your blue line up ultimately that gets your blue line up ultimately that gets your blue line up and then your questions are start coming and then your questions are start coming and then your questions are start coming up and then your focus can shift into, up and then your focus can shift into, up and then your focus can shift into, okay, how do I drive more depth? okay, how do I drive more depth? okay, how do I drive more depth? Right? Right? Right? And I want to really really emphasize And I want to really really emphasize And I want to really really emphasize this because if you hadn't done this, this because if you hadn't done this, this because if you hadn't done this, we would probably be doing, you know, we would probably be doing, you know, we would probably be doing, you know, half of where we are today. So, this is half of where we are today. So, this is half of where we are today. So, this is a very very important part. And then as a very very important part. And then as a very very important part. And then as engineers, if you spend all your effort, engineers, if you spend all your effort, engineers, if you spend all your effort, you want to have a good product, make you want to have a good product, make you want to have a good product, make sure that the activation and the change sure that the activation and the change sure that the activation and the change management is like lined up, like post management is like lined up, like post management is like lined up, like post launch of the product as well.

Now, you run into another issue. Okay, Now, you run into another issue. Okay, you are let's say that four to six you are let's say that four to six you are let's say that four to six months down the road. Right? months down the road. Right? months down the road. Right? What happens is you successfully What happens is you successfully What happens is you successfully launched the product. You are first like launched the product. You are first like launched the product. You are first like rockstars in the company. rockstars in the company. rockstars in the company. Right? People literally show you on the Right? People literally show you on the Right? People literally show you on the corridor like, "Hey, your product is corridor like, "Hey, your product is corridor like, "Hey, your product is awesome. We can talk to our data now. We awesome. We can talk to our data now. We awesome. We can talk to our data now. We don't need to wait on the queue to like, don't need to wait on the queue to like, don't need to wait on the queue to like, you know, get access to like analysts to you know, get access to like analysts to you know, get access to like analysts to answer our questions in every 2 weeks answer our questions in every 2 weeks answer our questions in every 2 weeks and so on. Right?" And after a couple of and so on. Right?" And after a couple of and so on. Right?" And after a couple of months, they start coming back to you months, they start coming back to you months, they start coming back to you with frustrations. Say, "I cannot do with frustrations. Say, "I cannot do with frustrations. Say, "I cannot do this in the product anymore. Right? I I this in the product anymore. Right? I I this in the product anymore. Right? I I would like to I mean, I saw this other would like to I mean, I saw this other would like to I mean, I saw this other AI product that does this and so on." AI product that does this and so on." AI product that does this and so on." This is what I call the collapsing of This is what I call the collapsing of This is what I call the collapsing of the wow factor. the wow factor. the wow factor. Okay? So, initially you are cool, Okay? So, initially you are cool, Okay? So, initially you are cool, but then but then but then and that becomes a habit, right? You and that becomes a habit, right? You and that becomes a habit, right? You basically change their habit and it basically change their habit and it basically change their habit and it becomes standard for them. Now, you need becomes standard for them. Now, you need becomes standard for them. Now, you need to raise the bar again. to raise the bar again. to raise the bar again. So, the journey that we usually see with So, the journey that we usually see with So, the journey that we usually see with the sales teams is like you start with the sales teams is like you start with the sales teams is like you start with talk to your data. How do we get you out talk to your data. How do we get you out talk to your data. How do we get you out of those like, you know, hundreds of of those like, you know, hundreds of of those like, you know, hundreds of dashboards situation, dependency to the dashboards situation, dependency to the dashboards situation, dependency to the analyst, and then first we'll basically analyst, and then first we'll basically analyst, and then first we'll basically like democratize the data for you so like democratize the data for you so like democratize the data for you so that you can basically talk to your that you can basically talk to your that you can basically talk to your data. data. data. Then the next wave comes with all the Then the next wave comes with all the Then the next wave comes with all the MCP connections, right? All the MCP connections, right? All the MCP connections, right? All the integrations that you are building.

integrations that you are building. integrations that you are building. Now it becomes like automate my Now it becomes like automate my Now it becomes like automate my workflows. workflows. workflows. We literally have now sellers who are We literally have now sellers who are We literally have now sellers who are going to use our agent basically to going to use our agent basically to going to use our agent basically to monitor their inbox, they monitor their monitor their inbox, they monitor their monitor their inbox, they monitor their Slack channels, you know, keep track of Slack channels, you know, keep track of Slack channels, you know, keep track of all the customer questions coming about all the customer questions coming about all the customer questions coming about like product questions, uh use the agent like product questions, uh use the agent like product questions, uh use the agent to draft responses that save that in to draft responses that save that in to draft responses that save that in Gmail, review them afterwards like send Gmail, review them afterwards like send Gmail, review them afterwards like send those things out, right? Or they those things out, right? Or they those things out, right? Or they automate their like outreach workflows automate their like outreach workflows automate their like outreach workflows and so on. Okay, that's great. Now I and so on. Okay, that's great. Now I and so on. Okay, that's great. Now I became an orchestrator, right? I'm became an orchestrator, right? I'm became an orchestrator, right? I'm basically automating my workflow basically automating my workflow basically automating my workflow workflows. workflows. workflows. Then the next thing you see start Then the next thing you see start Then the next thing you see start happening is teams, they get these like, happening is teams, they get these like, happening is teams, they get these like, you know, tool democratization, this you know, tool democratization, this you know, tool democratization, this empowerment coming to them, right? empowerment coming to them, right? empowerment coming to them, right? Because historically a lot of these Because historically a lot of these Because historically a lot of these go-to-market teams, they have been go-to-market teams, they have been go-to-market teams, they have been always in the backlog of someone, always in the backlog of someone, always in the backlog of someone, backlog of of an IT team or like trying backlog of of an IT team or like trying backlog of of an IT team or like trying to get a SaaS budget to learn and get a to get a SaaS budget to learn and get a to get a SaaS budget to learn and get a vendor on board to actually like enable vendor on board to actually like enable vendor on board to actually like enable something. something. something. And now all of a sudden And now all of a sudden And now all of a sudden they're able to build team skills. they're able to build team skills. they're able to build team skills. They're able to build like, you know, They're able to build like, you know, They're able to build like, you know, the custom dashboards that are basically the custom dashboards that are basically the custom dashboards that are basically like fully, you know, optimized for what like fully, you know, optimized for what like fully, you know, optimized for what their team needs. Are able to like their team needs. Are able to like their team needs. Are able to like deploy applications, automations, deploy applications, automations, deploy applications, automations, alerts, and things like that, right?

alerts, and things like that, right? alerts, and things like that, right? And then the comes the phase of And then the comes the phase of And then the comes the phase of hyper-personalization, hyper-personalization, hyper-personalization, right? Everyone is able to now like get right? Everyone is able to now like get right? Everyone is able to now like get everything personalized for them, not everything personalized for them, not everything personalized for them, not only for themselves, but also for their only for themselves, but also for their only for themselves, but also for their customers with living context of customers with living context of customers with living context of customers, contacts, and things like customers, contacts, and things like customers, contacts, and things like that. that. that. I think the main message I want to give I think the main message I want to give I think the main message I want to give here is here is here is if you just do the first stage, if you just do the first stage, if you just do the first stage, and if you just wait there, and if you just wait there, and if you just wait there, you will get disrupted in a month or you will get disrupted in a month or you will get disrupted in a month or two, two, two, right? Because now you already raised right? Because now you already raised right? Because now you already raised their expectations, that already became their expectations, that already became their expectations, that already became a baseline, and then they will find a baseline, and then they will find a baseline, and then they will find another product that does better than another product that does better than another product that does better than you, and right now the switch is very you, and right now the switch is very you, and right now the switch is very easy. They're going to just switch over easy. They're going to just switch over easy. They're going to just switch over night. Okay? So, you need to keep night. Okay? So, you need to keep night. Okay? So, you need to keep iterating. You need to keep that wow iterating. You need to keep that wow iterating. You need to keep that wow factor, and I cannot just rely on the factor, and I cannot just rely on the factor, and I cannot just rely on the fact that, you know, what I built so far fact that, you know, what I built so far fact that, you know, what I built so far is going to stay cool forever. And the next thing is, how do you deal And the next thing is, how do you deal with basically the changing technology? with basically the changing technology? with basically the changing technology? So, I talked to a lot of customers. So, I talked to a lot of customers. So, I talked to a lot of customers. And then, you know, it sometimes you run And then, you know, it sometimes you run And then, you know, it sometimes you run into these customers, big enterprises, into these customers, big enterprises, into these customers, big enterprises, very big brands. And then they are still very big brands. And then they are still very big brands. And then they are still trying to purchase that perfect trying to purchase that perfect trying to purchase that perfect architecture. architecture. architecture. They're trying to like test different They're trying to like test different They're trying to like test different frameworks. They're trying to see how frameworks. They're trying to see how frameworks. They're trying to see how the, you know, the technology is the, you know, the technology is the, you know, the technology is maturing and everything and so on. But, maturing and everything and so on. But, maturing and everything and so on. But, the thing that they don't do is they the thing that they don't do is they the thing that they don't do is they don't build, and then they don't launch, don't build, and then they don't launch, don't build, and then they don't launch, and they don't learn.

and they don't learn. and they don't learn. Right? All these blue boxes that you see Right? All these blue boxes that you see Right? All these blue boxes that you see here, those are all the things we added here, those are all the things we added here, those are all the things we added after the launch. after the launch. after the launch. Right? When we literally first launched Right? When we literally first launched Right? When we literally first launched the agent, it was a nine-page long agent the agent, it was a nine-page long agent the agent, it was a nine-page long agent instructions. instructions. instructions. It was couple of Cortex analyst tools, It was couple of Cortex analyst tools, It was couple of Cortex analyst tools, semantic views. It was a Cortex search semantic views. It was a Cortex search semantic views. It was a Cortex search service for our unstructured data. And service for our unstructured data. And service for our unstructured data. And we were managing the agent instructions we were managing the agent instructions we were managing the agent instructions versions out of a Google Doc. That's how versions out of a Google Doc. That's how versions out of a Google Doc. That's how we launched it. To 6,000 people. Right? we launched it. To 6,000 people. Right? we launched it. To 6,000 people. Right? Now we realized, okay, it's not going to Now we realized, okay, it's not going to Now we realized, okay, it's not going to work out. Let's figure out CICD. It's work out. Let's figure out CICD. It's work out. Let's figure out CICD. It's not going to work out. Let's figure out not going to work out. Let's figure out not going to work out. Let's figure out our basically eval infrastructure with our basically eval infrastructure with our basically eval infrastructure with all the like the unit test, routing all the like the unit test, routing all the like the unit test, routing test, and everything. Right? Then we test, and everything. Right? Then we test, and everything. Right? Then we start basically like coming to a point start basically like coming to a point start basically like coming to a point where, for example, we were creating all where, for example, we were creating all where, for example, we were creating all these like business processes and these like business processes and these like business processes and workflows. We couldn't fit them into the workflows. We couldn't fit them into the workflows. We couldn't fit them into the agent instructions anymore. And then the agent instructions anymore. And then the agent instructions anymore. And then the skills came, and we were like, "Oh, skills came, and we were like, "Oh, skills came, and we were like, "Oh, perfect. Let's build a skill library." perfect. Let's build a skill library." perfect. Let's build a skill library." You know, then the MCPs came. Perfect. You know, then the MCPs came. Perfect. You know, then the MCPs came. Perfect. But now, like we have to put bunch of But now, like we have to put bunch of But now, like we have to put bunch of other instructions to basically other instructions to basically other instructions to basically orchestrate that, we hit the limits on orchestrate that, we hit the limits on orchestrate that, we hit the limits on the agent instructions. What do we do? the agent instructions. What do we do? the agent instructions. What do we do? Okay, let's do the progressive Okay, let's do the progressive Okay, let's do the progressive disclosures. disclosures. disclosures. Right? And then user memory comes, task Right? And then user memory comes, task Right? And then user memory comes, task scheduling comes. We want to go beyond scheduling comes. We want to go beyond scheduling comes. We want to go beyond the chat screen and then, you know, chat the chat screen and then, you know, chat the chat screen and then, you know, chat interface and start doing the Slack interface and start doing the Slack interface and start doing the Slack interface and things like that. If I interface and things like that. If I interface and things like that. If I look at the PRD and the architectural look at the PRD and the architectural look at the PRD and the architectural diagram we wrote in the beginning of the diagram we wrote in the beginning of the diagram we wrote in the beginning of the project, if I compare to this project, if I compare to this project, if I compare to this architecture we have now, 80% of it It architecture we have now, 80% of it It architecture we have now, 80% of it It match.

match. match. Okay? So, like if you look at our Okay? So, like if you look at our Okay? So, like if you look at our sprints, sprints, sprints, like maybe 60-70% of the work we are like maybe 60-70% of the work we are like maybe 60-70% of the work we are doing is adding new features, improving doing is adding new features, improving doing is adding new features, improving quality, and all kind of things. quality, and all kind of things. quality, and all kind of things. But 30-40% of the work is that we are But 30-40% of the work is that we are But 30-40% of the work is that we are constantly re-architecting with the new constantly re-architecting with the new constantly re-architecting with the new technology. technology. technology. So, this is a time where like you need So, this is a time where like you need So, this is a time where like you need to get your hands dirty, you need to run to get your hands dirty, you need to run to get your hands dirty, you need to run with the new technology, and then you with the new technology, and then you with the new technology, and then you shouldn't be like, you know, too much shouldn't be like, you know, too much shouldn't be like, you know, too much tied to your architecture. You should be tied to your architecture. You should be tied to your architecture. You should be okay to like pivot very easily, so that okay to like pivot very easily, so that okay to like pivot very easily, so that you can basically double on down on you can basically double on down on you can basically double on down on these like new capabilities and things these like new capabilities and things these like new capabilities and things like that. like that. like that. And then the longer you wait, the more, And then the longer you wait, the more, And then the longer you wait, the more, you know, you lose towards your you know, you lose towards your you know, you lose towards your competition, because if your competition competition, because if your competition competition, because if your competition is doing these kind of things like 3-4 is doing these kind of things like 3-4 is doing these kind of things like 3-4 months ahead of you, right? That means months ahead of you, right? That means months ahead of you, right? That means that they're also getting more that they're also getting more that they're also getting more customers. Um last thing is I would really, really Um last thing is I would really, really recommend investing in your logs. Okay? recommend investing in your logs. Okay? recommend investing in your logs. Okay? Because they create the basically the Because they create the basically the Because they create the basically the feedback loop. feedback loop. feedback loop. So, first of all, technically it's very So, first of all, technically it's very So, first of all, technically it's very fun. Okay? So, you basically use LLMs to fun. Okay? So, you basically use LLMs to fun. Okay? So, you basically use LLMs to like classify your logs and things like like classify your logs and things like like classify your logs and things like that. As I said, like we have 1.2 that. As I said, like we have 1.2 that. As I said, like we have 1.2 million questions, we get 40,000 million questions, we get 40,000 million questions, we get 40,000 questions every week. It's technically questions every week. It's technically questions every week. It's technically very fun, you know, how you do that at very fun, you know, how you do that at very fun, you know, how you do that at scale without breaking the bank and so scale without breaking the bank and so scale without breaking the bank and so on. You know, our data scientists love on. You know, our data scientists love on. You know, our data scientists love working on those things, and then they working on those things, and then they working on those things, and then they really experiment with new things.

really experiment with new things. really experiment with new things. But as a result of that, what we get is But as a result of that, what we get is But as a result of that, what we get is we get a very extremely detailed we get a very extremely detailed we get a very extremely detailed breakdown of topics and, you know, breakdown of topics and, you know, breakdown of topics and, you know, things that we are having. I'm just to things that we are having. I'm just to things that we are having. I'm just to showing you the top category showing you the top category showing you the top category categorization level there, but then categorization level there, but then categorization level there, but then basically we are able to track like, you basically we are able to track like, you basically we are able to track like, you know, what kind of questions they are know, what kind of questions they are know, what kind of questions they are asking. We are able to break down each asking. We are able to break down each asking. We are able to break down each of those categories to subcategories, of those categories to subcategories, of those categories to subcategories, you know, they are able to get like you know, they are able to get like you know, they are able to get like detailed example questions, this and detailed example questions, this and detailed example questions, this and that, and so on. that, and so on. that, and so on. All good, but how do we use that? Then All good, but how do we use that? Then All good, but how do we use that? Then we start creating the basically the we start creating the basically the we start creating the basically the feedback loops. feedback loops. feedback loops. Right? I know, I mean, I still Right? I know, I mean, I still Right? I know, I mean, I still interview, of course, users, but now I interview, of course, users, but now I interview, of course, users, but now I see in real time what my feature gaps see in real time what my feature gaps see in real time what my feature gaps are. are. are. I'm clearly seeing what people are I'm clearly seeing what people are I'm clearly seeing what people are asking and we are not able to answer or asking and we are not able to answer or asking and we are not able to answer or where we have a like a quality issue, where we have a like a quality issue, where we have a like a quality issue, where they are swearing at the agent or where they are swearing at the agent or where they are swearing at the agent or like at repeating their question, so like at repeating their question, so like at repeating their question, so that we see where to improve. that we see where to improve. that we see where to improve. For sales enablement is a goldmine. For sales enablement is a goldmine. For sales enablement is a goldmine. Let's say that we launch a new product. Let's say that we launch a new product. Let's say that we launch a new product. Usually, you know, they would need to Usually, you know, they would need to Usually, you know, they would need to interview maybe 100 sellers a week to be interview maybe 100 sellers a week to be interview maybe 100 sellers a week to be able to understand like how basically able to understand like how basically able to understand like how basically the you know, the topics are changing the you know, the topics are changing the you know, the topics are changing where there's gaps in terms of like where there's gaps in terms of like where there's gaps in terms of like knowledge documents, battle cards. I see knowledge documents, battle cards. I see knowledge documents, battle cards. I see that in real time in a minute or two by that in real time in a minute or two by that in real time in a minute or two by just asking an element question. And just asking an element question. And just asking an element question. And then we can then, you know, connect to then we can then, you know, connect to then we can then, you know, connect to Confluence, we can connect to Jira, we Confluence, we can connect to Jira, we Confluence, we can connect to Jira, we can connect to Slack channels, we can can connect to Slack channels, we can can connect to Slack channels, we can ingest the PRDs, and in couple of ingest the PRDs, and in couple of ingest the PRDs, and in couple of minutes we can actually like, you know, minutes we can actually like, you know, minutes we can actually like, you know, generate battle cards, sales enablement generate battle cards, sales enablement generate battle cards, sales enablement document and then feed it back into the document and then feed it back into the document and then feed it back into the agent.

agent. agent. Right? I mean, you cannot do that kind Right? I mean, you cannot do that kind Right? I mean, you cannot do that kind of a like a feedback loop with humans, of a like a feedback loop with humans, of a like a feedback loop with humans, right? So then you we can basically right? So then you we can basically right? So then you we can basically automate these kind of things. automate these kind of things. automate these kind of things. Um within the sales organization, there Um within the sales organization, there Um within the sales organization, there will be different teams that are good will be different teams that are good will be different teams that are good trying to connect each other. They are trying to connect each other. They are trying to connect each other. They are trying to maybe target similar accounts trying to maybe target similar accounts trying to maybe target similar accounts from different angles. They don't know from different angles. They don't know from different angles. They don't know about each other. We do. We are now able about each other. We do. We are now able about each other. We do. We are now able to ping them. And then we are able to to ping them. And then we are able to to ping them. And then we are able to basically do matchmaking. basically do matchmaking. basically do matchmaking. Right? I'm just giving you couple of Right? I'm just giving you couple of Right? I'm just giving you couple of examples, but this is also one of those examples, but this is also one of those examples, but this is also one of those areas where like you start building your areas where like you start building your areas where like you start building your AI platform, you start building your AI platform, you start building your AI platform, you start building your architecture. architecture. architecture. The first features are difficult to get The first features are difficult to get The first features are difficult to get out. The next ones are easy. And then out. The next ones are easy. And then out. The next ones are easy. And then once you start tapping into your logs, once you start tapping into your logs, once you start tapping into your logs, this this like hockey stick exponential this this like hockey stick exponential this this like hockey stick exponential thing actually starts happening and it's thing actually starts happening and it's thing actually starts happening and it's magical. So, if you were to take a couple of So, if you were to take a couple of things from this talk, things from this talk, things from this talk, like quality over coverage. like quality over coverage. like quality over coverage. I'm very, very like religious about I'm very, very like religious about I'm very, very like religious about this. this. this. If you go for the coverage, If you go for the coverage, If you go for the coverage, you are going to shoot yourself in the you are going to shoot yourself in the you are going to shoot yourself in the foot. foot. foot. Okay? Okay? Okay? Change management. A lot of engineers Change management. A lot of engineers Change management. A lot of engineers doesn't think about this, right? A lot doesn't think about this, right? A lot doesn't think about this, right? A lot of these AI initiatives, they don't fail of these AI initiatives, they don't fail of these AI initiatives, they don't fail because there's an issue with the because there's an issue with the because there's an issue with the technology, there's an issue with that technology, there's an issue with that technology, there's an issue with that as assuming you did the first one, as assuming you did the first one, as assuming you did the first one, right? Right? So, they fail actually in right? Right? So, they fail actually in right? Right? So, they fail actually in activation.

activation. activation. So, make sure that you have a plan for So, make sure that you have a plan for So, make sure that you have a plan for that, especially in larger organizations that, especially in larger organizations that, especially in larger organizations where we are dealing with like 6,000 where we are dealing with like 6,000 where we are dealing with like 6,000 go-to-market users, right? go-to-market users, right? go-to-market users, right? Um again, don't forget this concept of Um again, don't forget this concept of Um again, don't forget this concept of collapsing law factor. collapsing law factor. collapsing law factor. You cannot stay where you are. You You cannot stay where you are. You You cannot stay where you are. You cannot just say that, "Hey, I did an cannot just say that, "Hey, I did an cannot just say that, "Hey, I did an innovation. I'm going to surf that for a innovation. I'm going to surf that for a innovation. I'm going to surf that for a year." year." year." You know, every time people are happy, You know, every time people are happy, You know, every time people are happy, you should be paranoid. You should be you should be paranoid. You should be you should be paranoid. You should be like, "Okay, what am I going to show like, "Okay, what am I going to show like, "Okay, what am I going to show them in a month or two now?" them in a month or two now?" them in a month or two now?" How do I basically keep that excitement How do I basically keep that excitement How do I basically keep that excitement going on? going on? going on? Right? Right? Right? Build fast with today's stack. Like, Build fast with today's stack. Like, Build fast with today's stack. Like, don't try to invest in these like high, don't try to invest in these like high, don't try to invest in these like high, you know, super like plat, you know, you know, super like plat, you know, you know, super like plat, you know, architectures and have these like 6 9 architectures and have these like 6 9 architectures and have these like 6 9 months of long projects and things like months of long projects and things like months of long projects and things like that. How do you turn around these that. How do you turn around these that. How do you turn around these things in weeks, days, and so on? things in weeks, days, and so on? things in weeks, days, and so on? And just be comfortable with the fact And just be comfortable with the fact And just be comfortable with the fact that you are constantly going to be that you are constantly going to be that you are constantly going to be re-architecting. That's fine. re-architecting. That's fine. re-architecting. That's fine. Right? Right? Right? Just don't over-invest in the current Just don't over-invest in the current Just don't over-invest in the current architecture. Just make sure that you architecture. Just make sure that you architecture. Just make sure that you keep your like flexibility out there. keep your like flexibility out there. keep your like flexibility out there. Um and then the feedback loops. I think Um and then the feedback loops. I think Um and then the feedback loops. I think that's what kind of like gives you that's what kind of like gives you that's what kind of like gives you really that like, you know, the really that like, you know, the really that like, you know, the incremental part of like that hockey incremental part of like that hockey incremental part of like that hockey stick exponential part of the thing.

stick exponential part of the thing. stick exponential part of the thing. Um we constantly publish like blog Um we constantly publish like blog Um we constantly publish like blog posts, I mean, where we kind of like try posts, I mean, where we kind of like try posts, I mean, where we kind of like try to have our, you know, learnings shared to have our, you know, learnings shared to have our, you know, learnings shared with uh our customers and so on. Like, with uh our customers and so on. Like, with uh our customers and so on. Like, we have blog posts on like how we do we have blog posts on like how we do we have blog posts on like how we do agent instructions, how we do our agent instructions, how we do our agent instructions, how we do our structured data with semantic views, you structured data with semantic views, you structured data with semantic views, you know, how we basically build our know, how we basically build our know, how we basically build our rack-based like knowledge assistants, rack-based like knowledge assistants, rack-based like knowledge assistants, the non-technical side of the story, the non-technical side of the story, the non-technical side of the story, like how do you derive change like how do you derive change like how do you derive change management, and so on. So, feel free to management, and so on. So, feel free to management, and so on. So, feel free to check those. check those. check those. Um and yeah, I think that's the end of Um and yeah, I think that's the end of Um and yeah, I think that's the end of my talk. my talk. my talk. Okay. >> We have time for one question. >> We have time for one question. Okay, there you go. >> Thanks for the talk. Um I don't know if >> Thanks for the talk. Um I don't know if you already said this, but I saw in the you already said this, but I saw in the you already said this, but I saw in the the titles of the articles Snowflake the titles of the articles Snowflake the titles of the articles Snowflake Intelligence. Is that an underlying Intelligence. Is that an underlying Intelligence. Is that an underlying context or layer that the tool or system context or layer that the tool or system context or layer that the tool or system you built was on top of, or was that you built was on top of, or was that you built was on top of, or was that the tool itself or something else? the tool itself or something else? the tool itself or something else? >> Yeah, Snowflake Intelligence, we renamed >> Yeah, Snowflake Intelligence, we renamed >> Yeah, Snowflake Intelligence, we renamed that to Snowflake Co-work a couple of that to Snowflake Co-work a couple of that to Snowflake Co-work a couple of weeks ago in our summit. That's weeks ago in our summit. That's weeks ago in our summit. That's basically our no-code agent platform basically our no-code agent platform basically our no-code agent platform that we basically build have available that we basically build have available that we basically build have available for our business users.

for our business users. for our business users. I mean, the advantage of that is that I mean, the advantage of that is that I mean, the advantage of that is that all of these tools are on like, you all of these tools are on like, you all of these tools are on like, you know, know, know, you know, Cortex analyst or Cortex you know, Cortex analyst or Cortex you know, Cortex analyst or Cortex search or Cortex sense, a lot of those search or Cortex sense, a lot of those search or Cortex sense, a lot of those things are basically comes out of the things are basically comes out of the things are basically comes out of the box. box. box. We made a strategic choice for our We made a strategic choice for our We made a strategic choice for our internal thing where we said that, internal thing where we said that, internal thing where we said that, "Look, it is important that we bring all "Look, it is important that we bring all "Look, it is important that we bring all our data together." And we do that in our data together." And we do that in our data together." And we do that in Snowflake. We bring all the first-party, Snowflake. We bring all the first-party, Snowflake. We bring all the first-party, the third-party data, all the Salesforce the third-party data, all the Salesforce the third-party data, all the Salesforce data, everything, the call transcripts, data, everything, the call transcripts, data, everything, the call transcripts, and so on, all together. And then these and so on, all together. And then these and so on, all together. And then these agents then can basically basically agents then can basically basically agents then can basically basically inherit a lot of the role-based access inherit a lot of the role-based access inherit a lot of the role-based access controls and so on. And I literally can controls and so on. And I literally can controls and so on. And I literally can deploy these agents without writing a deploy these agents without writing a deploy these agents without writing a single line of code, right? single line of code, right? single line of code, right? And then, you know, you don't need to And then, you know, you don't need to And then, you know, you don't need to worry about the UI, the chat UI comes worry about the UI, the chat UI comes worry about the UI, the chat UI comes out of the box, and so on. And then we out of the box, and so on. And then we out of the box, and so on. And then we have been the customer zero of that like have been the customer zero of that like have been the customer zero of that like internally to build this ourselves. And internally to build this ourselves. And internally to build this ourselves. And then, you know, our customers are able then, you know, our customers are able then, you know, our customers are able to go and then build similar things to go and then build similar things to go and then build similar things basically on Snowflake over platform as basically on Snowflake over platform as basically on Snowflake over platform as well. And it comes with the guardrails well. And it comes with the guardrails well. And it comes with the guardrails and things where you don't really need and things where you don't really need and things where you don't really need to worry about them going very, you to worry about them going very, you to worry about them going very, you know, know, know, crazy on, you know, what data sources to crazy on, you know, what data sources to crazy on, you know, what data sources to do things, and so on. So we are able to do things, and so on. So we are able to do things, and so on. So we are able to do a lot of curation. We are able to do do a lot of curation. We are able to do do a lot of curation. We are able to do a lot of security guardrails in there as a lot of security guardrails in there as a lot of security guardrails in there as well.