← Back
AI Engineer August 14, 2026 22m

The Rise of CaaS: Context-as-a-Service for Agentic AI — Omer Primor, Bright Data

Read full transcript 17 segments
  1. So hi everyone. Thank you so much for So hi everyone. Thank you so much for taking the time to join this session. I taking the time to join this session. I taking the time to join this session. I hope I'll or at least I can guarantee hope I'll or at least I can guarantee hope I'll or at least I can guarantee I'll do whatever it takes uh to make it I'll do whatever it takes uh to make it I'll do whatever it takes uh to make it worth your time. My name is Om. I lead worth your time. My name is Om. I lead worth your time. My name is Om. I lead the product marketing team over at the product marketing team over at the product marketing team over at Bright Data. Just by maybe a quick show Bright Data. Just by maybe a quick show Bright Data. Just by maybe a quick show of hands, who here is familiar with of hands, who here is familiar with of hands, who here is familiar with Bright Data? Bright Data? Bright Data? Okay, we can do better. I'll pass it on Okay, we can do better. I'll pass it on Okay, we can do better. I'll pass it on to our brand team. Bright Data is a web to our brand team. Bright Data is a web to our brand team. Bright Data is a web data company. Basically, we help more data company. Basically, we help more data company. Basically, we help more than 20,000 teams around the world, than 20,000 teams around the world, than 20,000 teams around the world, including more than 70% of the world's including more than 70% of the world's including more than 70% of the world's biggest AI labs to extract data from the biggest AI labs to extract data from the biggest AI labs to extract data from the web. Just to put this in perspective of web. Just to put this in perspective of web. Just to put this in perspective of what scale we're talking about, we're what scale we're talking about, we're what scale we're talking about, we're talking well over 50 billion pages, talking well over 50 billion pages, talking well over 50 billion pages, HTMLs every day. More than 20 pabytes of HTMLs every day. More than 20 pabytes of HTMLs every day. More than 20 pabytes of video, audio, and other media data. So video, audio, and other media data. So video, audio, and other media data. So that's just a perspective of the type of that's just a perspective of the type of that's just a perspective of the type of work that we do at Bright Data. work that we do at Bright Data. work that we do at Bright Data. But enough about us. Personally, I But enough about us. Personally, I But enough about us. Personally, I joined Bright Data about three years joined Bright Data about three years joined Bright Data about three years ago, which essentially gave me front row ago, which essentially gave me front row ago, which essentially gave me front row seats at everything around AI and the seats at everything around AI and the seats at everything around AI and the web and how they started to actually web and how they started to actually web and how they started to actually connect. It sounds very old but if you connect. It sounds very old but if you connect. It sounds very old but if you think about it only maybe less than two think about it only maybe less than two think about it only maybe less than two years ago right we were able to start years ago right we were able to start years ago right we were able to start using access the web and search the web using access the web and search the web using access the web and search the web through cloud or through uh chpt that through cloud or through uh chpt that through cloud or through uh chpt that option didn't even exist in the earlier option didn't even exist in the earlier option didn't even exist in the earlier versions right so that connection that versions right so that connection that versions right so that connection that way in which both AI and the web are way in which both AI and the web are way in which both AI and the web are starting to converge is something that starting to converge is something that starting to converge is something that is still evolving and evolving rapidly is still evolving and evolving rapidly is still evolving and evolving rapidly and that's part of what I want to try and that's part of what I want to try and that's part of what I want to try and shed some light on today and talk and shed some light on today and talk and shed some light on today and talk about a new emerging breed of companies.

  2. about a new emerging breed of companies. about a new emerging breed of companies. That's coming out of this connection. That's coming out of this connection. That's coming out of this connection. I think we can basically agree that the I think we can basically agree that the I think we can basically agree that the web is by far the world's greatest web is by far the world's greatest web is by far the world's greatest source of data. At least historically source of data. At least historically source of data. At least historically when it comes to bright air, that's all when it comes to bright air, that's all when it comes to bright air, that's all we cared about, right? It's helping our we cared about, right? It's helping our we cared about, right? It's helping our customers extract data from the web. But customers extract data from the web. But customers extract data from the web. But with the emergence of AI and the with the emergence of AI and the with the emergence of AI and the emergence more recently of of AI agents emergence more recently of of AI agents emergence more recently of of AI agents that need to do knowledge work, right? that need to do knowledge work, right? that need to do knowledge work, right? The web is no longer just a source of The web is no longer just a source of The web is no longer just a source of data. We can actually start looking at data. We can actually start looking at data. We can actually start looking at it as a source of context. context in it as a source of context. context in it as a source of context. context in the sense that if I do knowledge work the sense that if I do knowledge work the sense that if I do knowledge work and I have knowledge agents that support and I have knowledge agents that support and I have knowledge agents that support my work I want to go out to the web find my work I want to go out to the web find my work I want to go out to the web find the information that I need use it as the information that I need use it as the information that I need use it as context but kept keep on working so the context but kept keep on working so the context but kept keep on working so the data itself is only a step in the data itself is only a step in the data itself is only a step in the process for something bigger for the process for something bigger for the process for something bigger for the actions I need to take for the actions I need to take for the actions I need to take for the conclusions I need to draw and for every conclusions I need to draw and for every conclusions I need to draw and for every downstream application that follows downstream application that follows downstream application that follows uh the first one's to figure this one uh the first one's to figure this one uh the first one's to figure this one out oh sorry even before that but one out oh sorry even before that but one out oh sorry even before that but one thing that I want all of us to bear in thing that I want all of us to bear in thing that I want all of us to bear in mind because this is going to follow us mind because this is going to follow us mind because this is going to follow us through the rest of this conversation through the rest of this conversation through the rest of this conversation The web is messy. It's unstructured. And The web is messy. It's unstructured. And The web is messy. It's unstructured. And most importantly, it changes all the most importantly, it changes all the most importantly, it changes all the time. This is a chart that shows data time. This is a chart that shows data time. This is a chart that shows data decay, right? That it's an analysis done decay, right? That it's an analysis done decay, right? That it's an analysis done by our team that shows data decay.

  3. by our team that shows data decay. by our team that shows data decay. Basically, how long after a new page, a Basically, how long after a new page, a Basically, how long after a new page, a new piece of content goes live, it is no new piece of content goes live, it is no new piece of content goes live, it is no longer relevant, right? So, social longer relevant, right? So, social longer relevant, right? So, social media, it's easy for us to understand. media, it's easy for us to understand. media, it's easy for us to understand. It's far less than a day. But also news, It's far less than a day. But also news, It's far less than a day. But also news, finance, retail, 30 days later, data finance, retail, 30 days later, data finance, retail, 30 days later, data that was collected, it's mostly no that was collected, it's mostly no that was collected, it's mostly no longer relevant. And when we acknowledge longer relevant. And when we acknowledge longer relevant. And when we acknowledge that this simple notion, we understand that this simple notion, we understand that this simple notion, we understand that extracting context from the web or that extracting context from the web or that extracting context from the web or relying on the web is not a snapshot. relying on the web is not a snapshot. relying on the web is not a snapshot. It's not a one-time effort. It's not It's not a one-time effort. It's not It's not a one-time effort. It's not even a monthly effort. It's something even a monthly effort. It's something even a monthly effort. It's something that we need to keep on doing. It's that we need to keep on doing. It's that we need to keep on doing. It's something that we need to look at as an something that we need to look at as an something that we need to look at as an ongoing process and something that we ongoing process and something that we ongoing process and something that we need to be mindful of. need to be mindful of. need to be mindful of. The first ones to figure it out were of The first ones to figure it out were of The first ones to figure it out were of course search companies, right? Only course search companies, right? Only course search companies, right? Only what three years ago we were on the far what three years ago we were on the far what three years ago we were on the far left, right? This is this is in our left, right? This is this is in our left, right? This is this is in our lifetime, right? Three years ago, we lifetime, right? Three years ago, we lifetime, right? Three years ago, we were on the far left. Everything was were on the far left. Everything was were on the far left. Everything was Google. There was no there was complete Google. There was no there was complete Google. There was no there was complete and total dominance up until three years and total dominance up until three years and total dominance up until three years ago for the past 20 or so years, right? ago for the past 20 or so years, right? ago for the past 20 or so years, right? That's what we're talking about purely That's what we're talking about purely That's what we're talking about purely for humans. Search something, go collect for humans. Search something, go collect for humans. Search something, go collect the information you need, and carry on.

  4. the information you need, and carry on. the information you need, and carry on. Then fast forward maybe one and a half Then fast forward maybe one and a half Then fast forward maybe one and a half years ago, two years ago, search began years ago, two years ago, search began years ago, two years ago, search began to appear within the LLN within the chat to appear within the LLN within the chat to appear within the LLN within the chat bots, right? Which already started bots, right? Which already started bots, right? Which already started blurring the line between humans and and blurring the line between humans and and blurring the line between humans and and agents because now the same bots also agents because now the same bots also agents because now the same bots also have the same web search available. So have the same web search available. So have the same web search available. So the same LMS have the same access the same LMS have the same access the same LMS have the same access through API for the bot. So we started through API for the bot. So we started through API for the bot. So we started seeing that convergence happening, seeing that convergence happening, seeing that convergence happening, right? And so for the first time, we're right? And so for the first time, we're right? And so for the first time, we're seeing more and more traffic flowing seeing more and more traffic flowing seeing more and more traffic flowing down, search traffic, search intent down, search traffic, search intent down, search traffic, search intent flowing down these channels, not only to flowing down these channels, not only to flowing down these channels, not only to Google. And last but not least, right, Google. And last but not least, right, Google. And last but not least, right, we now see a whole breed of companies, we now see a whole breed of companies, we now see a whole breed of companies, the AI search companies. I'm sure you're the AI search companies. I'm sure you're the AI search companies. I'm sure you're familiar with them. I caught a talk familiar with them. I caught a talk familiar with them. I caught a talk yesterday by Will, the CEO of Exa and yesterday by Will, the CEO of Exa and yesterday by Will, the CEO of Exa and Parallel and you.com and Tavili and a Parallel and you.com and Tavili and a Parallel and you.com and Tavili and a bunch of others. They are purely built bunch of others. They are purely built bunch of others. They are purely built and indexing the web especially for and indexing the web especially for and indexing the web especially for agents. They're not even looking at the agents. They're not even looking at the agents. They're not even looking at the humans involved anymore. humans involved anymore. humans involved anymore. So Google's dominance when it comes to So Google's dominance when it comes to So Google's dominance when it comes to search, if Google was synonymous of web search, if Google was synonymous of web search, if Google was synonymous of web search, that is very much shaking. When search, that is very much shaking. When search, that is very much shaking. When there's blood in the water, the sharks there's blood in the water, the sharks there's blood in the water, the sharks come just last week. Amazon come just last week. Amazon come just last week. Amazon Amazon announced, I don't know how many Amazon announced, I don't know how many Amazon announced, I don't know how many of you saw it, that they developed their of you saw it, that they developed their of you saw it, that they developed their own index and started allowing the own index and started allowing the own index and started allowing the ability to retrieve data from the web to ability to retrieve data from the web to ability to retrieve data from the web to retrieve contacts for agents on agent retrieve contacts for agents on agent retrieve contacts for agents on agent core. Amazon developed their own search core. Amazon developed their own search core. Amazon developed their own search engine.

  5. engine. engine. Two weeks before that it was Microsoft. Two weeks before that it was Microsoft. Two weeks before that it was Microsoft. Microsoft always had skin in the game. I Microsoft always had skin in the game. I Microsoft always had skin in the game. I know one or two% of the world search know one or two% of the world search know one or two% of the world search traffic went to Microsoft but they have traffic went to Microsoft but they have traffic went to Microsoft but they have now repackaged it and launched it again now repackaged it and launched it again now repackaged it and launched it again as part of web byq as part of their as part of web byq as part of their as part of web byq as part of their suite for agentic development and suite for agentic development and suite for agentic development and orchestration. orchestration. orchestration. So we're seeing more and more this space So we're seeing more and more this space So we're seeing more and more this space becoming crowded. becoming crowded. becoming crowded. But when we're talking about context and But when we're talking about context and But when we're talking about context and we're looking at this through the lens we're looking at this through the lens we're looking at this through the lens of search, I believe it only tells us of search, I believe it only tells us of search, I believe it only tells us part of the story, right? I can search part of the story, right? I can search part of the story, right? I can search for, I don't know, what's the cost of a for, I don't know, what's the cost of a for, I don't know, what's the cost of a certain pair of sneakers this morning, certain pair of sneakers this morning, certain pair of sneakers this morning, right? In on a certain website. I cannot right? In on a certain website. I cannot right? In on a certain website. I cannot really search for how has that price really search for how has that price really search for how has that price changed over the last six months, what changed over the last six months, what changed over the last six months, what discounts it had, right? I can search discounts it had, right? I can search discounts it had, right? I can search for what open job positions we have at for what open job positions we have at for what open job positions we have at Bright Data. We do. I urge you to go Bright Data. We do. I urge you to go Bright Data. We do. I urge you to go have a look but I can't see how how that have a look but I can't see how how that have a look but I can't see how how that that was a chart and how that changed that was a chart and how that changed that was a chart and how that changed over time and how the headcount of the over time and how the headcount of the over time and how the headcount of the company changed over time and all of company changed over time and all of company changed over time and all of that information existed on the web that information existed on the web that information existed on the web simply back then. So when we actually simply back then. So when we actually simply back then. So when we actually start to think about it we understand start to think about it we understand start to think about it we understand that there's much more context in the that there's much more context in the that there's much more context in the web that than what web search allows us web that than what web search allows us web that than what web search allows us to extract and this is what we started to extract and this is what we started to extract and this is what we started seeing in the recent years a whole new seeing in the recent years a whole new seeing in the recent years a whole new breed of companies rising. We like to breed of companies rising. We like to breed of companies rising. We like to call them internally casts, context as a call them internally casts, context as a call them internally casts, context as a service because that's what they do.

  6. service because that's what they do. service because that's what they do. They allow agents to tap into them MCP, They allow agents to tap into them MCP, They allow agents to tap into them MCP, CLI, uh just pure good old API and CLI, uh just pure good old API and CLI, uh just pure good old API and actually start extracting data to to actually start extracting data to to actually start extracting data to to retrieve data so they can reason over retrieve data so they can reason over retrieve data so they can reason over for whatever knowledge work they are for whatever knowledge work they are for whatever knowledge work they are responsible for. We see this happening responsible for. We see this happening responsible for. We see this happening in e-commerce. We see this happening in in e-commerce. We see this happening in in e-commerce. We see this happening in travel. We see this happening in in travel. We see this happening in in travel. We see this happening in in finance, in market research, in uh in finance, in market research, in uh in finance, in market research, in uh in HR, HR, HR, in real estate, in a bunch of other in real estate, in a bunch of other in real estate, in a bunch of other domains. I'll show a few examples in a domains. I'll show a few examples in a domains. I'll show a few examples in a second. Right? What all of these have in second. Right? What all of these have in second. Right? What all of these have in common is that they don't only just common is that they don't only just common is that they don't only just discover the web, you know, in terms of discover the web, you know, in terms of discover the web, you know, in terms of think crawling, think searching, think think crawling, think searching, think think crawling, think searching, think all of that, accessing, extracting the all of that, accessing, extracting the all of that, accessing, extracting the data and indexing it. They take it a data and indexing it. They take it a data and indexing it. They take it a step further. They actually develop step further. They actually develop step further. They actually develop knowledge graphs to start structuring knowledge graphs to start structuring knowledge graphs to start structuring all of the entities and to ddup them and all of the entities and to ddup them and all of the entities and to ddup them and they start enriching them with a lot of they start enriching them with a lot of they start enriching them with a lot of different sources. So they actually different sources. So they actually different sources. So they actually start merging all that data. If you start merging all that data. If you start merging all that data. If you think about it, they kind of behave like think about it, they kind of behave like think about it, they kind of behave like vertical search engines, right? they are vertical search engines, right? they are vertical search engines, right? they are a very very very good search engine for a very very very good search engine for a very very very good search engine for something very specific something very specific something very specific and and it's it's already in full motion and and it's it's already in full motion and and it's it's already in full motion right so as I said we see this in right so as I said we see this in right so as I said we see this in finance and in market research and finance and in market research and finance and in market research and retail and e-commerce and GTM and sales retail and e-commerce and GTM and sales retail and e-commerce and GTM and sales intelligence what all of these companies intelligence what all of these companies intelligence what all of these companies by the way have in common they're all by the way have in common they're all by the way have in common they're all part of brighta's startup program if you part of brighta's startup program if you part of brighta's startup program if you are a builder and this is a hot space to are a builder and this is a hot space to are a builder and this is a hot space to go in because I think we're only tapping go in because I think we're only tapping go in because I think we're only tapping the surface I invite you to scan this the surface I invite you to scan this the surface I invite you to scan this and apply up to $20,000 in credits and and apply up to $20,000 in credits and and apply up to $20,000 in credits and all sorts of co-arketing but that's all sorts of co-arketing but that's all sorts of co-arketing but that's enough self-promotion.

  7. So CAS as as an industry is already in So CAS as as an industry is already in full bloom and as always with these full bloom and as always with these full bloom and as always with these situations right also the traditional situations right also the traditional situations right also the traditional players aren't left too much behind players aren't left too much behind players aren't left too much behind there at the bottom you see good old there at the bottom you see good old there at the bottom you see good old data as a service you see zoom info data as a service you see zoom info data as a service you see zoom info right by researching for this uh right by researching for this uh right by researching for this uh presentation today I also saw that they presentation today I also saw that they presentation today I also saw that they launched that thing at the top it's launched that thing at the top it's launched that thing at the top it's called gtm.ai AI. You can only imagine called gtm.ai AI. You can only imagine called gtm.ai AI. You can only imagine how much they paid for that domain. But how much they paid for that domain. But how much they paid for that domain. But they launched as a secondary brand for they launched as a secondary brand for they launched as a secondary brand for Zoom Info that is catering specifically Zoom Info that is catering specifically Zoom Info that is catering specifically for the need of agents. Look at look at for the need of agents. Look at look at for the need of agents. Look at look at the wording, right? They talk about GTM the wording, right? They talk about GTM the wording, right? They talk about GTM work, right? That knowledge work, that work, right? That knowledge work, that work, right? That knowledge work, that research that you do when you need to research that you do when you need to research that you do when you need to prospect, when you need to do prospect, when you need to do prospect, when you need to do headhunting, when whatever it is you headhunting, when whatever it is you headhunting, when whatever it is you need to do that involves people mostly need to do that involves people mostly need to do that involves people mostly straight from cloud code, straight from straight from cloud code, straight from straight from cloud code, straight from codeex or any other agent, they codeex or any other agent, they codeex or any other agent, they understand the gap, right? So yeah, it's understand the gap, right? So yeah, it's understand the gap, right? So yeah, it's funny to think of Cass as an evolution funny to think of Cass as an evolution funny to think of Cass as an evolution of DAS and it is in a way, but it's of DAS and it is in a way, but it's of DAS and it is in a way, but it's catering for a very specific need. As catering for a very specific need. As catering for a very specific need. As much as the AI search engines are much as the AI search engines are much as the AI search engines are different than Google, when agents need different than Google, when agents need different than Google, when agents need them, it's different than people.

  8. When we let this one sink that at the When we let this one sink that at the very least we have two different types very least we have two different types very least we have two different types of paths to complete knowledge work as of paths to complete knowledge work as of paths to complete knowledge work as an agent, we can start thinking about an agent, we can start thinking about an agent, we can start thinking about this in terms of web context this in terms of web context this in terms of web context engineering. We can start thinking about engineering. We can start thinking about engineering. We can start thinking about this in terms of how do I optimize for this in terms of how do I optimize for this in terms of how do I optimize for the specific task or more importantly the specific task or more importantly the specific task or more importantly when things come as it a as it is how do when things come as it a as it is how do when things come as it a as it is how do I optimize this for breeds of task how I optimize this for breeds of task how I optimize this for breeds of task how do I I do this for various parts of the do I I do this for various parts of the do I I do this for various parts of the organization that I'm building for if organization that I'm building for if organization that I'm building for if I'm an AI engineer I need to serve I'm an AI engineer I need to serve I'm an AI engineer I need to serve different teams they may have different different teams they may have different different teams they may have different needs it's very tempting to throw AI needs it's very tempting to throw AI needs it's very tempting to throw AI search at all of them but maybe that's search at all of them but maybe that's search at all of them but maybe that's not optimal maybe I need a combination not optimal maybe I need a combination not optimal maybe I need a combination of both maybe I can start seeing all of both maybe I can start seeing all of both maybe I can start seeing all sorts of cost efficiencies emerge emerge sorts of cost efficiencies emerge emerge sorts of cost efficiencies emerge emerge from that. So for the second half of from that. So for the second half of from that. So for the second half of this presentation, we actually went this presentation, we actually went this presentation, we actually went ahead and created a test. This is not a ahead and created a test. This is not a ahead and created a test. This is not a benchmark. You won't see any any benchmark. You won't see any any benchmark. You won't see any any something concrete that I can say with something concrete that I can say with something concrete that I can say with great confidence other than the actual great confidence other than the actual great confidence other than the actual research that we did because we wanted research that we did because we wanted research that we did because we wanted to start unraveling the different to start unraveling the different to start unraveling the different considerations and how do these two considerations and how do these two considerations and how do these two stack up against each other. So we stack up against each other. So we stack up against each other. So we designed a test. We went for something designed a test. We went for something designed a test. We went for something basic. We said okay let's take a company basic. We said okay let's take a company basic. We said okay let's take a company an entity and try and enrich it across an entity and try and enrich it across an entity and try and enrich it across 25 different fields. Some of them are 25 different fields. Some of them are 25 different fields. Some of them are very easy. you know the company domain, very easy. you know the company domain, very easy. you know the company domain, the name uh the headquarters, but some the name uh the headquarters, but some the name uh the headquarters, but some are more challenging, right? Things are more challenging, right? Things are more challenging, right? Things about hiring and people and something.

  9. about hiring and people and something. about hiring and people and something. And we build a simple agent, a loop in a And we build a simple agent, a loop in a And we build a simple agent, a loop in a loop that knows that uses loop that knows that uses loop that knows that uses Opus 4.8 as the harness and it starts to Opus 4.8 as the harness and it starts to Opus 4.8 as the harness and it starts to go over field by field, go out, search go over field by field, go out, search go over field by field, go out, search for it or retrieve it from the cast, do for it or retrieve it from the cast, do for it or retrieve it from the cast, do it again and again and again until it it again and again and again until it it again and again and again until it completes and brings back set some completes and brings back set some completes and brings back set some guardrails, you know, like budget and guardrails, you know, like budget and guardrails, you know, like budget and stuff just to keep it fair. and I want stuff just to keep it fair. and I want stuff just to keep it fair. and I want to share with you the results. So the to share with you the results. So the to share with you the results. So the first thing that we would care about first thing that we would care about first thing that we would care about right being knowledge work would be uh right being knowledge work would be uh right being knowledge work would be uh sorry we ran it 100 times on all of the sorry we ran it 100 times on all of the sorry we ran it 100 times on all of the sponsors of today's event. sponsors of today's event. sponsors of today's event. So the first thing that we saw in terms So the first thing that we saw in terms So the first thing that we saw in terms of coverage is that there's pretty good of coverage is that there's pretty good of coverage is that there's pretty good convergence. They all did fairly well, convergence. They all did fairly well, convergence. They all did fairly well, right? I'll get to the two at the bottom right? I'll get to the two at the bottom right? I'll get to the two at the bottom in a second. So search were consistent in a second. So search were consistent in a second. So search were consistent performance. One of the major cast performance. One of the major cast performance. One of the major cast providers were also very well. The third providers were also very well. The third providers were also very well. The third one by the way you can see unlocker and one by the way you can see unlocker and one by the way you can see unlocker and SER. SER is good old data uh good old SER. SER is good old data uh good old SER. SER is good old data uh good old Google. We basically did the same thing Google. We basically did the same thing Google. We basically did the same thing just with Google and it performed pretty just with Google and it performed pretty just with Google and it performed pretty well in extracting that information. well in extracting that information. well in extracting that information. Native is cloud's own uh search and you Native is cloud's own uh search and you Native is cloud's own uh search and you see that they converge really well. I see that they converge really well. I see that they converge really well. I was originally surprised about the two was originally surprised about the two was originally surprised about the two cast solutions at the bottom. It was cast solutions at the bottom. It was cast solutions at the bottom. It was counterintuitive. I expected cast to counterintuitive. I expected cast to counterintuitive. I expected cast to dominate this thing because that's you dominate this thing because that's you dominate this thing because that's you know you had one job right to map out know you had one job right to map out know you had one job right to map out all these companies. But but after all these companies. But but after all these companies. But but after diving into into it a bit more you you diving into into it a bit more you you diving into into it a bit more you you understand that well they are limited in understand that well they are limited in understand that well they are limited in the sense that they know what they have the sense that they know what they have the sense that they know what they have about an entity. If I ask it a question about an entity. If I ask it a question about an entity. If I ask it a question that is beyond that, they will never that is beyond that, they will never that is beyond that, they will never have that data, right? Unlike a searcher

  10. have that data, right? Unlike a searcher have that data, right? Unlike a searcher can go out and continue searching and can go out and continue searching and can go out and continue searching and exploring it. If they didn't collect exploring it. If they didn't collect exploring it. If they didn't collect data about the recent job hiring, it data about the recent job hiring, it data about the recent job hiring, it will never be there, right? So, it makes will never be there, right? So, it makes will never be there, right? So, it makes sense that they are a bit behind, but sense that they are a bit behind, but sense that they are a bit behind, but I'm sure at the same time that they have I'm sure at the same time that they have I'm sure at the same time that they have a lot of other advantages that we simply a lot of other advantages that we simply a lot of other advantages that we simply didn't ask for, a lot of other fields didn't ask for, a lot of other fields didn't ask for, a lot of other fields that they didn't have that aren't that they didn't have that aren't that they didn't have that aren't represented. So again it it creates some represented. So again it it creates some represented. So again it it creates some complexities when what how do we measure complexities when what how do we measure complexities when what how do we measure coverage when it relates to the specific coverage when it relates to the specific coverage when it relates to the specific job that we need to do rather than um in job that we need to do rather than um in job that we need to do rather than um in general when we the second thing we general when we the second thing we general when we the second thing we looked at was cost of course here it's looked at was cost of course here it's looked at was cost of course here it's we started seeing it spread out a bit so we started seeing it spread out a bit so we started seeing it spread out a bit so you can see that massive bulk in the you can see that massive bulk in the you can see that massive bulk in the center most of the search and the and center most of the search and the and center most of the search and the and the cast and even using uh Google right the cast and even using uh Google right the cast and even using uh Google right h converge to pretty much the same cost h converge to pretty much the same cost h converge to pretty much the same cost only different right the cast was just only different right the cast was just only different right the cast was just about the service itself what you pay about the service itself what you pay about the service itself what you pay the vendor, right? All of the other the vendor, right? All of the other the vendor, right? All of the other search solutions, you also needed a lot search solutions, you also needed a lot search solutions, you also needed a lot of token burn it burn to actually of token burn it burn to actually of token burn it burn to actually structure that data so you can actually structure that data so you can actually structure that data so you can actually act on it and and and use it as act on it and and and use it as act on it and and and use it as something retrievable, right? So, it's something retrievable, right? So, it's something retrievable, right? So, it's the same output native obscenely the same output native obscenely the same output native obscenely expensive. And the the cast on the expensive. And the the cast on the expensive. And the the cast on the right, I'm sure you're all familiar with right, I'm sure you're all familiar with right, I'm sure you're all familiar with the by far the most expensive in the the by far the most expensive in the the by far the most expensive in the industry. I will not name and shame industry. I will not name and shame industry. I will not name and shame them.

  11. them. them. Interesting. You see that small cast Interesting. You see that small cast Interesting. You see that small cast there at the the left that cast number there at the the left that cast number there at the the left that cast number two that were very cheap. They're also two that were very cheap. They're also two that were very cheap. They're also the ones that are here at the bottom, the ones that are here at the bottom, the ones that are here at the bottom, which is funny because what I believe is which is funny because what I believe is which is funny because what I believe is happening there is that we're seeing happening there is that we're seeing happening there is that we're seeing even within this industry niche players even within this industry niche players even within this industry niche players that have, you know, lower quality data that have, you know, lower quality data that have, you know, lower quality data but much cheaper. They're already but much cheaper. They're already but much cheaper. They're already carving that niche of the longtail, carving that niche of the longtail, carving that niche of the longtail, right, of small shops or small usage right, of small shops or small usage right, of small shops or small usage that don't want to pay as much and don't that don't want to pay as much and don't that don't want to pay as much and don't need as much data. And we're all seeing need as much data. And we're all seeing need as much data. And we're all seeing it them branch out there. it them branch out there. it them branch out there. Most of you here, I presume, are Most of you here, I presume, are Most of you here, I presume, are engineers. So there's a very evident engineers. So there's a very evident engineers. So there's a very evident question that we did not ask here which question that we did not ask here which question that we did not ask here which is what is the one thing that an is what is the one thing that an is what is the one thing that an engineer would care about? Thank you. Let's talk about scale. Thank you. Let's talk about scale. This is the cost not for the whole 100. This is the cost not for the whole 100. This is the cost not for the whole 100. This is the cost per one for one record. What happens if we need a million? Now What happens if we need a million? Now yes a million records will not fit in yes a million records will not fit in yes a million records will not fit in the context with obviously we're not the context with obviously we're not the context with obviously we're not talking about a single run that needs a talking about a single run that needs a talking about a single run that needs a million you can think about million in million you can think about million in million you can think about million in terms of the frequency right if I am a terms of the frequency right if I am a terms of the frequency right if I am a market research I do due diligence for market research I do due diligence for market research I do due diligence for private equity I revisit these companies private equity I revisit these companies private equity I revisit these companies all the time I ask more questions about all the time I ask more questions about all the time I ask more questions about them as the time goes by was there any them as the time goes by was there any them as the time goes by was there any new news about them was there anything new news about them was there anything new news about them was there anything that changed somebody joined somebody that changed somebody joined somebody that changed somebody joined somebody leave do they have new hires I keep on leave do they have new hires I keep on leave do they have new hires I keep on asking the same thing so when I'm asking the same thing so when I'm asking the same thing so when I'm talking about this the multiply by a talking about this the multiply by a talking about this the multiply by a million it's not just about the number million it's not just about the number million it's not just about the number of companies it's the frequency in which of companies it's the frequency in which of companies it's the frequency in which I'm asking it frequency is the cost

  12. I'm asking it frequency is the cost I'm asking it frequency is the cost killer when we talk about these and we killer when we talk about these and we killer when we talk about these and we need to acknowledge that right we're need to acknowledge that right we're need to acknowledge that right we're we're thinking about this in terms of we're thinking about this in terms of we're thinking about this in terms of web context engineering we're starting web context engineering we're starting web context engineering we're starting to look at it differently to look at it differently to look at it differently every repeated query costs the same as every repeated query costs the same as every repeated query costs the same as the first even if it brought back the the first even if it brought back the the first even if it brought back the exact same answers nothing changed pay exact same answers nothing changed pay exact same answers nothing changed pay up right no is false positives for sure up right no is false positives for sure up right no is false positives for sure go in token cost right we saw them all go in token cost right we saw them all go in token cost right we saw them all that there's very high token we know that there's very high token we know that there's very high token we know that doesn't that doesn't shrink think that doesn't that doesn't shrink think that doesn't that doesn't shrink think well over time there's always some well over time there's always some well over time there's always some volume element in terms of the cost but volume element in terms of the cost but volume element in terms of the cost but it's not the same as you know flatlining it's not the same as you know flatlining it's not the same as you know flatlining right and if we bring this back to right and if we bring this back to right and if we bring this back to knowledge work this is where we see knowledge work this is where we see knowledge work this is where we see teams that are starting to teams that are starting to teams that are starting to cut corners so I won't research this cut corners so I won't research this cut corners so I won't research this company every day I'll look at it once a company every day I'll look at it once a company every day I'll look at it once a week or once a month I won't ask that week or once a month I won't ask that week or once a month I won't ask that question now I will I don't want all the question now I will I don't want all the question now I will I don't want all the results I'll only take 10 results 20 results I'll only take 10 results 20 results I'll only take 10 results 20 results something we start so we already results something we start so we already results something we start so we already have the setup we have what we need to have the setup we have what we need to have the setup we have what we need to do the knowledge work but at the same do the knowledge work but at the same do the knowledge work but at the same time. We're not extracting all of the time. We're not extracting all of the time. We're not extracting all of the value because we're starting to be value because we're starting to be value because we're starting to be conscious about cost, right? Basically, conscious about cost, right? Basically, conscious about cost, right? Basically, we're renting context. We're not owning we're renting context. We're not owning we're renting context. We're not owning the context that we use. That is a very the context that we use. That is a very the context that we use. That is a very important distinction.

  13. important distinction. important distinction. Again, if we're good engineers and we Again, if we're good engineers and we Again, if we're good engineers and we ask ourselves, what about scale? The ask ourselves, what about scale? The ask ourselves, what about scale? The second and most obvious thing that will second and most obvious thing that will second and most obvious thing that will come to mind now, so how about we build come to mind now, so how about we build come to mind now, so how about we build it? What if we take all of that web data it? What if we take all of that web data it? What if we take all of that web data ourselves and stick it in some vector ourselves and stick it in some vector ourselves and stick it in some vector database and try and see what comes out database and try and see what comes out database and try and see what comes out of it? of it? of it? So I asked my AI engineer to do exactly So I asked my AI engineer to do exactly So I asked my AI engineer to do exactly that. that. that. Again, this is a test. This is not a Again, this is a test. This is not a Again, this is a test. This is not a benchmark or a full-blown operation. benchmark or a full-blown operation. benchmark or a full-blown operation. This is a day's work at best just to This is a day's work at best just to This is a day's work at best just to illustrate the concept and to show illustrate the concept and to show illustrate the concept and to show something about the cost efficiencies something about the cost efficiencies something about the cost efficiencies that you can generate potentially by that you can generate potentially by that you can generate potentially by doing it yourself potentially in doing it yourself potentially in doing it yourself potentially in specific scenarios. specific scenarios. specific scenarios. The test simple take the company name The test simple take the company name The test simple take the company name nothing but run it through Google find nothing but run it through Google find nothing but run it through Google find the relevant entries the relevant URLs the relevant entries the relevant URLs the relevant entries the relevant URLs of that company in various websites that of that company in various websites that of that company in various websites that have all of that information right you have all of that information right you have all of that information right you use search when you don't know the use search when you don't know the use search when you don't know the source but when we're talking about source but when we're talking about source but when we're talking about company enrichment we all know these company enrichment we all know these company enrichment we all know these sources we all know where that data sources we all know where that data sources we all know where that data comes from the cast also bring it from comes from the cast also bring it from comes from the cast also bring it from them zoom info bring it from them it's them zoom info bring it from them it's them zoom info bring it from them it's the same thing over and over again why the same thing over and over again why the same thing over and over again why not just go straight to the source why not just go straight to the source why not just go straight to the source why are we doing that middleman thing are we doing that middleman thing are we doing that middleman thing LinkedIn companies LinkedIn jobs LinkedIn companies LinkedIn jobs LinkedIn companies LinkedIn jobs crunchbas Right there we have scrapers crunchbas Right there we have scrapers crunchbas Right there we have scrapers for those. You just tap in and you start for those. You just tap in and you start for those. You just tap in and you start paying as a pay as you go. We build two paying as a pay as you go. We build two paying as a pay as you go. We build two dedicated scrapers. We have a new AI dedicated scrapers. We have a new AI dedicated scrapers. We have a new AI tool called scraper studio. It's tool called scraper studio. It's tool called scraper studio. It's basically lets you build a scraper for basically lets you build a scraper for basically lets you build a scraper for any website in less than 5 minutes. All any website in less than 5 minutes. All any website in less than 5 minutes. All powered by AI. And then it also have a powered by AI. And then it also have a powered by AI. And then it also have a self-healing function, right? So if the self-healing function, right? So if the self-healing function, right? So if the website changes, it fixes itself and website changes, it fixes itself and website changes, it fixes itself and keeps on going. Merge it all into one

  14. keeps on going. Merge it all into one keeps on going. Merge it all into one entity. Basic heristics. If there's entity. Basic heristics. If there's entity. Basic heristics. If there's conflict, choose that over that. And conflict, choose that over that. And conflict, choose that over that. And eventually we have a data set of these eventually we have a data set of these eventually we have a data set of these uh 100 companies. Zero AI cost involved. uh 100 companies. Zero AI cost involved. uh 100 companies. Zero AI cost involved. There's no tokens. There's no tokens. There's no tokens. Coverage fairly well. Not amazing. Not Coverage fairly well. Not amazing. Not Coverage fairly well. Not amazing. Not the best that we saw here, but stacking the best that we saw here, but stacking the best that we saw here, but stacking up pretty well. And again, this is just up pretty well. And again, this is just up pretty well. And again, this is just a day experiment. Probably not even as a day experiment. Probably not even as a day experiment. Probably not even as much. Okay, so again and again, very much. Okay, so again and again, very much. Okay, so again and again, very specific task, very limited context, specific task, very limited context, specific task, very limited context, very limited situation. very limited situation. very limited situation. you know tread uh tread lightly and then you know tread uh tread lightly and then you know tread uh tread lightly and then proceed with caution when it comes to proceed with caution when it comes to proceed with caution when it comes to conclusions. conclusions. conclusions. The real story is not this. The real The real story is not this. The real The real story is not this. The real story is this. That's what it cost to just go and fetch That's what it cost to just go and fetch that data that is out there. that data that is out there. that data that is out there. We think about knowledge graphs, we We think about knowledge graphs, we We think about knowledge graphs, we think about entities, but if you think think about entities, but if you think think about entities, but if you think about for example LinkedIn, the data is about for example LinkedIn, the data is about for example LinkedIn, the data is already structured in form of entities. already structured in form of entities. already structured in form of entities. There's an entity for a company, there's There's an entity for a company, there's There's an entity for a company, there's entity for a person, there's entity for entity for a person, there's entity for entity for a person, there's entity for a job and they're connected between a job and they're connected between a job and they're connected between them.

  15. them. them. Sometimes the ontology is already there. Sometimes the ontology is already there. Sometimes the ontology is already there. Again, this is not the most complicated Again, this is not the most complicated Again, this is not the most complicated of scenarios, but this is pretty damn of scenarios, but this is pretty damn of scenarios, but this is pretty damn good. Now, yes, it took time to set up. So, it's yes, it took time to set up. So, it's not really fair to compare apples to not really fair to compare apples to not really fair to compare apples to apples when it comes to the cost because apples when it comes to the cost because apples when it comes to the cost because these are out of the box. You can just these are out of the box. You can just these are out of the box. You can just tap into the API. That one that I just tap into the API. That one that I just tap into the API. That one that I just showed you required some setup. Let's showed you required some setup. Let's showed you required some setup. Let's say it's a week. Let's price it at say it's a week. Let's price it at say it's a week. Let's price it at $5,000 just to give us some perspective. $5,000 just to give us some perspective. $5,000 just to give us some perspective. We can actually start thinking about We can actually start thinking about We can actually start thinking about this in terms of a tipping point. We can this in terms of a tipping point. We can this in terms of a tipping point. We can actually start thinking about what is actually start thinking about what is actually start thinking about what is that tipping point in which it makes that tipping point in which it makes that tipping point in which it makes more sense for me to build it myself, more sense for me to build it myself, more sense for me to build it myself, right? Then keep on renting it. Now right? Then keep on renting it. Now right? Then keep on renting it. Now again, everything to the left of that again, everything to the left of that again, everything to the left of that dot in this case it was just over 15,000 dot in this case it was just over 15,000 dot in this case it was just over 15,000 entities or queries, right? When we entities or queries, right? When we entities or queries, right? When we think about it, so it made sense to do think about it, so it made sense to do think about it, so it made sense to do it at this point. Maybe it's not 15, it at this point. Maybe it's not 15, it at this point. Maybe it's not 15, maybe it's 30, maybe it's 100,000, maybe maybe it's 30, maybe it's 100,000, maybe maybe it's 30, maybe it's 100,000, maybe it's 10,000. Really depends on the use it's 10,000. Really depends on the use it's 10,000. Really depends on the use case. But there is a tipping point in case. But there is a tipping point in case. But there is a tipping point in which it actually makes sense to do it which it actually makes sense to do it which it actually makes sense to do it yourself. H which leads us to the fact yourself. H which leads us to the fact yourself. H which leads us to the fact that both AI search and casts and all that both AI search and casts and all that both AI search and casts and all these solutions they're very get good in these solutions they're very get good in these solutions they're very get good in the sense that you can just plug and the sense that you can just plug and the sense that you can just plug and play. But if your knowledge work needs play. But if your knowledge work needs play. But if your knowledge work needs right are persistent and consistent and right are persistent and consistent and right are persistent and consistent and to a certain degree may even continue to a certain degree may even continue to a certain degree may even continue escalating and growing then this is escalating and growing then this is escalating and growing then this is perhaps a direction to start perhaps a direction to start perhaps a direction to start considering. Maybe I can just go ahead considering. Maybe I can just go ahead considering. Maybe I can just go ahead and build my own. Cuz the nice thing and build my own. Cuz the nice thing and build my own. Cuz the nice thing about it is that all of the things that about it is that all of the things that about it is that all of the things that we see on the left up until the third we see on the left up until the third we see on the left up until the third part is

  16. part is part is upfront investment. And the most upfront investment. And the most upfront investment. And the most important thing that whatever retrieval important thing that whatever retrieval important thing that whatever retrieval happens later on from the from the happens later on from the from the happens later on from the from the agents, right, is free. Not really free, agents, right, is free. Not really free, agents, right, is free. Not really free, but you get what I mean, right? There's but you get what I mean, right? There's but you get what I mean, right? There's no added cost. I can just ask that no added cost. I can just ask that no added cost. I can just ask that question over and over again. I did not question over and over again. I did not question over and over again. I did not like the first answer. I'll ask it like the first answer. I'll ask it like the first answer. I'll ask it again. I'll ask it a hundred times until again. I'll ask it a hundred times until again. I'll ask it a hundred times until I get what I need. I I have no more I get what I need. I I have no more I get what I need. I I have no more fear, no more cutting corners, which is fear, no more cutting corners, which is fear, no more cutting corners, which is maybe the most important thing. And I'm maybe the most important thing. And I'm maybe the most important thing. And I'm leaving aside the fact that this is also leaving aside the fact that this is also leaving aside the fact that this is also custom business logic. I can connect it custom business logic. I can connect it custom business logic. I can connect it with my own data. There's all sorts of with my own data. There's all sorts of with my own data. There's all sorts of other advantages of owning it. You know, other advantages of owning it. You know, other advantages of owning it. You know, we'll keep it to the imagination. Remember, we asked about a million, not Remember, we asked about a million, not about 15,000. This compounds, this about 15,000. This compounds, this about 15,000. This compounds, this compounds greatly, compounds greatly, compounds greatly, right? We need that horizon. Remember, right? We need that horizon. Remember, right? We need that horizon. Remember, the web keeps changing. We saw the the web keeps changing. We saw the the web keeps changing. We saw the stainless data and how the data decays. stainless data and how the data decays. stainless data and how the data decays. So we need to be thinking about this in So we need to be thinking about this in So we need to be thinking about this in the long run and how this will evolve the long run and how this will evolve the long run and how this will evolve when we keep on asking the questions when we keep on asking the questions when we keep on asking the questions about the entities that we care about about the entities that we care about about the entities that we care about just to wrap it up. So AI search CAS just to wrap it up. So AI search CAS just to wrap it up. So AI search CAS they can get you very far when what you they can get you very far when what you they can get you very far when what you need is ad hoc when what you need is need is ad hoc when what you need is need is ad hoc when what you need is always changing when sometimes you look always changing when sometimes you look always changing when sometimes you look at different things even the mix and at different things even the mix and at different things even the mix and match of them for certain task use this match of them for certain task use this match of them for certain task use this for certain task use that you can I'm for certain task use that you can I'm for certain task use that you can I'm sure again I just showed a test there's sure again I just showed a test there's sure again I just showed a test there's a lot of ways to optimize it just like a lot of ways to optimize it just like a lot of ways to optimize it just like any other context engineering and make any other context engineering and make any other context engineering and make and use uh lighter models and use other and use uh lighter models and use other and use uh lighter models and use other stuff there's a lot of great stuff to be stuff there's a lot of great stuff to be stuff there's a lot of great stuff to be done but eventually the frequency will done but eventually the frequency will done but eventually the frequency will come and bite you in the ass when it come and bite you in the ass when it come and bite you in the ass when it comes to cost and that's something to be comes to cost and that's something to be comes to cost and that's something to be mindful of and there's a fair chance mindful of and there's a fair chance mindful of and there's a fair chance that that tipping point is much lower

  17. that that tipping point is much lower that that tipping point is much lower than you think. And that's something than you think. And that's something than you think. And that's something that when as we design these systems, that when as we design these systems, that when as we design these systems, when we think about web context when we think about web context when we think about web context engineering, we need to be mindful of engineering, we need to be mindful of engineering, we need to be mindful of that. And last but not least, the last that. And last but not least, the last that. And last but not least, the last slide we showed owned context compounds slide we showed owned context compounds slide we showed owned context compounds while rented decays, right? It's not a while rented decays, right? It's not a while rented decays, right? It's not a one-time task. Again, if it's a one-time one-time task. Again, if it's a one-time one-time task. Again, if it's a one-time question, use AI search. It will be question, use AI search. It will be question, use AI search. It will be amazing. when you need to do it over and amazing. when you need to do it over and amazing. when you need to do it over and over again, there's a fair chance that over again, there's a fair chance that over again, there's a fair chance that it will not it will not it will not that you're missing out on potential that you're missing out on potential that you're missing out on potential compounding effect and you are losing compounding effect and you are losing compounding effect and you are losing out. out. out. Thank you very much.

Summary

This session discusses the evolving connection between AI and the web, highlighting Bright Data's role in web data extraction for over 20,000 teams, including major AI labs. The key takeaway is that the web is transforming from a mere data source to a crucial context provider for AI agents performing knowledge work, enabling them to gather information and draw conclusions.

View original episode ↗