Postgres Replication at speed with PeerDB's Sai Srirampur

do you have a WPF application and want do you have a WPF application and want to take it to Mac OS or Linux avalonia to take it to Mac OS or Linux avalonia to take it to Mac OS or Linux avalonia xpf is a binary compatible crossplatform xpf is a binary compatible crossplatform xpf is a binary compatible crossplatform Fork of WPF it enables WPF apps to run Fork of WPF it enables WPF apps to run Fork of WPF it enables WPF apps to run on New platforms with minimal effort and on New platforms with minimal effort and on New platforms with minimal effort and maximum compatibility with a few tweaks maximum compatibility with a few tweaks maximum compatibility with a few tweaks to your project filed your WPF app and to your project filed your WPF app and to your project filed your WPF app and all its dependencies are ready for all its dependencies are ready for all its dependencies are ready for testing on new platforms start your app testing on new platforms start your app testing on new platforms start your app transformation Journey with a 30-day transformation Journey with a 30-day transformation Journey with a 30-day free trial head over to avalonia free trial head over to avalonia free trial head over to avalonia ui.net / Hansel minutes to get started ui.net / Hansel minutes to get started ui.net / Hansel minutes to get started today that's avalonia today that's avalonia today that's avalonia ui.net Hansel ui.net Hansel ui.net Hansel [Music] [Music] [Music] minutes hi I'm Scott Hansel and this is minutes hi I'm Scott Hansel and this is minutes hi I'm Scott Hansel and this is another episode of Hansel minutes today another episode of Hansel minutes today another episode of Hansel minutes today I'm chatting with s Krishna shumer he is I'm chatting with s Krishna shumer he is I'm chatting with s Krishna shumer he is a CEO and co-founder of Pier DB how are a CEO and co-founder of Pier DB how are a CEO and co-founder of Pier DB how are you sir good Scott uh I'm very glad to you sir good Scott uh I'm very glad to you sir good Scott uh I'm very glad to be here and excited about this podcast be here and excited about this podcast be here and excited about this podcast so yeah thanks for hanging out so uh you so yeah thanks for hanging out so uh you so yeah thanks for hanging out so uh you know you actually used to work at know you actually used to work at know you actually used to work at Microsoft uh but you've gone and started Microsoft uh but you've gone and started Microsoft uh but you've gone and started a company and you're in the bay there a company and you're in the bay there a company and you're in the bay there starting a new company Focus right now starting a new company Focus right now starting a new company Focus right now on postgress and it seems like postgress on postgress and it seems like postgress on postgress and it seems like postgress is having a moment the last five years is having a moment the last five years is having a moment the last five years has just been a huge growth time for has just been a huge growth time for has just been a huge growth time for postgress why do you think that is postgress why do you think that is postgress why do you think that is absolutely I think a couple of reasons absolutely I think a couple of reasons absolutely I think a couple of reasons right like I mean number one I think right like I mean number one I think right like I mean number one I think postgis has evolved in supporting like a postgis has evolved in supporting like a postgis has evolved in supporting like a plethora of use cases right like maybe 8

plethora of use cases right like maybe 8 plethora of use cases right like maybe 8 years ago postgis was used more like as years ago postgis was used more like as years ago postgis was used more like as a metadata store but now you know it has a metadata store but now you know it has a metadata store but now you know it has evolved to support like a bunch of use evolved to support like a bunch of use evolved to support like a bunch of use cases including like transactional cases including like transactional cases including like transactional analytical search that is number one analytical search that is number one analytical search that is number one number two it is open- Source companies number two it is open- Source companies number two it is open- Source companies these days like are looking forward for these days like are looking forward for these days like are looking forward for like software where there is no lock in like software where there is no lock in like software where there is no lock in right like this is with both like right like this is with both like right like this is with both like Enterprises as well as like smbs right Enterprises as well as like smbs right Enterprises as well as like smbs right like and that's the reason there is like like and that's the reason there is like like and that's the reason there is like a big shift of migrations from Oracle a big shift of migrations from Oracle a big shift of migrations from Oracle and SQL Server to postest because of and SQL Server to postest because of and SQL Server to postest because of Open Source and the trust that is number Open Source and the trust that is number Open Source and the trust that is number two number three is more like the two number three is more like the two number three is more like the community around it as it's open source community around it as it's open source community around it as it's open source like there's a lot of like you know like there's a lot of like you know like there's a lot of like you know support communities and like you know support communities and like you know support communities and like you know that is number three which is like the that is number three which is like the that is number three which is like the community and last but not the least is community and last but not the least is community and last but not the least is I mean it just works right like it is it I mean it just works right like it is it I mean it just works right like it is it is super usable basically right like and is super usable basically right like and is super usable basically right like and developers love something that is usable developers love something that is usable developers love something that is usable and that just works Scott right like and that just works Scott right like and that just works Scott right like that's the reason you're seeing this that's the reason you're seeing this that's the reason you're seeing this like you know shift and one interesting like you know shift and one interesting like you know shift and one interesting Insight I want to share is like even Insight I want to share is like even Insight I want to share is like even from just like from a finance standpoint from just like from a finance standpoint from just like from a finance standpoint right like postgress Revenue has grown right like postgress Revenue has grown right like postgress Revenue has grown from 1% of Oracle Revenue 5 years ago to from 1% of Oracle Revenue 5 years ago to from 1% of Oracle Revenue 5 years ago to now 10% of oracles Revenue so from a now 10% of oracles Revenue so from a now 10% of oracles Revenue so from a popularity standpoint it is the only popularity standpoint it is the only popularity standpoint it is the only database who's growing in like database who's growing in like database who's growing in like popularity this is from DB engines yeah popularity this is from DB engines yeah popularity this is from DB engines yeah these are a couple of reasons why you these are a couple of reasons why you these are a couple of reasons why you know postgress is taking off and I've know postgress is taking off and I've know postgress is taking off and I've been in this area for like eight years been in this area for like eight years been in this area for like eight years now and I absolutely love postgis yeah I now and I absolutely love postgis yeah I now and I absolutely love postgis yeah I think that it just works like that kind think that it just works like that kind think that it just works like that kind of like advertising really can't be of like advertising really can't be of like advertising really can't be overstated like when I came up in the overstated like when I came up in the overstated like when I came up in the 90s there was lots of database 90s there was lots of database 90s there was lots of database advertisements and in the magazines and advertisements and in the magazines and advertisements and in the magazines and you'd hear about this database or that

you'd hear about this database or that you'd hear about this database or that database but right now everyone's moving database but right now everyone's moving database but right now everyone's moving to postris because it simply works it to postris because it simply works it to postris because it simply works it really excels to like uh SQL standards really excels to like uh SQL standards really excels to like uh SQL standards SQL standards because the ANC groups the SQL standards because the ANC groups the SQL standards because the ANC groups the iso groups basically have these core SQL iso groups basically have these core SQL iso groups basically have these core SQL specifications and postrest basically specifications and postrest basically specifications and postrest basically conforms to most of them and I think conforms to most of them and I think conforms to most of them and I think that is part of it it just works that is part of it it just works that is part of it it just works Mystique absolutely yep just out of Mystique absolutely yep just out of Mystique absolutely yep just out of curiosity do you go back and forth on curiosity do you go back and forth on curiosity do you go back and forth on saying SQL or SQL because it seems like saying SQL or SQL because it seems like saying SQL or SQL because it seems like Microsoft people say SQL and Microsoft people say SQL and Microsoft people say SQL and non-microsoft people say SQL it's non-microsoft people say SQL it's non-microsoft people say SQL it's interesting that like I do both of them interesting that like I do both of them interesting that like I do both of them right like because I also come from like right like because I also come from like right like because I also come from like Microsoft right like I work for five Microsoft right like I work for five Microsoft right like I work for five years there so maybe when I'm speaking years there so maybe when I'm speaking years there so maybe when I'm speaking to developers I do sequel but when I'm to developers I do sequel but when I'm to developers I do sequel but when I'm in a podcast I say SQL right like so it in a podcast I say SQL right like so it in a podcast I say SQL right like so it it really changes yeah I was wondering it really changes yeah I was wondering it really changes yeah I was wondering if we were going to be able to solve if we were going to be able to solve if we were going to be able to solve that problem today for the people but it that problem today for the people but it that problem today for the people but it sounds like we will not be able to do sounds like we will not be able to do sounds like we will not be able to do that so for a lot of people postgress is that so for a lot of people postgress is that so for a lot of people postgress is like at the core of their business it's like at the core of their business it's like at the core of their business it's the number one place uh it's either the the number one place uh it's either the the number one place uh it's either the authoritative source for their data or authoritative source for their data or authoritative source for their data or it's going to become the authoritative it's going to become the authoritative it's going to become the authoritative source and I'm curious how people are source and I'm curious how people are source and I'm curious how people are moving their data into postris because moving their data into postris because moving their data into postris because for small databases I use like little for small databases I use like little for small databases I use like little client applications and I do what's client applications and I do what's client applications and I do what's called left-hand rightand work where I called left-hand rightand work where I called left-hand rightand work where I say input and output and I run it you say input and output and I run it you say input and output and I run it you know it takes 10 or 15 minutes this is know it takes 10 or 15 minutes this is know it takes 10 or 15 minutes this is not big data movement but for a not big data movement but for a not big data movement but for a warehouse or something like that you're warehouse or something like that you're warehouse or something like that you're going to need to move data in a assume a going to need to move data in a assume a going to need to move data in a assume a more sophisticated manner absolutely Ely more sophisticated manner absolutely Ely more sophisticated manner absolutely Ely so it really depends on like the type of so it really depends on like the type of so it really depends on like the type of the migrations Scot right like I mean the migrations Scot right like I mean

the migrations Scot right like I mean there are offline migrations and there there are offline migrations and there there are offline migrations and there are online migrations right offline are online migrations right offline are online migrations right offline migrations are migrations where the migrations are migrations where the migrations are migrations where the application can effort downtime these application can effort downtime these application can effort downtime these are like say tier three tier four kind are like say tier three tier four kind are like say tier three tier four kind of applications but then you have online of applications but then you have online of applications but then you have online migrations where you want to minimize migrations where you want to minimize migrations where you want to minimize the downtime as much as possible Right the downtime as much as possible Right the downtime as much as possible Right like keep it to like say few minutes or like keep it to like say few minutes or like keep it to like say few minutes or like even seconds and the tool depends like even seconds and the tool depends like even seconds and the tool depends on what is the type of the migration on what is the type of the migration on what is the type of the migration basically if it's offline migration basically if it's offline migration basically if it's offline migration postgress comes with like PG Dum PG postgress comes with like PG Dum PG postgress comes with like PG Dum PG restore which is pretty incredible I restore which is pretty incredible I restore which is pretty incredible I mean it does like migrations at a mean it does like migrations at a mean it does like migrations at a snapshot level it has like a bunch of snapshot level it has like a bunch of snapshot level it has like a bunch of options where you can configure options where you can configure options where you can configure parallelism you can configure like the parallelism you can configure like the parallelism you can configure like the format of the dump which is like you format of the dump which is like you format of the dump which is like you want it compressed or you want it like want it compressed or you want it like want it compressed or you want it like with plain text Etc right so that is one with plain text Etc right so that is one with plain text Etc right so that is one which works pretty well for like you which works pretty well for like you which works pretty well for like you know offline migrations and then for know offline migrations and then for know offline migrations and then for online migrations postgress has this online migrations postgress has this online migrations postgress has this concept of uh logical replication where concept of uh logical replication where concept of uh logical replication where you have the ability to take a snapshot you have the ability to take a snapshot you have the ability to take a snapshot in time and then uh read the changes in in time and then uh read the changes in in time and then uh read the changes in the transactional log periodically and the transactional log periodically and the transactional log periodically and then make sure that the target which is then make sure that the target which is then make sure that the target which is postgress is caught up basically right postgress is caught up basically right postgress is caught up basically right and and here I'm talking about mostly and and here I'm talking about mostly and and here I'm talking about mostly like you know postgress to postgress like you know postgress to postgress like you know postgress to postgress migrations right like you're running migrations right like you're running migrations right like you're running postgress on premise and you want to postgress on premise and you want to postgress on premise and you want to move to the cloud so PG Dum PG restore move to the cloud so PG Dum PG restore move to the cloud so PG Dum PG restore is for offline migrations uh logical is for offline migrations uh logical is for offline migrations uh logical replication is for like online replication is for like online replication is for like online migrations now coming to you know if the migrations now coming to you know if the migrations now coming to you know if the targets are like heterog genius right targets are like heterog genius right targets are like heterog genius right like say I have like oracles right I like say I have like oracles right I like say I have like oracles right I want to migrate like Oracle to like want to migrate like Oracle to like want to migrate like Oracle to like posters and it is pretty common these posters and it is pretty common these posters and it is pretty common these days with like Enterprise like companies days with like Enterprise like companies days with like Enterprise like companies who have like a fleet of Oracle who have like a fleet of Oracle who have like a fleet of Oracle databases right these are like hundreds databases right these are like hundreds databases right these are like hundreds and thousands and they want to migrate

and thousands and they want to migrate and thousands and they want to migrate them to like postris right like and I them to like postris right like and I them to like postris right like and I saw this very closely at Microsoft there saw this very closely at Microsoft there saw this very closely at Microsoft there are a bunch of tools right like for are a bunch of tools right like for are a bunch of tools right like for example Azure has this thing called example Azure has this thing called example Azure has this thing called Azure DMS and AWS has AWS DMS there are Azure DMS and AWS has AWS DMS there are Azure DMS and AWS has AWS DMS there are other like tools which help online other like tools which help online other like tools which help online migrations as well as offline migrations migrations as well as offline migrations migrations as well as offline migrations and they also use a similar concept and they also use a similar concept and they also use a similar concept where if it's online they use the where if it's online they use the where if it's online they use the transactional log right like where they transactional log right like where they transactional log right like where they take the initial snapshot and take the initial snapshot and take the initial snapshot and periodically like you know capture periodically like you know capture periodically like you know capture changes which is also called as change changes which is also called as change changes which is also called as change data capture and then put it to postgis data capture and then put it to postgis data capture and then put it to postgis basically right so this is about you basically right so this is about you basically right so this is about you know how customers can migrate to know how customers can migrate to know how customers can migrate to postris Scott if that answers your postris Scott if that answers your postris Scott if that answers your question well let me ask you this maybe question well let me ask you this maybe question well let me ask you this maybe for the for the audience there which for the for the audience there which for the for the audience there which could be a moment where we talk about could be a moment where we talk about could be a moment where we talk about the difference between a migration and a the difference between a migration and a the difference between a migration and a replication because there's like a replication because there's like a replication because there's like a moment in time snapshot but then you moment in time snapshot but then you moment in time snapshot but then you called out logging and change logs which called out logging and change logs which called out logging and change logs which I think is an important differentiation I think is an important differentiation I think is an important differentiation correct so in migration there is like a correct so in migration there is like a correct so in migration there is like a concept of initial snapshot where concept of initial snapshot where concept of initial snapshot where whatever data is currently present in whatever data is currently present in whatever data is currently present in tables you put it to like the target tables you put it to like the target tables you put it to like the target database say you have a terabyte of data database say you have a terabyte of data database say you have a terabyte of data right like already present in your right like already present in your right like already present in your Source like database you take a snapshot Source like database you take a snapshot Source like database you take a snapshot and put that terabyte like into the and put that terabyte like into the and put that terabyte like into the target database then while this is target database then while this is target database then while this is happening you basically capture all the happening you basically capture all the happening you basically capture all the changes in the transactional log changes in the transactional log changes in the transactional log basically and then uh the replication basically and then uh the replication basically and then uh the replication process reads the transaction log and process reads the transaction log and process reads the transaction log and there are different mechanisms like each there are different mechanisms like each there are different mechanisms like each database has its own thing right like database has its own thing right like database has its own thing right like for example postgress it's logical for example postgress it's logical for example postgress it's logical decoding right like Oracle has a decoding right like Oracle has a decoding right like Oracle has a different thing the replication part is different thing the replication part is different thing the replication part is like it reads these transaction logs like it reads these transaction logs like it reads these transaction logs it's a bunch of inserts updates and it's a bunch of inserts updates and it's a bunch of inserts updates and deletes right and then it's it applies

deletes right and then it's it applies deletes right and then it's it applies it to the Target database right so and it to the Target database right so and it to the Target database right so and and this is a part of the migration and this is a part of the migration and this is a part of the migration process now interesting thing you ident process now interesting thing you ident process now interesting thing you ident you you bring up is there are a bunch of you you bring up is there are a bunch of you you bring up is there are a bunch of workloads where it's not like you just workloads where it's not like you just workloads where it's not like you just migrate like once right like you want to migrate like once right like you want to migrate like once right like you want to keep a couple of databases in sync all keep a couple of databases in sync all keep a couple of databases in sync all the time a good example there is say you the time a good example there is say you the time a good example there is say you have like an operational oltp database have like an operational oltp database have like an operational oltp database which is postgress and you have your which is postgress and you have your which is postgress and you have your banking application running on postgress banking application running on postgress banking application running on postgress and you want to offload like a use case and you want to offload like a use case and you want to offload like a use case which is fraud detection which is more which is fraud detection which is more which is fraud detection which is more false under the analytical bucket to false under the analytical bucket to false under the analytical bucket to Warehouse or an analytical store right Warehouse or an analytical store right Warehouse or an analytical store right say snowflake right in that scenario you say snowflake right in that scenario you say snowflake right in that scenario you want to keep the operational store which want to keep the operational store which want to keep the operational store which is postgress in sync with the analytical is postgress in sync with the analytical is postgress in sync with the analytical store as fast as possible basically store as fast as possible basically store as fast as possible basically right like with low latency right and right like with low latency right and right like with low latency right and there you want to keep it in sync all there you want to keep it in sync all there you want to keep it in sync all the time basically right like and the the time basically right like and the the time basically right like and the latency requirements really depend on latency requirements really depend on latency requirements really depend on the use case right like if it's like a the use case right like if it's like a the use case right like if it's like a fraud detection use case the latency is fraud detection use case the latency is fraud detection use case the latency is more like I want within few seconds more like I want within few seconds more like I want within few seconds right but if it's more for like offline right but if it's more for like offline right but if it's more for like offline analytics where I I have my warehouse I analytics where I I have my warehouse I analytics where I I have my warehouse I get data from like multiple sources ours get data from like multiple sources ours get data from like multiple sources ours of like latency is also fine right yeah of like latency is also fine right yeah of like latency is also fine right yeah this is a really great point that you're this is a really great point that you're this is a really great point that you're calling out because so many people just calling out because so many people just calling out because so many people just love to say real time they're like I love to say real time they're like I love to say real time they're like I need this done in real time but it's need this done in real time but it's need this done in real time but it's never really real time right if I have never really real time right if I have never really real time right if I have you know shopping cart data data about you know shopping cart data data about you know shopping cart data data about how my sales are going and then I have a how my sales are going and then I have a how my sales are going and then I have a rep which is going to be real time in rep which is going to be real time in rep which is going to be real time in the sense of within seconds I want to the sense of within seconds I want to the sense of within seconds I want to see inventories updated but then I'm see inventories updated but then I'm see inventories updated but then I'm going to do sales reporting for the going to do sales reporting for the going to do sales reporting for the quarter that's going to be a different quarter that's going to be a different quarter that's going to be a different database like you point out it's going database like you point out it's going database like you point out it's going to be a replication what is your to be a replication what is your to be a replication what is your freshness guarantee then it could be 30

freshness guarantee then it could be 30 freshness guarantee then it could be 30 seconds but it could be a couple of seconds but it could be a couple of seconds but it could be a couple of hours it really just depends on how much hours it really just depends on how much hours it really just depends on how much money you want to spend I assume More money you want to spend I assume More money you want to spend I assume More Money More freshness 100% and that is Money More freshness 100% and that is Money More freshness 100% and that is not only for the replication tools but not only for the replication tools but not only for the replication tools but that's also for the warehouses for that's also for the warehouses for that's also for the warehouses for example snowflake charges based on how example snowflake charges based on how example snowflake charges based on how active is the warehouse right so the active is the warehouse right so the active is the warehouse right so the more you uh real time you keep the more more you uh real time you keep the more more you uh real time you keep the more cost you would incur so you're right cost you would incur so you're right cost you would incur so you're right like more the latency more costs if I like more the latency more costs if I like more the latency more costs if I was going to do a one-time migration was going to do a one-time migration was going to do a one-time migration with no need of freshness if I'm doing a with no need of freshness if I'm doing a with no need of freshness if I'm doing a hobby project there's lots of tools hobby project there's lots of tools hobby project there's lots of tools that'll let me do that dozens of them that'll let me do that dozens of them that'll let me do that dozens of them they usually have PG something PG loader they usually have PG something PG loader they usually have PG something PG loader PG chameleon PG this PG that there all PG chameleon PG this PG that there all PG chameleon PG this PG that there all different ways to get my data into different ways to get my data into different ways to get my data into postgress but if I want like to move postgress but if I want like to move postgress but if I want like to move data into postgress to and from that's data into postgress to and from that's data into postgress to and from that's where you thought there was an where you thought there was an where you thought there was an opportunity here and you started a whole opportunity here and you started a whole opportunity here and you started a whole business because you think it's so business because you think it's so business because you think it's so important this deserves its own focused important this deserves its own focused important this deserves its own focused business yeah awesome so at PB what we business yeah awesome so at PB what we business yeah awesome so at PB what we doing Scott is like uh we building like doing Scott is like uh we building like doing Scott is like uh we building like a first class data movement tool for a first class data movement tool for a first class data movement tool for postris and what I mean by this is like postris and what I mean by this is like postris and what I mean by this is like any use case that involves moving data any use case that involves moving data any use case that involves moving data in and out of postris we want to provide in and out of postris we want to provide in and out of postris we want to provide like a fast native and a costeffective like a fast native and a costeffective like a fast native and a costeffective like tool to make this happen and and a like tool to make this happen and and a like tool to make this happen and and a tool that scales right like that's also tool that scales right like that's also tool that scales right like that's also very important let's get into like what very important let's get into like what very important let's get into like what are the use cases where like you know are the use cases where like you know are the use cases where like you know customers like move data in and out of customers like move data in and out of customers like move data in and out of post right like first is we talked about post right like first is we talked about post right like first is we talked about migration right like where okay I have a migration right like where okay I have a migration right like where okay I have a fleet of Oracle databases and I want to fleet of Oracle databases and I want to fleet of Oracle databases and I want to migrate to postgress right like that is migrate to postgress right like that is migrate to postgress right like that is one next is you have getting data into one next is you have getting data into one next is you have getting data into postgress another use cases postgress

postgress another use cases postgress postgress another use cases postgress has this like PG Vector extension which has this like PG Vector extension which has this like PG Vector extension which lets you store like embeddings and do lets you store like embeddings and do lets you store like embeddings and do Vector search right like in this use Vector search right like in this use Vector search right like in this use case you basically have like case you basically have like case you basically have like unstructured data stored across multiple unstructured data stored across multiple unstructured data stored across multiple data sources and you want to you know data sources and you want to you know data sources and you want to you know create these embeddings in real time and create these embeddings in real time and create these embeddings in real time and keep it in sync with like postgis right keep it in sync with like postgis right keep it in sync with like postgis right like you want to do Transformations like you want to do Transformations like you want to do Transformations where you convert this unstructured data where you convert this unstructured data where you convert this unstructured data to embeddings and put it to like a to embeddings and put it to like a to embeddings and put it to like a postris for like you know Vector search postris for like you know Vector search postris for like you know Vector search right like this is number two right now right like this is number two right now right like this is number two right now let's get into like use cases where you let's get into like use cases where you let's get into like use cases where you move data out of postris right like it move data out of postris right like it move data out of postris right like it is what we talked about the fraud is what we talked about the fraud is what we talked about the fraud detection use case where you have a detection use case where you have a detection use case where you have a warehouse or an analytical store where warehouse or an analytical store where warehouse or an analytical store where you replicate and keep uh data in you replicate and keep uh data in you replicate and keep uh data in postgress in sync with like a warehouse postgress in sync with like a warehouse postgress in sync with like a warehouse like snowflake or big query that is like snowflake or big query that is like snowflake or big query that is number three right so across like you number three right so across like you number three right so across like you know all these real world use cases at know all these real world use cases at know all these real world use cases at PV are building like a tool that just PV are building like a tool that just PV are building like a tool that just works I mean this ties into like what we works I mean this ties into like what we works I mean this ties into like what we discussed before where a tool is like discussed before where a tool is like discussed before where a tool is like you know it's fast it's like you know you know it's fast it's like you know you know it's fast it's like you know native right like it supports like you native right like it supports like you native right like it supports like you know native postris features right like know native postris features right like know native postris features right like say data types right like you want to say data types right like you want to say data types right like you want to move like uh data types in a very native move like uh data types in a very native move like uh data types in a very native way say I have data types in postgress way say I have data types in postgress way say I have data types in postgress which are like arrays and I want to which are like arrays and I want to which are like arrays and I want to translate and I want to replicate them translate and I want to replicate them translate and I want to replicate them to Aras in like my target which is snowf to Aras in like my target which is snowf to Aras in like my target which is snowf fle so fast native and last but not the fle so fast native and last but not the fle so fast native and last but not the least cost- effective right like I think least cost- effective right like I think least cost- effective right like I think that angle is also very important that angle is also very important that angle is also very important because one thing that we hear from because one thing that we hear from because one thing that we hear from customers is existing data movement customers is existing data movement customers is existing data movement tools are very expensive expensive right tools are very expensive expensive right tools are very expensive expensive right like so one angle we we want to go after like so one angle we we want to go after like so one angle we we want to go after is also commoditizing data movement is also commoditizing data movement is also commoditizing data movement right like why you know existing data right like why you know existing data right like why you know existing data movement tools are like very expensive

movement tools are like very expensive movement tools are like very expensive that is also one thing which we are that is also one thing which we are that is also one thing which we are thinking of right like so a fast native thinking of right like so a fast native thinking of right like so a fast native and a cost- effective way to move data and a cost- effective way to move data and a cost- effective way to move data in and out of postgress Def find PB when in and out of postgress Def find PB when in and out of postgress Def find PB when you say that it's faster and this is my you say that it's faster and this is my you say that it's faster and this is my ignorance speaking like you you have ignorance speaking like you you have ignorance speaking like you you have said on the on the open source said on the on the open source said on the on the open source repository for pdb that it's 10 times repository for pdb that it's 10 times repository for pdb that it's 10 times faster than existing tools like that's faster than existing tools like that's faster than existing tools like that's seems impossible when you talk about an seems impossible when you talk about an seems impossible when you talk about an order of magnitude faster how do things order of magnitude faster how do things order of magnitude faster how do things get faster is it protocol level is it get faster is it protocol level is it get faster is it protocol level is it CPU bound is it IO bound or is it all of CPU bound is it IO bound or is it all of CPU bound is it IO bound or is it all of those things that's a great question those things that's a great question those things that's a great question right so I I'll give a couple of right so I I'll give a couple of right so I I'll give a couple of examples to you know answer this examples to you know answer this examples to you know answer this question so let's take PG dump PG question so let's take PG dump PG question so let's take PG dump PG restore right like a utility where which restore right like a utility where which restore right like a utility where which helps you migrate data from a postris helps you migrate data from a postris helps you migrate data from a postris database to another postris database database to another postris database database to another postris database let's talk about PB right like let's let's talk about PB right like let's let's talk about PB right like let's compare like both of these right like compare like both of these right like compare like both of these right like one feature that PB does which is not one feature that PB does which is not one feature that PB does which is not present in like pgum PG rest store is present in like pgum PG rest store is present in like pgum PG rest store is something called as parallel something called as parallel something called as parallel snapshotting right like so what we do is snapshotting right like so what we do is snapshotting right like so what we do is we basically logically partition a large we basically logically partition a large we basically logically partition a large table based on tle identifiers for table based on tle identifiers for table based on tle identifiers for postgress these are like you know postgress these are like you know postgress these are like you know identifiers which represent data identifiers which represent data identifiers which represent data directly on disk basically so we directly on disk basically so we directly on disk basically so we logically partition it and then stream logically partition it and then stream logically partition it and then stream these partitions in parallel to the these partitions in parallel to the these partitions in parallel to the Target database right so this would like Target database right so this would like Target database right so this would like boost performance by orders of magnitude boost performance by orders of magnitude boost performance by orders of magnitude right say you have a terabyte like of right say you have a terabyte like of right say you have a terabyte like of postris database that you want to postris database that you want to postris database that you want to migrate to like the cloud if you were migrate to like the cloud if you were migrate to like the cloud if you were using PG PG store it would take like a using PG PG store it would take like a using PG PG store it would take like a couple of days but with PB because of couple of days but with PB because of couple of days but with PB because of parallel snapshotting because of parallel snapshotting because of parallel snapshotting because of parallelism and also directly reading parallelism and also directly reading parallelism and also directly reading from disk this would be more like an from disk this would be more like an from disk this would be more like an hours of time right and this would save

hours of time right and this would save hours of time right and this would save like you know the amount of downtime like you know the amount of downtime like you know the amount of downtime also right like basically because if you also right like basically because if you also right like basically because if you doing PG D PG store you need to take a doing PG D PG store you need to take a doing PG D PG store you need to take a downtime of like two three days but then downtime of like two three days but then downtime of like two three days but then like if you were using pdb it would be like if you were using pdb it would be like if you were using pdb it would be few hours right and this kind of few hours right and this kind of few hours right and this kind of translates to other use cases also Scot translates to other use cases also Scot translates to other use cases also Scot where like you're replicating data from where like you're replicating data from where like you're replicating data from postgis to snowflake right and the postgis to snowflake right and the postgis to snowflake right and the initial snapshot will be a Critic iCal initial snapshot will be a Critic iCal initial snapshot will be a Critic iCal part right currently our customers with part right currently our customers with part right currently our customers with existing tools wait for four five days existing tools wait for four five days existing tools wait for four five days for the 5 terab like postgress database for the 5 terab like postgress database for the 5 terab like postgress database to be replicated to snowflake but with to be replicated to snowflake but with to be replicated to snowflake but with PB that comes within a day basically PB that comes within a day basically PB that comes within a day basically right so this is one feature where like right so this is one feature where like right so this is one feature where like we give this performance how do you we give this performance how do you we give this performance how do you balance the you have you you you've balance the you have you you you've balance the you have you you you've started a company but you have an open started a company but you have an open started a company but you have an open source project you have a cloud offering source project you have a cloud offering source project you have a cloud offering coming when you do this open source for coming when you do this open source for coming when you do this open source for an open source database are you an open source database are you an open source database are you concerned that someone is going to take concerned that someone is going to take concerned that someone is going to take your kind of secret herbs and spices your kind of secret herbs and spices your kind of secret herbs and spices and move it into their own thing like and move it into their own thing like and move it into their own thing like how do you keep value you keep the light how do you keep value you keep the light how do you keep value you keep the light on while presenting such a huge amount on while presenting such a huge amount on while presenting such a huge amount of value for like 10x performance GES my of value for like 10x performance GES my of value for like 10x performance GES my roots are from open source because like roots are from open source because like roots are from open source because like I worked at a database startup called I worked at a database startup called I worked at a database startup called citus which was open source which got citus which was open source which got citus which was open source which got acquired by Microsoft right like and in acquired by Microsoft right like and in acquired by Microsoft right like and in Microsoft I was in the open source like Microsoft I was in the open source like Microsoft I was in the open source like databases team that is number one the databases team that is number one the databases team that is number one the second is even my co-founder comes from second is even my co-founder comes from second is even my co-founder comes from like an open source background he was like an open source background he was like an open source background he was like the ninth largest contributor of like the ninth largest contributor of like the ninth largest contributor of flutter which is like a huge open source flutter which is like a huge open source flutter which is like a huge open source like repository right so the reason we like repository right so the reason we like repository right so the reason we chose open source is because if we are chose open source is because if we are chose open source is because if we are building something around post it has to building something around post it has to building something around post it has to be open source right like that is what be open source right like that is what be open source right like that is what like the community prefers and that is like the community prefers and that is like the community prefers and that is the way we also like operate that is the way we also like operate that is the way we also like operate that is number one number two like obviously we

number one number two like obviously we number one number two like obviously we are running a business right and we it are running a business right and we it are running a business right and we it is important to make money for that is important to make money for that is important to make money for that piece right like we have like a managed piece right like we have like a managed piece right like we have like a managed offering where we make it super easy to offering where we make it super easy to offering where we make it super easy to like deploy and you know manage pdb we like deploy and you know manage pdb we like deploy and you know manage pdb we believe that like that will be important believe that like that will be important believe that like that will be important for customers who run workloads at scale for customers who run workloads at scale for customers who run workloads at scale and in production B basically right so and in production B basically right so and in production B basically right so with open source that would take like with open source that would take like with open source that would take like you know few months of effort right like you know few months of effort right like you know few months of effort right like to deploy to set up and manage whereas to deploy to set up and manage whereas to deploy to set up and manage whereas with the managed offering it's like more with the managed offering it's like more with the managed offering it's like more simple and the reason it's like open simple and the reason it's like open simple and the reason it's like open source takes few months is because of source takes few months is because of source takes few months is because of the nature of the problems Scot right the nature of the problems Scot right the nature of the problems Scot right like I mean we are like solving a like I mean we are like solving a like I mean we are like solving a problem which entails scale right like problem which entails scale right like problem which entails scale right like we are solving problem for customers we are solving problem for customers we are solving problem for customers like who run postgress at their heart of like who run postgress at their heart of like who run postgress at their heart of their data stack they have like tens of their data stack they have like tens of their data stack they have like tens of postgress databases or like they have postgress databases or like they have postgress databases or like they have terabytes of like postgress data right terabytes of like postgress data right terabytes of like postgress data right so from a revenue and a finance so from a revenue and a finance so from a revenue and a finance standpoint I consider open source to be standpoint I consider open source to be standpoint I consider open source to be more like a trust Builder right like more like a trust Builder right like more like a trust Builder right like then you know a threat that is number then you know a threat that is number then you know a threat that is number two and number three in regards to two and number three in regards to two and number three in regards to getting worried about like other getting worried about like other getting worried about like other competitors would use the magic sauce competitors would use the magic sauce competitors would use the magic sauce Etc I think that I I always believe that Etc I think that I I always believe that Etc I think that I I always believe that like you know we are building this I do like you know we are building this I do like you know we are building this I do this like 24 hours a day kind of a thing this like 24 hours a day kind of a thing this like 24 hours a day kind of a thing right like similar with my team right right like similar with my team right right like similar with my team right and we we are less worried about that and we we are less worried about that and we we are less worried about that because if someone is taking they cannot because if someone is taking they cannot because if someone is taking they cannot do as good as a job as us right like do as good as a job as us right like do as good as a job as us right like Scott so these are the reasons like you Scott so these are the reasons like you Scott so these are the reasons like you know we chose open source one is you know we chose open source one is you know we chose open source one is you know trust next is you know it won't be know trust next is you know it won't be know trust next is you know it won't be a barrier for our like Revenue Etc a barrier for our like Revenue Etc a barrier for our like Revenue Etc rather it would be an enabler and third rather it would be an enabler and third rather it would be an enabler and third is you know we are not worried that is you know we are not worried that is you know we are not worried that another player would take this and like another player would take this and like another player would take this and like you know uh uh run this on their own you know uh uh run this on their own you know uh uh run this on their own because as we're doing this all the time because as we're doing this all the time because as we're doing this all the time we will do a better job at it yeah yeah we will do a better job at it yeah yeah we will do a better job at it yeah yeah yeah was it hard to pick hostress as the

yeah was it hard to pick hostress as the yeah was it hard to pick hostress as the center of the Hub like you're not a center of the Hub like you're not a center of the Hub like you're not a generic maybe I'm speaking wrong here generic maybe I'm speaking wrong here generic maybe I'm speaking wrong here but you're not that you're not a generic but you're not that you're not a generic but you're not that you're not a generic data mover you're a postgress focused data mover you're a postgress focused data mover you're a postgress focused data mover but you'll go to Kafka you'll data mover but you'll go to Kafka you'll data mover but you'll go to Kafka you'll go to event hubs you'll go to snowflake go to event hubs you'll go to snowflake go to event hubs you'll go to snowflake you'll go to big query but you didn't go you'll go to big query but you didn't go you'll go to big query but you didn't go many to to many you know what I mean you many to to many you know what I mean you many to to many you know what I mean you went one to many was why why postgress went one to many was why why postgress went one to many was why why postgress like we talked about why it's popular like we talked about why it's popular like we talked about why it's popular was that a hard decision or do you think was that a hard decision or do you think was that a hard decision or do you think you'll go many to many someday we want you'll go many to many someday we want you'll go many to many someday we want to build a data movement tool whose to build a data movement tool whose to build a data movement tool whose focus is quality over breadth Scott focus is quality over breadth Scott focus is quality over breadth Scott right like I think that is one important right like I think that is one important right like I think that is one important like premise on which like PB is built like premise on which like PB is built like premise on which like PB is built that's the reason you know any connector that's the reason you know any connector that's the reason you know any connector we built right let it be the source or we built right let it be the source or we built right let it be the source or the target you want to provide a the target you want to provide a the target you want to provide a connector that works with at scale connector that works with at scale connector that works with at scale basically that's the reason you know we basically that's the reason you know we basically that's the reason you know we are like building for postgress because are like building for postgress because are like building for postgress because that's where my background lies right that's where my background lies right that's where my background lies right like and also there is a need right like like and also there is a need right like like and also there is a need right like because we we talked about this right because we we talked about this right because we we talked about this right like postgress from a scale standpoint like postgress from a scale standpoint like postgress from a scale standpoint from a use cases standpoint is growing from a use cases standpoint is growing from a use cases standpoint is growing right like the amount of data being right like the amount of data being right like the amount of data being stored and moved through postgress is stored and moved through postgress is stored and moved through postgress is significantly growing that's the reason significantly growing that's the reason significantly growing that's the reason we started with postgress and then you we started with postgress and then you we started with postgress and then you know down the line if you ask me whether know down the line if you ask me whether know down the line if you ask me whether we will expand or not our plan is to we will expand or not our plan is to we will expand or not our plan is to like nail like the postris data movement like nail like the postris data movement like nail like the postris data movement story right and we believe that like story right and we believe that like story right and we believe that like this peer-to-peer architecture that this peer-to-peer architecture that this peer-to-peer architecture that we're doing where like we take two data we're doing where like we take two data we're doing where like we take two data stores and provide like the the best stores and provide like the the best stores and provide like the the best possible experience can be expanded to possible experience can be expanded to possible experience can be expanded to like other operational data stores as like other operational data stores as like other operational data stores as well we go into like MySQL or like well we go into like MySQL or like well we go into like MySQL or like mongodb etc etc so this is one new thing mongodb etc etc so this is one new thing mongodb etc etc so this is one new thing that like you know we are defining where that like you know we are defining where that like you know we are defining where a data movement tool should be built a data movement tool should be built a data movement tool should be built with a peer-to-peer architecture than a with a peer-to-peer architecture than a with a peer-to-peer architecture than a huband Spoke architecture peer-to-peer

huband Spoke architecture peer-to-peer huband Spoke architecture peer-to-peer architecture is more like you take a architecture is more like you take a architecture is more like you take a couple of data stores and provide like couple of data stores and provide like couple of data stores and provide like the best possible experience then you the best possible experience then you the best possible experience then you know you focus on more breadth right know you focus on more breadth right know you focus on more breadth right like where you convert like data from like where you convert like data from like where you convert like data from all these sources to a common format you all these sources to a common format you all these sources to a common format you lose out a lot and won't be that lose out a lot and won't be that lose out a lot and won't be that reliable right so this is one thing that reliable right so this is one thing that reliable right so this is one thing that we are defining with pdb down the line we are defining with pdb down the line we are defining with pdb down the line once we nail postgress sta movement once we nail postgress sta movement once we nail postgress sta movement story for postgress we will expand to story for postgress we will expand to story for postgress we will expand to other operational stores as well that other operational stores as well that other operational stores as well that makes sense it seems like it's a good makes sense it seems like it's a good makes sense it seems like it's a good place to start though like you are a you place to start though like you are a you place to start though like you are a you know you're an ETL tool for postgress know you're an ETL tool for postgress know you're an ETL tool for postgress and you're also implementing stuff at a and you're also implementing stuff at a and you're also implementing stuff at a native level like this is not a generic native level like this is not a generic native level like this is not a generic you know like when I do data movement you know like when I do data movement you know like when I do data movement it's usually a a for Loop this is using it's usually a a for Loop this is using it's usually a a for Loop this is using native infrastructure optimizations that native infrastructure optimizations that native infrastructure optimizations that are specific to to postgress like that are specific to to postgress like that are specific to to postgress like that IAL load is parallelized and the things IAL load is parallelized and the things IAL load is parallelized and the things that you talked about one of the other that you talked about one of the other that you talked about one of the other things that I find really challenging is things that I find really challenging is things that I find really challenging is is Fault tolerance like it's bigger than is Fault tolerance like it's bigger than is Fault tolerance like it's bigger than just retries isn't it yes it is bigger just retries isn't it yes it is bigger just retries isn't it yes it is bigger than retries and that is one very than retries and that is one very than retries and that is one very important thing for example in pdb we important thing for example in pdb we important thing for example in pdb we have like a state machine which make have like a state machine which make have like a state machine which make sure that if something fails right like sure that if something fails right like sure that if something fails right like say the source crashes or like you know say the source crashes or like you know say the source crashes or like you know PB crashes right like it can happen or PB crashes right like it can happen or PB crashes right like it can happen or like the target crashes right like we like the target crashes right like we like the target crashes right like we Auto recover right like for this like Auto recover right like for this like Auto recover right like for this like there's a robust State machine for there's a robust State machine for there's a robust State machine for example in our par initial load the item example in our par initial load the item example in our par initial load the item potency is at a partition level right potency is at a partition level right potency is at a partition level right like we move data in bulk and if one of like we move data in bulk and if one of like we move data in bulk and if one of the things crashes we don't start the the things crashes we don't start the the things crashes we don't start the entire thing again right like we keep entire thing again right like we keep entire thing again right like we keep track of okay what partitions have been track of okay what partitions have been track of okay what partitions have been done what partitions have not been done done what partitions have not been done done what partitions have not been done and then you know like we have Auto and then you know like we have Auto and then you know like we have Auto retry mechanism and we use this like

retry mechanism and we use this like retry mechanism and we use this like open source tool called temporal uh open source tool called temporal uh open source tool called temporal uh which helps with this so that is also which helps with this so that is also which helps with this so that is also one thing which makes it like fall one thing which makes it like fall one thing which makes it like fall tolerant and like you know has Auto tolerant and like you know has Auto tolerant and like you know has Auto retries Etc and that's the difference retries Etc and that's the difference retries Etc and that's the difference between like a tool and a platform right between like a tool and a platform right between like a tool and a platform right like I mean here like you know we are like I mean here like you know we are like I mean here like you know we are building something that works in building something that works in building something that works in production basically interesting so production basically interesting so production basically interesting so temporal basically includes things like temporal basically includes things like temporal basically includes things like durable execution abstractions and durable execution abstractions and durable execution abstractions and things like that uh recovery logic all things like that uh recovery logic all things like that uh recovery logic all that stuff is built in so are you that stuff is built in so are you that stuff is built in so are you building on top of that open source building on top of that open source building on top of that open source project to make your execution durable project to make your execution durable project to make your execution durable exactly so it's basically an exactly so it's basically an exactly so it's basically an orchestration tool so it lets you like orchestration tool so it lets you like orchestration tool so it lets you like Define like activities work flows and Define like activities work flows and Define like activities work flows and all of that right so but obviously we all of that right so but obviously we all of that right so but obviously we leverage temporal so we build on top of leverage temporal so we build on top of leverage temporal so we build on top of it right we need to Define what are our it right we need to Define what are our it right we need to Define what are our activities what are our workflows Etc activities what are our workflows Etc activities what are our workflows Etc and then it has like mechanisms which and then it has like mechanisms which and then it has like mechanisms which will Auto retry say an activity fails will Auto retry say an activity fails will Auto retry say an activity fails for some reason right like say the for some reason right like say the for some reason right like say the source is down and like you know the source is down and like you know the source is down and like you know the activity fails right then it has activity fails right then it has activity fails right then it has mechanisms to retry basically and uh mechanisms to retry basically and uh mechanisms to retry basically and uh this is one very core component of our this is one very core component of our this is one very core component of our architecture now obviously it's very I architecture now obviously it's very I architecture now obviously it's very I can go into like how you know build to can go into like how you know build to can go into like how you know build to build stuff in temporal but no of course build stuff in temporal but no of course build stuff in temporal but no of course of course but I think what I'm hearing of course but I think what I'm hearing of course but I think what I'm hearing you say though is that if you're going you say though is that if you're going you say though is that if you're going to do these kind of complicated to do these kind of complicated to do these kind of complicated workflows you're going to synchronize workflows you're going to synchronize workflows you're going to synchronize data you're building on top of well data you're building on top of well data you're building on top of well understood Concepts you're not understood Concepts you're not understood Concepts you're not Reinventing the wheel from a workflow Reinventing the wheel from a workflow Reinventing the wheel from a workflow perspective and then you're using perspective and then you're using perspective and then you're using temporal to capture those changes to the temporal to capture those changes to the temporal to capture those changes to the database but then you're dividing up database but then you're dividing up database but then you're dividing up that process that CDC process up into that process that CDC process up into that process that CDC process up into activities and that gives you kind of a activities and that gives you kind of a activities and that gives you kind of a Rel a more reliable data synchronization

Rel a more reliable data synchronization Rel a more reliable data synchronization 100% 100% And like I I mean I I'll be 100% 100% And like I I mean I I'll be 100% 100% And like I I mean I I'll be very transparent here like we had like a very transparent here like we had like a very transparent here like we had like a five minutes outage yesterday because of five minutes outage yesterday because of five minutes outage yesterday because of some reason where like ipv4 IPv6 some reason where like ipv4 IPv6 some reason where like ipv4 IPv6 transition Etc and the beauty of like transition Etc and the beauty of like transition Etc and the beauty of like this state machine that we built on this state machine that we built on this state machine that we built on temporal is it just Auto recovered so temporal is it just Auto recovered so temporal is it just Auto recovered so like we have like tens of like PB like we have like tens of like PB like we have like tens of like PB instances running on the cloud and 5 instances running on the cloud and 5 instances running on the cloud and 5 minutes there was an outage and it came minutes there was an outage and it came minutes there was an outage and it came up and that didn't impact customers that up and that didn't impact customers that up and that didn't impact customers that much the reason is because you know the much the reason is because you know the much the reason is because you know the latency five minutes is totally fine latency five minutes is totally fine latency five minutes is totally fine right like and then five minutes like it right like and then five minutes like it right like and then five minutes like it came up and because of the state machine came up and because of the state machine came up and because of the state machine and temporal it just like you know and temporal it just like you know and temporal it just like you know continued from where it left off and we continued from where it left off and we continued from where it left off and we didn't have to do anything basically didn't have to do anything basically didn't have to do anything basically Squad so this happened yesterday night Squad so this happened yesterday night Squad so this happened yesterday night so I was awake till like 3:00 a.m. but so I was awake till like 3:00 a.m. but so I was awake till like 3:00 a.m. but see you know that's both you know I'm see you know that's both you know I'm see you know that's both you know I'm sorry that you had a 5 minute outage but sorry that you had a 5 minute outage but sorry that you had a 5 minute outage but also just the fact that we live in a also just the fact that we live in a also just the fact that we live in a world where you have to apologize for a world where you have to apologize for a world where you have to apologize for a five minute outage is uh is surprising five minute outage is uh is surprising five minute outage is uh is surprising to me but it's so cool when stuff like to me but it's so cool when stuff like to me but it's so cool when stuff like that just works it kind of validates I that just works it kind of validates I that just works it kind of validates I mean it must be stressful because you mean it must be stressful because you mean it must be stressful because you were up but doesn't it kind of validate were up but doesn't it kind of validate were up but doesn't it kind of validate your architecture and you go you know we your architecture and you go you know we your architecture and you go you know we didn't lose a bite we didn't lose a row didn't lose a bite we didn't lose a row didn't lose a bite we didn't lose a row and for five minutes we had a heart and for five minutes we had a heart and for five minutes we had a heart attack but everything is cool now and it attack but everything is cool now and it attack but everything is cool now and it retried and it got it that robustness is retried and it got it that robustness is retried and it got it that robustness is there exactly we we thought of it the there exactly we we thought of it the there exactly we we thought of it the same way because it was our first outage same way because it was our first outage same way because it was our first outage for 5 minutes and then it really like for 5 minutes and then it really like for 5 minutes and then it really like you know validated the design right like you know validated the design right like you know validated the design right like and then we were like looking at each and then we were like looking at each and then we were like looking at each replication job whether this is like replication job whether this is like replication job whether this is like continuing ET and it was like we didn't continuing ET and it was like we didn't continuing ET and it was like we didn't have to no manual intervention Scot it have to no manual intervention Scot it have to no manual intervention Scot it just continued so you know that makes me just continued so you know that makes me just continued so you know that makes me wonder if like things like chaos wonder if like things like chaos wonder if like things like chaos engineering and you know randomly engineering and you know randomly engineering and you know randomly pulling the plug would be a good way to pulling the plug would be a good way to pulling the plug would be a good way to validate Your Design I'm sure you do validate Your Design I'm sure you do validate Your Design I'm sure you do have you know robustness tests and

have you know robustness tests and have you know robustness tests and things like that but this was your first things like that but this was your first things like that but this was your first battle tested production down time for a battle tested production down time for a battle tested production down time for a moment there exactly exactly we have moment there exactly exactly we have moment there exactly exactly we have those mechanisms where where we do scale those mechanisms where where we do scale those mechanisms where where we do scale testing and where we bring things down testing and where we bring things down testing and where we bring things down and like you know make sure that like it and like you know make sure that like it and like you know make sure that like it works but then at a scale right like I works but then at a scale right like I works but then at a scale right like I mean these were like multiple customer mean these were like multiple customer mean these were like multiple customer deployments right that were getting deployments right that were getting deployments right that were getting affected so it was like more real than affected so it was like more real than affected so it was like more real than like our internal tests called so it like our internal tests called so it like our internal tests called so it sounds like you're hoping that pdb is sounds like you're hoping that pdb is sounds like you're hoping that pdb is going to become a tool and everyone's going to become a tool and everyone's going to become a tool and everyone's tool box and it's going to sit alongside tool box and it's going to sit alongside tool box and it's going to sit alongside all those other great tools people use all those other great tools people use all those other great tools people use every you know we think of tools like PG every you know we think of tools like PG every you know we think of tools like PG admin and things like that but you know admin and things like that but you know admin and things like that but you know grafana and Tableau and all the grafana and Tableau and all the grafana and Tableau and all the different places that are consuming data different places that are consuming data different places that are consuming data pdb will be in the center of that Hub pdb will be in the center of that Hub pdb will be in the center of that Hub and start to move that data wherever and start to move that data wherever and start to move that data wherever wants to beay the place where we fit in wants to beay the place where we fit in wants to beay the place where we fit in right Scott like I mean the modern data right Scott like I mean the modern data right Scott like I mean the modern data stack typically what happens is you have stack typically what happens is you have stack typically what happens is you have a bunch of data sources from which you a bunch of data sources from which you a bunch of data sources from which you move data to like the warehouse and then move data to like the warehouse and then move data to like the warehouse and then in Warehouse like you know you do your in Warehouse like you know you do your in Warehouse like you know you do your like Transformations Etc and you have like Transformations Etc and you have like Transformations Etc and you have like you know tblo powerbi and these like you know tblo powerbi and these like you know tblo powerbi and these kind of like tools which consume data kind of like tools which consume data kind of like tools which consume data from like you know the warehouse it from like you know the warehouse it from like you know the warehouse it could be like custom applications also could be like custom applications also could be like custom applications also and then they do insights basically and then they do insights basically and then they do insights basically right so the way PB fits in is the data right so the way PB fits in is the data right so the way PB fits in is the data movement piece right like okay you have movement piece right like okay you have movement piece right like okay you have a bunch of data sources but majority of a bunch of data sources but majority of a bunch of data sources but majority of the data to your Warehouse is coming the data to your Warehouse is coming the data to your Warehouse is coming from postgress basically right because from postgress basically right because from postgress basically right because that's your system of record that's that's your system of record that's that's your system of record that's where that's on which your business is where that's on which your business is where that's on which your business is running that piece we want to like super running that piece we want to like super running that piece we want to like super optimize we will be complimenting other optimize we will be complimenting other optimize we will be complimenting other tools a good fit would be a customer who tools a good fit would be a customer who tools a good fit would be a customer who is running postgress at their heart of is running postgress at their heart of is running postgress at their heart of the data stack and they move majority of the data stack and they move majority of the data stack and they move majority of the data to their warehouse from

the data to their warehouse from the data to their warehouse from postgress that will be like a good fit postgress that will be like a good fit postgress that will be like a good fit for like PB that's cool and then from for like PB that's cool and then from for like PB that's cool and then from data freshness perspective you said it data freshness perspective you said it data freshness perspective you said it can be tens of seconds uh even at large can be tens of seconds uh even at large can be tens of seconds uh even at large throughputs but it can also be you know throughputs but it can also be you know throughputs but it can also be you know whatever you set it really comes down to whatever you set it really comes down to whatever you set it really comes down to how hard you want to you how big the how hard you want to you how big the how hard you want to you how big the pipe is and how much data you have pipe is and how much data you have pipe is and how much data you have coming in even if it's tens of thousands coming in even if it's tens of thousands coming in even if it's tens of thousands of transactions a second you can keep uh of transactions a second you can keep uh of transactions a second you can keep uh with that change data capture you can with that change data capture you can with that change data capture you can keep things fresh cut exactly right like keep things fresh cut exactly right like keep things fresh cut exactly right like I think that really depends on uh the I think that really depends on uh the I think that really depends on uh the customer the refresh interval is customer the refresh interval is customer the refresh interval is completely configurable right like they completely configurable right like they completely configurable right like they can specify they want to do this sync can specify they want to do this sync can specify they want to do this sync every 1 hour or like every few minutes every 1 hour or like every few minutes every 1 hour or like every few minutes or every few seconds and and that is or every few seconds and and that is or every few seconds and and that is completely configurable and this depends completely configurable and this depends completely configurable and this depends on the use case like we have a customer on the use case like we have a customer on the use case like we have a customer who is doing fraud detection for who is doing fraud detection for who is doing fraud detection for invoices and then they you know have it invoices and then they you know have it invoices and then they you know have it set to like more a minute basically and set to like more a minute basically and set to like more a minute basically and then we have another customer who is then we have another customer who is then we have another customer who is doing it more for like you know doing it more for like you know doing it more for like you know centralizing data from postgress and centralizing data from postgress and centralizing data from postgress and like they have offline analytics they like they have offline analytics they like they have offline analytics they have machine learning jobs that are have machine learning jobs that are have machine learning jobs that are running they you know do it like every running they you know do it like every running they you know do it like every two hours so that is completely two hours so that is completely two hours so that is completely configurable and editable also so like configurable and editable also so like configurable and editable also so like once the replication job like starts you once the replication job like starts you once the replication job like starts you can also edit this like refresh inel if can also edit this like refresh inel if can also edit this like refresh inel if there are new use cases and you want it there are new use cases and you want it there are new use cases and you want it more real time sure like you make one more real time sure like you make one more real time sure like you make one hour to like 10 seconds I as a small hour to like 10 seconds I as a small hour to like 10 seconds I as a small user can go and try the open source user can go and try the open source user can go and try the open source product I can do like micro you know product I can do like micro you know product I can do like micro you know small amounts of data but this this small amounts of data but this this small amounts of data but this this could be used for you know billions of could be used for you know billions of could be used for you know billions of active rows There's No Limit this that's active rows There's No Limit this that's active rows There's No Limit this that's so cool that I could use this on my so cool that I could use this on my so cool that I could use this on my computer but I could also use the hosted computer but I could also use the hosted computer but I could also use the hosted Cloud option to move billions of rows Cloud option to move billions of rows Cloud option to move billions of rows around at the same time absolutely and

around at the same time absolutely and around at the same time absolutely and yesterday we were looking at how much yesterday we were looking at how much yesterday we were looking at how much data we moved in 3 hours basically it it data we moved in 3 hours basically it it data we moved in 3 hours basically it it came to 3 terabytes so in 3 hours like came to 3 terabytes so in 3 hours like came to 3 terabytes so in 3 hours like across all the customers right PP is across all the customers right PP is across all the customers right PP is moving around 3 terabytes basically so moving around 3 terabytes basically so moving around 3 terabytes basically so if you calculate like for a month it if you calculate like for a month it if you calculate like for a month it would be like hundreds of terabytes would be like hundreds of terabytes would be like hundreds of terabytes right so the system we have designed is right so the system we have designed is right so the system we have designed is works at scale basically right like works at scale basically right like works at scale basically right like billions of rows like you know terabytes billions of rows like you know terabytes billions of rows like you know terabytes of data right like so yeah so that's of data right like so yeah so that's of data right like so yeah so that's pretty cool people can check you out at pretty cool people can check you out at pretty cool people can check you out at pb. and they can try it for free they pb. and they can try it for free they pb. and they can try it for free they can check it out you can also go and get can check it out you can also go and get can check it out you can also go and get involved in the community there's a involved in the community there's a involved in the community there's a quick start there's also a link to the quick start there's also a link to the quick start there's also a link to the GitHub where folks can take a look at GitHub where folks can take a look at GitHub where folks can take a look at the source code and the build t that is the source code and the build t that is the source code and the build t that is up on GitHub thanks so much for chatting up on GitHub thanks so much for chatting up on GitHub thanks so much for chatting with me today and for sharing what with me today and for sharing what with me today and for sharing what you're working on thanks Scot like you're working on thanks Scot like you're working on thanks Scot like thanks for the opportunity and like it's thanks for the opportunity and like it's thanks for the opportunity and like it's great chatting with you so yeah I've great chatting with you so yeah I've great chatting with you so yeah I've been chatting with Sai Krishna shumer been chatting with Sai Krishna shumer been chatting with Sai Krishna shumer he's the CEO and co-founder of Pier DB he's the CEO and co-founder of Pier DB he's the CEO and co-founder of Pier DB this has been another episode of Hansel this has been another episode of Hansel this has been another episode of Hansel minutes and we'll see you again next minutes and we'll see you again next minutes and we'll see you again next [Music]