Harness the Power of Kubernetes with Pod Disruption Budgets

hello and welcome to this short hello and welcome to this short kubernetes tutorial today we're going to kubernetes tutorial today we're going to kubernetes tutorial today we're going to talk about pot disruption budgets what talk about pot disruption budgets what talk about pot disruption budgets what are they and what function to they are they and what function to they are they and what function to they perform so I have an AKs cluster running perform so I have an AKs cluster running perform so I have an AKs cluster running here in my Azure tenant and I'm just here in my Azure tenant and I'm just here in my Azure tenant and I'm just going to show you some files that I have going to show you some files that I have going to show you some files that I have here I have a backend and front end yo here I have a backend and front end yo here I have a backend and front end yo these are just small these are just small these are just small deployments and I'm going to apply all deployments and I'm going to apply all deployments and I'm going to apply all of these files Now by just running K of these files Now by just running K of these files Now by just running K apply K apply K apply K Dot and what I'm doing here is I have a Dot and what I'm doing here is I have a Dot and what I'm doing here is I have a customization file and this calls all of customization file and this calls all of customization file and this calls all of the other files so I can very quickly the other files so I can very quickly the other files so I can very quickly apply all of these files and what it has apply all of these files and what it has apply all of these files and what it has done it has um if I switch to the Misha done it has um if I switch to the Misha done it has um if I switch to the Misha namespace I just created what it has done it has created created what it has done it has created a back end deployment here and a front a back end deployment here and a front a back end deployment here and a front end deployment so if I get deploy here end deployment so if I get deploy here end deployment so if I get deploy here so I have a deployment of back end with so I have a deployment of back end with so I have a deployment of back end with one replica and a deployment of a front one replica and a deployment of a front one replica and a deployment of a front end with three end with three end with three replicas and now I have my AKs cluster replicas and now I have my AKs cluster replicas and now I have my AKs cluster and I'm just going to quickly start and I'm just going to quickly start and I'm just going to quickly start a upgrade of the a upgrade of the a upgrade of the cluster so we go to upgrade version and cluster so we go to upgrade version and cluster so we go to upgrade version and we are just going to upgrade to we are just going to upgrade to we are just going to upgrade to 127 and this takes a little bit of time 127 and this takes a little bit of time 127 and this takes a little bit of time so I'm going to start this before I do so I'm going to start this before I do so I'm going to start this before I do my explanation so my explanation so my explanation so what is happening in the background now

what is happening in the background now what is happening in the background now is that kubernetes is going to is that kubernetes is going to is that kubernetes is going to take my three nodes or I have two nodes take my three nodes or I have two nodes take my three nodes or I have two nodes here here I have two nodes and what here here I have two nodes and what here here I have two nodes and what kubernetes is going to do is it's going kubernetes is going to do is it's going kubernetes is going to do is it's going to create a new node in the system node to create a new node in the system node to create a new node in the system node pool and it is going to uh spin that up pool and it is going to uh spin that up pool and it is going to uh spin that up with the new version and then it's going with the new version and then it's going with the new version and then it's going to drain this old node so all of the to drain this old node so all of the to drain this old node so all of the pods that are on that node you can see here that it has um CP AKs you can see here that it has um CP AKs system we can see here that it system we can see here that it system we can see here that it has quite a few pods R running on it has quite a few pods R running on it has quite a few pods R running on it right so it's going to take each of right so it's going to take each of right so it's going to take each of these pods and reschedule it to the new these pods and reschedule it to the new these pods and reschedule it to the new node and that takes some time so the new node and that takes some time so the new node and that takes some time so the new node is spinning up now and it takes node is spinning up now and it takes node is spinning up now and it takes some time but as it is going to um re some time but as it is going to um re some time but as it is going to um re schedule all of these pods what can schedule all of these pods what can schedule all of these pods what can happen if you don't have configured happen if you don't have configured happen if you don't have configured anything is that if all of these pods anything is that if all of these pods anything is that if all of these pods were running on the same node you were running on the same node you were running on the same node you actually risk that kubernetes evicts all actually risk that kubernetes evicts all actually risk that kubernetes evicts all of those Pods at the same time and that of those Pods at the same time and that of those Pods at the same time and that we we want to avoid that right we don't we we want to avoid that right we don't we we want to avoid that right we don't want that to happen because our want that to happen because our want that to happen because our application is running in three replicas application is running in three replicas application is running in three replicas and the whole idea with that is that we and the whole idea with that is that we and the whole idea with that is that we want to have it highly available to make want to have it highly available to make want to have it highly available to make sure that there are no sure that there are no sure that there are no disruptions and how do we do that how do

disruptions and how do we do that how do disruptions and how do we do that how do we make sure that when we reschedule a we make sure that when we reschedule a we make sure that when we reschedule a workload to a new workload to a new workload to a new node um that it is going to make sure node um that it is going to make sure node um that it is going to make sure that there is at least one replica that there is at least one replica that there is at least one replica available at all times well that's where available at all times well that's where available at all times well that's where the Pod disruption budgets come in so if the Pod disruption budgets come in so if the Pod disruption budgets come in so if I do K get pod disruption budget we now see that we have two pod budget we now see that we have two pod disruptions here pod disruption budgets disruptions here pod disruption budgets disruptions here pod disruption budgets here these I have configured in my yaml here these I have configured in my yaml here these I have configured in my yaml files and we'll take a look at the yl in files and we'll take a look at the yl in files and we'll take a look at the yl in a a a moment but what we see here is we have a moment but what we see here is we have a moment but what we see here is we have a backend pdb and a front end backend pdb and a front end backend pdb and a front end pdb they have a Min available of one and pdb they have a Min available of one and pdb they have a Min available of one and they have allow disruptions of zero and they have allow disruptions of zero and they have allow disruptions of zero and two so our frontend deployment is two so our frontend deployment is two so our frontend deployment is running with three replicas and the way running with three replicas and the way running with three replicas and the way I've configured it is that it has an I've configured it is that it has an I've configured it is that it has an allowance of two disruptions so what I'm allowance of two disruptions so what I'm allowance of two disruptions so what I'm effectively saying is effectively saying is effectively saying is that one of them must be available at that one of them must be available at that one of them must be available at all times and all times and all times and therefore if I show you the yaml of this therefore if I show you the yaml of this therefore if I show you the yaml of this file so um frontend on yl here we see file so um frontend on yl here we see file so um frontend on yl here we see the frontend PO disruption budget here I the frontend PO disruption budget here I the frontend PO disruption budget here I have specified that I must have one Min have specified that I must have one Min have specified that I must have one Min available so at all time times one of available so at all time times one of available so at all time times one of these pods needs to be available at all these pods needs to be available at all these pods needs to be available at all times and therefore the Pod disruption

times and therefore the Pod disruption times and therefore the Pod disruption is now has now the the PO disruption is now has now the the PO disruption is now has now the the PO disruption budget has calculated say oh I have budget has calculated say oh I have budget has calculated say oh I have three I have a Max of one so I can have three I have a Max of one so I can have three I have a Max of one so I can have an allowance of two disruptions it works an allowance of two disruptions it works an allowance of two disruptions it works very clean and the way it works I was very clean and the way it works I was very clean and the way it works I was given a presentation on this um at work given a presentation on this um at work given a presentation on this um at work uh a month AG go and then I came up with uh a month AG go and then I came up with uh a month AG go and then I came up with this metaphor that I want to share with this metaphor that I want to share with this metaphor that I want to share with you so it is very similar to this you so it is very similar to this you so it is very similar to this situation like I'm I'm old enough to situation like I'm I'm old enough to situation like I'm I'm old enough to have been in a post office we don't have have been in a post office we don't have have been in a post office we don't have post offices anymore in the Netherlands post offices anymore in the Netherlands post offices anymore in the Netherlands as far as I'm aware but you have you as far as I'm aware but you have you as far as I'm aware but you have you have these situations where you have have these situations where you have have these situations where you have where you have a few tellers sitting where you have a few tellers sitting where you have a few tellers sitting behind a counter like this right now behind a counter like this right now behind a counter like this right now let's say this guy is is the manager let's say this guy is is the manager let's say this guy is is the manager he's the manager and he manages the he's the manager and he manages the he's the manager and he manages the three tellers that are sitting here and three tellers that are sitting here and three tellers that are sitting here and helping helping helping clients now in a situation where there clients now in a situation where there clients now in a situation where there is no PO disruption budget in is no PO disruption budget in is no PO disruption budget in kubernetes we risk that if we drain a kubernetes we risk that if we drain a kubernetes we risk that if we drain a node that all of these PS get evicted at node that all of these PS get evicted at node that all of these PS get evicted at the same time and the equivalent would the same time and the equivalent would the same time and the equivalent would be well we have three tellers and all of be well we have three tellers and all of be well we have three tellers and all of them want to take take a coffee break them want to take take a coffee break them want to take take a coffee break and they're going to do it all at the and they're going to do it all at the and they're going to do it all at the same time well what is going to happen same time well what is going to happen same time well what is going to happen if there's nobody here to help the if there's nobody here to help the if there's nobody here to help the customers you're going to have a large

customers you're going to have a large customers you're going to have a large line of customers and they are going to line of customers and they are going to line of customers and they are going to be all very unhappy because they are not be all very unhappy because they are not be all very unhappy because they are not getting any help so what this manager is getting any help so what this manager is getting any help so what this manager is going to say he's going to say guys it's going to say he's going to say guys it's going to say he's going to say guys it's fine that you take a break every once in fine that you take a break every once in fine that you take a break every once in a while but please communicate with each a while but please communicate with each a while but please communicate with each other to make sure that at least one of other to make sure that at least one of other to make sure that at least one of you guys is sitting here to help the you guys is sitting here to help the you guys is sitting here to help the customers so that our customers remain customers so that our customers remain customers so that our customers remain happy so these these bank tellers are happy so these these bank tellers are happy so these these bank tellers are then the pods in the metaphor and this then the pods in the metaphor and this then the pods in the metaphor and this guy here you know what he is he is the guy here you know what he is he is the guy here you know what he is he is the Pod disruption budget he is the Pod disruption budget he is the Pod disruption budget he is the manager um specifying okay we need at manager um specifying okay we need at manager um specifying okay we need at least one here so two of you guys can least one here so two of you guys can least one here so two of you guys can take a break at the same time you can take a break at the same time you can take a break at the same time you can catch up on the catch up on the catch up on the weekend weekend weekend so so so this guy is the manager and let's see if this guy is the manager and let's see if this guy is the manager and let's see if our um po disruption budget uh is doing our um po disruption budget uh is doing our um po disruption budget uh is doing its work already if I get the nodes its work already if I get the nodes its work already if I get the nodes now I now see that I have now I now see that I have now I now see that I have a uh okay so my workload node has been uh okay so my workload node has been replaced that has gone successfully but replaced that has gone successfully but replaced that has gone successfully but now we see I have two system nodes here now we see I have two system nodes here now we see I have two system nodes here right I have one of version 126 and I right I have one of version 126 and I right I have one of version 126 and I have one of uh version have one of uh version have one of uh version 127 127 127 well what is happening here is that the well what is happening here is that the well what is happening here is that the node actually is unable to drain

node actually is unable to drain node actually is unable to drain now because I have a poish ruption now because I have a poish ruption now because I have a poish ruption budget and how can we see that we can budget and how can we see that we can budget and how can we see that we can see that in the events so if I just get see that in the events so if I just get see that in the events so if I just get the events of the entire cluster K get the events of the entire cluster K get the events of the entire cluster K get events d a let's see do we see it here events d a let's see do we see it here events d a let's see do we see it here already no no well let a grap for already no no well let a grap for already no no well let a grap for pdb grap pdb and here we go here we see pdb grap pdb and here we go here we see pdb grap pdb and here we go here we see um actually my head is probably blocking um actually my head is probably blocking um actually my head is probably blocking it so let me just move myself up a bit it so let me just move myself up a bit it so let me just move myself up a bit so here we see this comment eviction so here we see this comment eviction so here we see this comment eviction blocked by too many requests usually a blocked by too many requests usually a blocked by too many requests usually a pdb so our AKs upgrade is now still pdb so our AKs upgrade is now still pdb so our AKs upgrade is now still ongoing right we are uh maybe five ongoing right we are uh maybe five ongoing right we are uh maybe five minutes in already but it is still in a minutes in already but it is still in a minutes in already but it is still in a upgrading running upgrading running upgrading running status and our pot disruption budget is status and our pot disruption budget is status and our pot disruption budget is doing its work like I'm doing if I do K doing its work like I'm doing if I do K doing its work like I'm doing if I do K get nodes now we still have this node uh there we now we still have this node uh there we have one ready and one is unable to have one ready and one is unable to have one ready and one is unable to drain and if I check my P disruption drain and if I check my P disruption drain and if I check my P disruption budget K pdb we still see our allowed budget K pdb we still see our allowed budget K pdb we still see our allowed disruption so how do we fix this we saw disruption so how do we fix this we saw disruption so how do we fix this we saw it has been it was available in the it has been it was available in the it has been it was available in the events so kubernetes is telling us that events so kubernetes is telling us that events so kubernetes is telling us that something is wrong K get something is wrong K get something is wrong K get events of all Nam events of all Nam events of all Nam spaces grab

spaces grab spaces grab pdb it is still saying eviction blocked pdb it is still saying eviction blocked pdb it is still saying eviction blocked by too many by too many by too many requests and how do we fix it well we requests and how do we fix it well we requests and how do we fix it well we can either remove the pot disruption can either remove the pot disruption can either remove the pot disruption budget or we can use it as it is budget or we can use it as it is budget or we can use it as it is intended and we can just increase the intended and we can just increase the intended and we can just increase the replicas of our deployment so our Target replicas of our deployment so our Target replicas of our deployment so our Target for the P disruption budget is the back for the P disruption budget is the back for the P disruption budget is the back end so if end so if end so if I I'm in my Misha namespace if I then K I I'm in my Misha namespace if I then K I I'm in my Misha namespace if I then K scale scale scale deploy and then I'm going to check get deploy and then I'm going to check get deploy and then I'm going to check get the HTTP the HTTP the HTTP backend and I'm going to say replicas is backend and I'm going to say replicas is backend and I'm going to say replicas is two now the deployment is two now the deployment is two now the deployment is scaled we see the container is scaled we see the container is scaled we see the container is creating and if I just do a k creating and if I just do a k creating and if I just do a k watch N1 and then Cube CTL get pods let's see ah we see that both have pods let's see ah we see that both have now been now been now been restarted and if I now get the notes restarted and if I now get the notes restarted and if I now get the notes maybe it is deleted already yep here we maybe it is deleted already yep here we maybe it is deleted already yep here we go the node is already go the node is already go the node is already deleted and it I've now demonstrated deleted and it I've now demonstrated deleted and it I've now demonstrated that the upgrade was still pending it that the upgrade was still pending it that the upgrade was still pending it was waiting for this node to be able to was waiting for this node to be able to was waiting for this node to be able to drain and at the moment when I scaled up drain and at the moment when I scaled up drain and at the moment when I scaled up the the the deployment our pod disruption budget was

deployment our pod disruption budget was deployment our pod disruption budget was fulfilled and now we see that the old fulfilled and now we see that the old fulfilled and now we see that the old node has been removed and all of my node has been removed and all of my node has been removed and all of my nodes are now at version nodes are now at version nodes are now at version 127 and if I do K get PD B now we see 127 and if I do K get PD B now we see 127 and if I do K get PD B now we see that because I've scaled up my that because I've scaled up my that because I've scaled up my deployment I have now one allowed deployment I have now one allowed deployment I have now one allowed disruption of the backend disruption of the backend disruption of the backend pdb pdb pdb right it works really well and this is right it works really well and this is right it works really well and this is just a a very small example of course just a a very small example of course just a a very small example of course this is just this is just this is just to show you the concepts but if we take to show you the concepts but if we take to show you the concepts but if we take a quick look at the documentation we see a quick look at the documentation we see a quick look at the documentation we see that we have a um Min available that we have a um Min available that we have a um Min available so you here you so you here you so you here you saying um evictions are allowed as long saying um evictions are allowed as long saying um evictions are allowed as long as they leave behind five or more as they leave behind five or more as they leave behind five or more healthy pods if you set a Min available healthy pods if you set a Min available healthy pods if you set a Min available of five right so um how that works of five right so um how that works of five right so um how that works is if I have 10 replicas and I say Min is if I have 10 replicas and I say Min is if I have 10 replicas and I say Min available then kubernetes is allowed to available then kubernetes is allowed to available then kubernetes is allowed to evict five of them at a time you also evict five of them at a time you also evict five of them at a time you also have a setting that's called Max have a setting that's called Max have a setting that's called Max unavailable so that means that if I have unavailable so that means that if I have unavailable so that means that if I have a deployment of 10 replicas and I say a a deployment of 10 replicas and I say a a deployment of 10 replicas and I say a Max unavailable of two then it's only Max unavailable of two then it's only Max unavailable of two then it's only going to reschedule two Pods at a time going to reschedule two Pods at a time going to reschedule two Pods at a time so you can get very granular into with so you can get very granular into with so you can get very granular into with this and you can also set it as

this and you can also set it as this and you can also set it as percentages not it doesn't matter if my percentages not it doesn't matter if my percentages not it doesn't matter if my application has 100 replicas or 10 or application has 100 replicas or 10 or application has 100 replicas or 10 or five if I say there is a Max unavailable five if I say there is a Max unavailable five if I say there is a Max unavailable of 20% say then you can always make be of 20% say then you can always make be of 20% say then you can always make be sure that only 20% of your application sure that only 20% of your application sure that only 20% of your application is down and this is where you can is down and this is where you can is down and this is where you can really harness the power of kubernetes really harness the power of kubernetes really harness the power of kubernetes because if you have all of your because if you have all of your because if you have all of your applications running with multiple applications running with multiple applications running with multiple replicas correctly configured with po replicas correctly configured with po replicas correctly configured with po disruption budgets then you can be sure disruption budgets then you can be sure disruption budgets then you can be sure that you can just do upgrades in the that you can just do upgrades in the that you can just do upgrades in the background and everything is going to be background and everything is going to be background and everything is going to be rescheduled without any downtime so rescheduled without any downtime so rescheduled without any downtime so there is still it is still saying it's there is still it is still saying it's there is still it is still saying it's upgrading running but from my experience upgrading running but from my experience upgrading running but from my experience I know that it takes a bit of time for I know that it takes a bit of time for I know that it takes a bit of time for it to fully complete and um because all it to fully complete and um because all it to fully complete and um because all of my nodes are um now on the new of my nodes are um now on the new of my nodes are um now on the new version I know that the the upgrade has version I know that the the upgrade has version I know that the the upgrade has been completed uh been completed uh been completed uh successfully and if you configure this successfully and if you configure this successfully and if you configure this properly for all of your applications properly for all of your applications properly for all of your applications you can just do automatic updates like you can just do automatic updates like you can just do automatic updates like here you can have automatic here you can have automatic here you can have automatic upgrade you can configure that that upgrade you can configure that that upgrade you can configure that that every stable version that you can enable every stable version that you can enable every stable version that you can enable it for every patch version and it can it for every patch version and it can it for every patch version and it can just auto update for all the patch just auto update for all the patch just auto update for all the patch versions you don't even have to do versions you don't even have to do versions you don't even have to do anything and if you correct if you

anything and if you correct if you anything and if you correct if you configure your cluster correctly if you configure your cluster correctly if you configure your cluster correctly if you have all of your replicas applications have all of your replicas applications have all of your replicas applications with multiple replicas you can have a with multiple replicas you can have a with multiple replicas you can have a auto updating highly available auto updating highly available auto updating highly available infrastructure uh for your all of your infrastructure uh for your all of your infrastructure uh for your all of your applications for all your teams to run applications for all your teams to run applications for all your teams to run the workloads on so pot disruption the workloads on so pot disruption the workloads on so pot disruption budgets are really powerful and I hope budgets are really powerful and I hope budgets are really powerful and I hope you're going to use them in your you're going to use them in your you're going to use them in your applications thank you so much for applications thank you so much for applications thank you so much for watching and I hope to see you in the watching and I hope to see you in the watching and I hope to see you in the next video