GitHub Copilot Update: It Didn't Age Well

A couple of weeks ago, I made a video A couple of weeks ago, I made a video about how I switched from cursor as my about how I switched from cursor as my about how I switched from cursor as my daily driver coding IDE into GitHub daily driver coding IDE into GitHub daily driver coding IDE into GitHub Copilot. And I have to say that video Copilot. And I have to say that video Copilot. And I have to say that video didn't really age well. So, I promised didn't really age well. So, I promised didn't really age well. So, I promised you an update when June comes around. you an update when June comes around. you an update when June comes around. And this is that update. So, here's the And this is that update. So, here's the And this is that update. So, here's the video I was talking about. And as I said video I was talking about. And as I said video I was talking about. And as I said in the pinned commenter, I actually in the pinned commenter, I actually in the pinned commenter, I actually recorded this video a couple of weeks recorded this video a couple of weeks recorded this video a couple of weeks before it was set to be published. And before it was set to be published. And before it was set to be published. And lucky for me, Microsoft or rather GitHub lucky for me, Microsoft or rather GitHub lucky for me, Microsoft or rather GitHub announced a very significant pricing announced a very significant pricing announced a very significant pricing change to GitHub Copilot. So, a lot of change to GitHub Copilot. So, a lot of change to GitHub Copilot. So, a lot of you asked what about the changes when you asked what about the changes when you asked what about the changes when June comes around. So, let's actually June comes around. So, let's actually June comes around. So, let's actually open GitHub Copilot and talk about this open GitHub Copilot and talk about this open GitHub Copilot and talk about this some more. So, here's what we are going some more. So, here's what we are going some more. So, here's what we are going to do. I'm going to take GitHub Copilot to do. I'm going to take GitHub Copilot to do. I'm going to take GitHub Copilot and put it to the test and I'll compare and put it to the test and I'll compare and put it to the test and I'll compare it to Cloud Code. So, actually before it to Cloud Code. So, actually before it to Cloud Code. So, actually before recording this video, I had used up my recording this video, I had used up my recording this video, I had used up my entire C-pilot allowance for the month entire C-pilot allowance for the month entire C-pilot allowance for the month of June. I believe I get about 7,000 of June. I believe I get about 7,000 of June. I believe I get about 7,000 credits and right now Copilot is on a credits and right now Copilot is on a credits and right now Copilot is on a credit based system where each operation credit based system where each operation credit based system where each operation consumes a given number of credits. consumes a given number of credits. consumes a given number of credits. Previously it was on a per request basis Previously it was on a per request basis Previously it was on a per request basis or something like that which allowed you or something like that which allowed you or something like that which allowed you to get insane usage with Copilot and was to get insane usage with Copilot and was to get insane usage with Copilot and was the sole reason I was using it for the sole reason I was using it for the sole reason I was using it for months because I could just get so much months because I could just get so much months because I could just get so much usage out of it in the monthly usage out of it in the monthly usage out of it in the monthly allowance. Now, come the month of June, allowance. Now, come the month of June, allowance. Now, come the month of June, this changes and I took it for a spin this changes and I took it for a spin this changes and I took it for a spin one afternoon and was able to spend 100% one afternoon and was able to spend 100% one afternoon and was able to spend 100% of my credits in a couple of hours of, of my credits in a couple of hours of, of my credits in a couple of hours of, let's say, fairly serious coding, what let's say, fairly serious coding, what let's say, fairly serious coding, what you would do in your typical day job.

you would do in your typical day job. you would do in your typical day job. Now, I was just planning to rant about Now, I was just planning to rant about Now, I was just planning to rant about this in this video and show you some this in this video and show you some this in this video and show you some stuff I was working with cloud code. stuff I was working with cloud code. stuff I was working with cloud code. However, when I open it today, I see However, when I open it today, I see However, when I open it today, I see that my credits have been reset. So, now that my credits have been reset. So, now that my credits have been reset. So, now we have a good opportunity to compare it we have a good opportunity to compare it we have a good opportunity to compare it to Cloud Code. So, here's what we're to Cloud Code. So, here's what we're to Cloud Code. So, here's what we're going to do. I'm going to give the same going to do. I'm going to give the same going to do. I'm going to give the same prompt to GitHub copilot and to cloud prompt to GitHub copilot and to cloud prompt to GitHub copilot and to cloud code. I'll be using the latest model code. I'll be using the latest model code. I'll be using the latest model from anropic which is cloud fable 5. from anropic which is cloud fable 5. from anropic which is cloud fable 5. It's their most capable model and It's their most capable model and It's their most capable model and released a couple of days ago. I'm going released a couple of days ago. I'm going released a couple of days ago. I'm going to set the effort to max just so that we to set the effort to max just so that we to set the effort to max just so that we can see how much tokens we can burn with can see how much tokens we can burn with can see how much tokens we can burn with copilot. So let me type out the prompt. copilot. So let me type out the prompt. copilot. So let me type out the prompt. And here's actually what I want to And here's actually what I want to And here's actually what I want to build. This is an application that uses build. This is an application that uses build. This is an application that uses Dapper the distributed application Dapper the distributed application Dapper the distributed application runtime. And Dapper has this very cool runtime. And Dapper has this very cool runtime. And Dapper has this very cool feature called workflows where you can feature called workflows where you can feature called workflows where you can create a workflow that consists of a set create a workflow that consists of a set create a workflow that consists of a set of activities. And here's what it looks of activities. And here's what it looks of activities. And here's what it looks like. Now, although it seems that these like. Now, although it seems that these like. Now, although it seems that these actually execute sequentially, Dapper actually execute sequentially, Dapper actually execute sequentially, Dapper actually turns these into durable actually turns these into durable actually turns these into durable functions. So these are going to execute functions. So these are going to execute functions. So these are going to execute asynchronously one by one and Dapper is asynchronously one by one and Dapper is asynchronously one by one and Dapper is going to manage this orchestration. I'll going to manage this orchestration. I'll going to manage this orchestration. I'll make a more detailed video about this at make a more detailed video about this at make a more detailed video about this at some point in the future probably, but some point in the future probably, but some point in the future probably, but this is going to be useful enough for this is going to be useful enough for this is going to be useful enough for our example. So in this workflow, we are our example. So in this workflow, we are our example. So in this workflow, we are processing the payment for the customer processing the payment for the customer processing the payment for the customer in step number two and reserving the in step number two and reserving the in step number two and reserving the stock in step number three. So what I stock in step number three. So what I stock in step number three. So what I want to do inside of our prompt is just want to do inside of our prompt is just want to do inside of our prompt is just implement some sort of compensating implement some sort of compensating implement some sort of compensating action when charging the customer action when charging the customer action when charging the customer completes, but updating the inventory completes, but updating the inventory completes, but updating the inventory fails for whatever reason. And here's fails for whatever reason. And here's fails for whatever reason. And here's the prompt. Really nothing special. I'm

the prompt. Really nothing special. I'm the prompt. Really nothing special. I'm just telling it, I want you to implement just telling it, I want you to implement just telling it, I want you to implement a compensating action in the order a compensating action in the order a compensating action in the order processing workflow when step number two processing workflow when step number two processing workflow when step number two completes. But step number three fails, completes. But step number three fails, completes. But step number three fails, we need to refund the orders payment. we need to refund the orders payment. we need to refund the orders payment. You should also write test cases for You should also write test cases for You should also write test cases for this so we make sure it's implemented this so we make sure it's implemented this so we make sure it's implemented properly. So I'm going to send this off properly. So I'm going to send this off properly. So I'm going to send this off to copilot on cloud fable 5. I'm using to copilot on cloud fable 5. I'm using to copilot on cloud fable 5. I'm using agent mode and I'm using autopilot so agent mode and I'm using autopilot so agent mode and I'm using autopilot so that it doesn't ask any questions and that it doesn't ask any questions and that it doesn't ask any questions and just implements everything end to end. just implements everything end to end. just implements everything end to end. So let's send this off. And while this So let's send this off. And while this So let's send this off. And while this is working, I'll switch to a different is working, I'll switch to a different is working, I'll switch to a different window. And here we're going to give the window. And here we're going to give the window. And here we're going to give the same prompt to cloud code inside of VS same prompt to cloud code inside of VS same prompt to cloud code inside of VS Code, which is how I've been using it Code, which is how I've been using it Code, which is how I've been using it for the most part. I don't know. I just for the most part. I don't know. I just for the most part. I don't know. I just like the VS Code UI a little bit better like the VS Code UI a little bit better like the VS Code UI a little bit better and I'm definitely not a CLI guy and the and I'm definitely not a CLI guy and the and I'm definitely not a CLI guy and the Cloud Code desktop app isn't just clunky Cloud Code desktop app isn't just clunky Cloud Code desktop app isn't just clunky if you ask me. So I'm setting this to if you ask me. So I'm setting this to if you ask me. So I'm setting this to use Fable Max effort. I'm using auto use Fable Max effort. I'm using auto use Fable Max effort. I'm using auto mode so everything completes to end. mode so everything completes to end. mode so everything completes to end. Send off the same prompt and just let Send off the same prompt and just let Send off the same prompt and just let them work and we're going to compare the them work and we're going to compare the them work and we're going to compare the changes once they are done. Now I do changes once they are done. Now I do changes once they are done. Now I do want to comment a bit on the usage that want to comment a bit on the usage that want to comment a bit on the usage that you can get with Cloud Code. So, I you can get with Cloud Code. So, I you can get with Cloud Code. So, I subscribed to their default $20 a month subscribed to their default $20 a month subscribed to their default $20 a month plan, which is very similar to Copilot, plan, which is very similar to Copilot, plan, which is very similar to Copilot, and you definitely get more usage out of and you definitely get more usage out of and you definitely get more usage out of the box. Now, this isn't a sponsored the box. Now, this isn't a sponsored the box. Now, this isn't a sponsored video of anything. I know somebody's video of anything. I know somebody's video of anything. I know somebody's going to write this in the comment. I'm going to write this in the comment. I'm going to write this in the comment. I'm just sharing my experience with these just sharing my experience with these just sharing my experience with these tools and what's working for me right tools and what's working for me right tools and what's working for me right now or not. I did contemplate going back now or not. I did contemplate going back now or not. I did contemplate going back to cursor, but that bad experience to cursor, but that bad experience to cursor, but that bad experience really turned me off. So, I said, why really turned me off. So, I said, why really turned me off. So, I said, why not take a look at cloud code again? And not take a look at cloud code again? And not take a look at cloud code again? And boy, was I surprised when I started boy, was I surprised when I started boy, was I surprised when I started using it. You could say the AI mind using it. You could say the AI mind using it. You could say the AI mind virus has got to me yet again. Let's

virus has got to me yet again. Let's virus has got to me yet again. Let's check up on Copilot and see what's going check up on Copilot and see what's going check up on Copilot and see what's going on. So, it's still working. Now, some on. So, it's still working. Now, some on. So, it's still working. Now, some commentary on Cloud Fable 5 while this commentary on Cloud Fable 5 while this commentary on Cloud Fable 5 while this is going on. First of all, it's is going on. First of all, it's is going on. First of all, it's incredibly capable. It's going to incredibly capable. It's going to incredibly capable. It's going to implement basically any feature that you implement basically any feature that you implement basically any feature that you throw at it end to end, write test throw at it end to end, write test throw at it end to end, write test cases. It's going to debug everything, cases. It's going to debug everything, cases. It's going to debug everything, run the app, and smoke test that run the app, and smoke test that run the app, and smoke test that everything works correctly. It can spin everything works correctly. It can spin everything works correctly. It can spin off sub agents to do some spikes if it off sub agents to do some spikes if it off sub agents to do some spikes if it gets stuck. Just an amazing model to gets stuck. Just an amazing model to gets stuck. Just an amazing model to work with. However, it's freaking work with. However, it's freaking work with. However, it's freaking expensive and you're probably going to expensive and you're probably going to expensive and you're probably going to burn through your $20 a month Claude burn through your $20 a month Claude burn through your $20 a month Claude Code plan in a couple of hours, which is Code plan in a couple of hours, which is Code plan in a couple of hours, which is why I upgraded to the $100 a month plan. why I upgraded to the $100 a month plan. why I upgraded to the $100 a month plan. And I was actually using it for a while And I was actually using it for a while And I was actually using it for a while until I figured out that I was probably until I figured out that I was probably until I figured out that I was probably also going to end up spending all of the also going to end up spending all of the also going to end up spending all of the limits there. So, I promptly updated to limits there. So, I promptly updated to limits there. So, I promptly updated to the $200 a month cloud code plan. And the $200 a month cloud code plan. And the $200 a month cloud code plan. And right now, I think I get enough usage right now, I think I get enough usage right now, I think I get enough usage out of it for what I'm currently working out of it for what I'm currently working out of it for what I'm currently working on. Now, if you don't believe me, here's on. Now, if you don't believe me, here's on. Now, if you don't believe me, here's my usage on the $200 a month plan. And my usage on the $200 a month plan. And my usage on the $200 a month plan. And Entropic actually did reset all of the Entropic actually did reset all of the Entropic actually did reset all of the limits after they released Fable 5. So, limits after they released Fable 5. So, limits after they released Fable 5. So, this is just my usage in the couple of this is just my usage in the couple of this is just my usage in the couple of days after Fable was released. I ended days after Fable was released. I ended days after Fable was released. I ended up using 58% of my weekly allowance, and up using 58% of my weekly allowance, and up using 58% of my weekly allowance, and I was really pushing it to the limits.

I was really pushing it to the limits. I was really pushing it to the limits. So, if we take a look at my last 7 days So, if we take a look at my last 7 days So, if we take a look at my last 7 days of usage, you can see I spent about 18 of usage, you can see I spent about 18 of usage, you can see I spent about 18 million tokens while working on a secret million tokens while working on a secret million tokens while working on a secret project that I'm going to reveal more project that I'm going to reveal more project that I'm going to reveal more about in some upcoming videos. But for about in some upcoming videos. But for about in some upcoming videos. But for now, I'm going to keep it a secret while now, I'm going to keep it a secret while now, I'm going to keep it a secret while I polish everything out. Most of this I polish everything out. Most of this I polish everything out. Most of this usage was on Fable 5, so about, let me usage was on Fable 5, so about, let me usage was on Fable 5, so about, let me see, 13 million tokens. And then a huge see, 13 million tokens. And then a huge see, 13 million tokens. And then a huge majority of that was like 2 days ago majority of that was like 2 days ago majority of that was like 2 days ago when I really ran it through the roof. when I really ran it through the roof. when I really ran it through the roof. So, before that, my average spend was So, before that, my average spend was So, before that, my average spend was around 3 million tokens a day. If it around 3 million tokens a day. If it around 3 million tokens a day. If it might be interesting and let me know in might be interesting and let me know in might be interesting and let me know in the comments, I might consider making a the comments, I might consider making a the comments, I might consider making a separate video on how you can burn so separate video on how you can burn so separate video on how you can burn so much tokens while not producing slop and much tokens while not producing slop and much tokens while not producing slop and making sure that your application still making sure that your application still making sure that your application still works as expected. Going back to GitHub works as expected. Going back to GitHub works as expected. Going back to GitHub Copilot in VS Code, we can see it made Copilot in VS Code, we can see it made Copilot in VS Code, we can see it made some changes and it looks like Copilot some changes and it looks like Copilot some changes and it looks like Copilot went on some tangent exploring something went on some tangent exploring something went on some tangent exploring something about the workflow context which is an about the workflow context which is an about the workflow context which is an internal data type to Dapper workflows. internal data type to Dapper workflows. internal data type to Dapper workflows. So, it's probably not all too relevant So, it's probably not all too relevant So, it's probably not all too relevant to what I was asking it to build. And in to what I was asking it to build. And in to what I was asking it to build. And in the meantime, cloud code seems to have the meantime, cloud code seems to have the meantime, cloud code seems to have done the work. So, let's take a look at done the work. So, let's take a look at done the work. So, let's take a look at what changes it made. So, inside of the what changes it made. So, inside of the what changes it made. So, inside of the order processing workflow, you can see order processing workflow, you can see order processing workflow, you can see it added another step here. So, when it added another step here. So, when it added another step here. So, when step number three fails and throws a step number three fails and throws a step number three fails and throws a workflow task failed exception, we workflow task failed exception, we workflow task failed exception, we execute a compensating action by calling execute a compensating action by calling execute a compensating action by calling another activity, the refund payment another activity, the refund payment another activity, the refund payment activity, which it created as another activity, which it created as another activity, which it created as another class. Now let me check if it also class. Now let me check if it also class. Now let me check if it also registered it in program CS. Yes, it registered it in program CS. Yes, it registered it in program CS. Yes, it did. So this checks out. The inventory did. So this checks out. The inventory did. So this checks out. The inventory service now fails for some product. It

service now fails for some product. It service now fails for some product. It seems to have added this for testing seems to have added this for testing seems to have added this for testing purposes. Then we've got some test cases purposes. Then we've got some test cases purposes. Then we've got some test cases here. And there's a test case for our here. And there's a test case for our here. And there's a test case for our compensating action. And it also added a compensating action. And it also added a compensating action. And it also added a bonus in the app host. So there's a bonus in the app host. So there's a bonus in the app host. So there's a command on the Aspire dashboard that command on the Aspire dashboard that command on the Aspire dashboard that allows me to place an order that will be allows me to place an order that will be allows me to place an order that will be refunded. So this would be a textbook refunded. So this would be a textbook refunded. So this would be a textbook implementation of this feature. It was implementation of this feature. It was implementation of this feature. It was definitely overkill for cloud fable. On definitely overkill for cloud fable. On definitely overkill for cloud fable. On the other side, co-pilot is still the other side, co-pilot is still the other side, co-pilot is still working, but it is making some progress working, but it is making some progress working, but it is making some progress and looks like it's going to be done and looks like it's going to be done and looks like it's going to be done pretty soon. So, it falls back to the pretty soon. So, it falls back to the pretty soon. So, it falls back to the same structure here calling the refund same structure here calling the refund same structure here calling the refund payment activity. Now, this is the part payment activity. Now, this is the part payment activity. Now, this is the part that really won me over with GitHub that really won me over with GitHub that really won me over with GitHub Copilot that you get to see the diff of Copilot that you get to see the diff of Copilot that you get to see the diff of what the agent generated once you gave what the agent generated once you gave what the agent generated once you gave it a prompt. So, look, it's going to it a prompt. So, look, it's going to it a prompt. So, look, it's going to converge to the same implementation and converge to the same implementation and converge to the same implementation and it makes sense because they're using the it makes sense because they're using the it makes sense because they're using the same model. Now, cloud code is same model. Now, cloud code is same model. Now, cloud code is noticeably faster. It also use a dummy noticeably faster. It also use a dummy noticeably faster. It also use a dummy product ID in the inventory service for product ID in the inventory service for product ID in the inventory service for testing purposes. This is probably going testing purposes. This is probably going testing purposes. This is probably going to wrap up sooner or later. I don't to wrap up sooner or later. I don't to wrap up sooner or later. I don't know. But nonetheless, I'm pretty know. But nonetheless, I'm pretty know. But nonetheless, I'm pretty confident that we're going to get the confident that we're going to get the confident that we're going to get the same solution that we did with cloud same solution that we did with cloud same solution that we did with cloud code, just slower. Now, the problem with code, just slower. Now, the problem with code, just slower. Now, the problem with Copilot is that this single prompt right Copilot is that this single prompt right Copilot is that this single prompt right now, which hasn't ran to its end, paid now, which hasn't ran to its end, paid now, which hasn't ran to its end, paid up 7% of my monthly allowance in Copilot up 7% of my monthly allowance in Copilot up 7% of my monthly allowance in Copilot credits. Now, you let me know what you credits. Now, you let me know what you credits. Now, you let me know what you think. Is this something that is think. Is this something that is think. Is this something that is sustainable and that you could keep sustainable and that you could keep sustainable and that you could keep using for an entire month? In my using for an entire month? In my using for an entire month? In my perspective, no. And you get a lot more perspective, no. And you get a lot more perspective, no. And you get a lot more value out of the baseline cloud code value out of the baseline cloud code value out of the baseline cloud code subscription. I'll leave it up to you to subscription. I'll leave it up to you to subscription. I'll leave it up to you to use whatever you like. I'm not really use whatever you like. I'm not really use whatever you like. I'm not really pushing one or the other. I'm currently pushing one or the other. I'm currently pushing one or the other. I'm currently using cloud code. I'm pretty happy with

using cloud code. I'm pretty happy with using cloud code. I'm pretty happy with it. My main reasoning for this was that it. My main reasoning for this was that it. My main reasoning for this was that I was already using anthropic models I was already using anthropic models I was already using anthropic models anyway. So, why not just go to the main anyway. So, why not just go to the main anyway. So, why not just go to the main provider as well as they will probably provider as well as they will probably provider as well as they will probably give you the best prices. Overall, we give you the best prices. Overall, we give you the best prices. Overall, we can definitely agree that the prices for can definitely agree that the prices for can definitely agree that the prices for these AI agents are heavily subsidized. these AI agents are heavily subsidized. these AI agents are heavily subsidized. And I'm expecting the prices to go up And I'm expecting the prices to go up And I'm expecting the prices to go up rather than go down, at least in the rather than go down, at least in the rather than go down, at least in the near future, unless compute and near future, unless compute and near future, unless compute and inference becomes cheaper, but it's not inference becomes cheaper, but it's not inference becomes cheaper, but it's not really looking like that's going to be really looking like that's going to be really looking like that's going to be the case. And I just have to say I'm a the case. And I just have to say I'm a the case. And I just have to say I'm a little disappointed that I can continue little disappointed that I can continue little disappointed that I can continue working with Copilot because I really working with Copilot because I really working with Copilot because I really enjoyed the developer experience and enjoyed the developer experience and enjoyed the developer experience and being able to view the diffs whenever I being able to view the diffs whenever I being able to view the diffs whenever I send a prompt to the agent. I'm not sure send a prompt to the agent. I'm not sure send a prompt to the agent. I'm not sure if this is something that's possible if this is something that's possible if this is something that's possible with cloud code. As far as I know right with cloud code. As far as I know right with cloud code. As far as I know right now, it isn't, but if anybody knows a now, it isn't, but if anybody knows a now, it isn't, but if anybody knows a solution for this, please let me know. I solution for this, please let me know. I solution for this, please let me know. I would love to have the same experience would love to have the same experience would love to have the same experience inside of Cloud Code. So, finally, inside of Cloud Code. So, finally, inside of Cloud Code. So, finally, Copilot completed the work. It used up Copilot completed the work. It used up Copilot completed the work. It used up about 600 credits. Now, in Copilot's about 600 credits. Now, in Copilot's about 600 credits. Now, in Copilot's defense, I was using Cloud Fable 5, defense, I was using Cloud Fable 5, defense, I was using Cloud Fable 5, which is incredibly expensive. I was which is incredibly expensive. I was which is incredibly expensive. I was also using it on max effort, which is also using it on max effort, which is also using it on max effort, which is absolutely overkill for what I asked you absolutely overkill for what I asked you absolutely overkill for what I asked you to do. But just giving you an example of to do. But just giving you an example of to do. But just giving you an example of what this usage might cost you. And on what this usage might cost you. And on what this usage might cost you. And on Pro Plus, this took 9% of my credits.

Pro Plus, this took 9% of my credits. Pro Plus, this took 9% of my credits. So, in my opinion, this is definitely So, in my opinion, this is definitely So, in my opinion, this is definitely not worth it. And sadly, Copilot isn't not worth it. And sadly, Copilot isn't not worth it. And sadly, Copilot isn't usable for me, unless I could somehow usable for me, unless I could somehow usable for me, unless I could somehow have a large number of credits to spend. have a large number of credits to spend. have a large number of credits to spend. That being said, I still think it's an That being said, I still think it's an That being said, I still think it's an amazing product. I do love using it, amazing product. I do love using it, amazing product. I do love using it, just the price aspect doesn't justify it just the price aspect doesn't justify it just the price aspect doesn't justify it for me. So, there's my June update for for me. So, there's my June update for for me. So, there's my June update for Copilot. Copilot out, lot in. I'm Copilot. Copilot out, lot in. I'm Copilot. Copilot out, lot in. I'm actually using $200 a month plan as I'm actually using $200 a month plan as I'm actually using $200 a month plan as I'm working on a new project which I hope to working on a new project which I hope to working on a new project which I hope to share more about in some upcoming video. share more about in some upcoming video. share more about in some upcoming video. Until then, thanks a lot for watching Until then, thanks a lot for watching Until then, thanks a lot for watching and stay awesome.