Setting up local development with Podman for my AI app & Local AI models

Figured I was going to do a little bit Figured I was going to do a little bit of more work on the app. of more work on the app. of more work on the app. And And And I was also going to answer a question I was also going to answer a question I was also going to answer a question here that is actually quite relevant for here that is actually quite relevant for here that is actually quite relevant for everyone about Vim and everyone about Vim and everyone about Vim and everything. everything. everything. Oh, cool. The chat keeps working if I Oh, cool. The chat keeps working if I Oh, cool. The chat keeps working if I just have it open like this. That's just have it open like this. That's just have it open like this. That's nice. Okay, so we have some people tuning in Okay, so we have some people tuning in already. That's good. already. That's good. already. That's good. I'll wait for a bit to I'll wait for a bit to I'll wait for a bit to for other people to come. Lives are cool idea. Yeah, right? Lives are cool idea. Yeah, right? So I realized with the previous one that So I realized with the previous one that So I realized with the previous one that my camera was a bit my camera was a bit my camera was a bit blocking the chat. So I'm trying to blocking the chat. So I'm trying to blocking the chat. So I'm trying to be a little bit better with that. be a little bit better with that. be a little bit better with that. So that So that So that you can actually see the chat on the you can actually see the chat on the you can actually see the chat on the video.

I figured I would do an hour or so I figured I would do an hour or so 1 and 1/2 hours of more work on the app that I'm building. that I'm building. And I'm actually going to try to see if And I'm actually going to try to see if And I'm actually going to try to see if I can record a video as I'm doing this. I can record a video as I'm doing this. I can record a video as I'm doing this. So now that we have Yeah, we have 12 So now that we have Yeah, we have 12 So now that we have Yeah, we have 12 people already. I'll just start the people already. I'll just start the people already. I'll just start the stream by stream by stream by trying to record a video. trying to record a video. trying to record a video. Start recording. Okay. Start recording. Okay. Start recording. Okay. So So So the question here is from William. This the question here is from William. This the question here is from William. This is in my is in my is in my CubeCraft. This is where I teach DevOps CubeCraft. This is where I teach DevOps CubeCraft. This is where I teach DevOps to engineers and I help people land to engineers and I help people land to engineers and I help people land DevOps jobs. DevOps jobs. DevOps jobs. We land about a job every single week. We land about a job every single week. We land about a job every single week. And this is where people get to ask me And this is where people get to ask me And this is where people get to ask me and the community questions directly and and the community questions directly and and the community questions directly and then I record private videos for them then I record private videos for them then I record private videos for them like this. like this. like this. Uh and here is a question by William. Uh and here is a question by William. Uh and here is a question by William. Will AI speech-to-text models render Vim Will AI speech-to-text models render Vim Will AI speech-to-text models render Vim obsolete? obsolete? obsolete? For the last 30 years, being a skilled For the last 30 years, being a skilled For the last 30 years, being a skilled Vim user will bring us some serious Vim user will bring us some serious Vim user will bring us some serious productivity gains.

productivity gains. productivity gains. But now as speech-to-text models But now as speech-to-text models But now as speech-to-text models are getting so good, why not just talk are getting so good, why not just talk are getting so good, why not just talk for almost everything you do on a for almost everything you do on a for almost everything you do on a computer? computer? computer? As such, do these models render Vim As such, do these models render Vim As such, do these models render Vim obsolete? If not obsolete, do they make obsolete? If not obsolete, do they make obsolete? If not obsolete, do they make Vim less useful than it once was? Vim less useful than it once was? Vim less useful than it once was? So So So there is already some discussion going there is already some discussion going there is already some discussion going on here. on here. on here. And this is actually something that I And this is actually something that I And this is actually something that I have been thinking about a lot recently. have been thinking about a lot recently. have been thinking about a lot recently. And the And the And the thing is that thing is that thing is that the the the the the the issue here is is for DevOps specifically issue here is is for DevOps specifically issue here is is for DevOps specifically is that is that is that the the main thing I'm thinking about is the the main thing I'm thinking about is the the main thing I'm thinking about is for first of all, debugging from the for first of all, debugging from the for first of all, debugging from the command line. command line. command line. So So So like when you're a There there there are like when you're a There there there are like when you're a There there there are environments that I've worked in that environments that I've worked in that environments that I've worked in that are still just running Linux virtual are still just running Linux virtual are still just running Linux virtual machines, Linux servers everywhere, machines, Linux servers everywhere, machines, Linux servers everywhere, right? right? right? So you need to be able to go in there So you need to be able to go in there So you need to be able to go in there and configure something if something and configure something if something and configure something if something goes wrong.

goes wrong. goes wrong. Like ideally you'll be doing this with Like ideally you'll be doing this with Like ideally you'll be doing this with Ansible, but like as you are creating Ansible, but like as you are creating Ansible, but like as you are creating the environments or something like that, the environments or something like that, the environments or something like that, there is still a situation where you there is still a situation where you there is still a situation where you could be on a command line with nothing could be on a command line with nothing could be on a command line with nothing but your Vim skills and the man pages to but your Vim skills and the man pages to but your Vim skills and the man pages to fix something. fix something. fix something. So I do still think that it is um So I do still think that it is um So I do still think that it is um necessary. necessary. necessary. And the the other one the other And the the other one the other And the the other one the other um um um what do you call it? Argument here what do you call it? Argument here what do you call it? Argument here is that like is that like is that like I I got to say that I almost never write I I got to say that I almost never write I I got to say that I almost never write code anymore, right? But what I do do is code anymore, right? But what I do do is code anymore, right? But what I do do is I I I for example, in this particular repo for example, in this particular repo for example, in this particular repo if I open up if I open up if I open up um um um like just the fact that I can just do VR like just the fact that I can just do VR like just the fact that I can just do VR and open up my recent um files like this and open up my recent um files like this and open up my recent um files like this to me is already something is very to me is already something is very to me is already something is very useful, right? So V dots and I can just useful, right? So V dots and I can just useful, right? So V dots and I can just open, I can check out my docs and I can open, I can check out my docs and I can open, I can check out my docs and I can easily navigate around here. Like that easily navigate around here. Like that easily navigate around here. Like that is not trivial. The fact that I can do is not trivial. The fact that I can do is not trivial. The fact that I can do this without the this without the this without the uh uh uh using the mouse and being able to like using the mouse and being able to like using the mouse and being able to like um go through the information, edit it um go through the information, edit it um go through the information, edit it very easily, nicely.

very easily, nicely. very easily, nicely. That is something that I've built up That is something that I've built up That is something that I've built up over the years and I do not I wouldn't over the years and I do not I wouldn't over the years and I do not I wouldn't like to lose this skill. like to lose this skill. like to lose this skill. However, what I I do not However, what I I do not However, what I I do not actually use this to write code anymore. actually use this to write code anymore. actually use this to write code anymore. I'm not writing code by hand at all. I'm not writing code by hand at all. I'm not writing code by hand at all. Like I write markdown files, I write all Like I write markdown files, I write all Like I write markdown files, I write all sorts of things in Vim. sorts of things in Vim. sorts of things in Vim. But very rarely do I actually write But very rarely do I actually write But very rarely do I actually write code. Like I can edit the code if I if I code. Like I can edit the code if I if I code. Like I can edit the code if I if I want to want to want to uh uh uh let's see. let's see. let's see. The the back end SRC The the back end SRC The the back end SRC right back end. If I want to check out right back end. If I want to check out right back end. If I want to check out and edit the code, and edit the code, and edit the code, then I would definitely navigate it with then I would definitely navigate it with then I would definitely navigate it with my Vim key bindings and I can go to the my Vim key bindings and I can go to the my Vim key bindings and I can go to the definition of the functions, etc. It's definition of the functions, etc. It's definition of the functions, etc. It's very um very um very um it's a good skill to have, but should it's a good skill to have, but should it's a good skill to have, but should you still learn it? you still learn it? you still learn it? Because the writing of it does not Because the writing of it does not Because the writing of it does not happen by um happen by um happen by um writing in files anymore. Like writing writing in files anymore. Like writing writing in files anymore. Like writing code hardly will be done by humans any code hardly will be done by humans any code hardly will be done by humans any longer. You still need to be able to longer. You still need to be able to longer. You still need to be able to read the code, but you can read the code, but you can read the code, but you can the the writing of it almost always the the writing of it almost always the the writing of it almost always happens by just dictating to the models.

happens by just dictating to the models. happens by just dictating to the models. And so I agree with that in for And so I agree with that in for And so I agree with that in for specifically for writing code. specifically for writing code. specifically for writing code. Now, the only reason why I'm still going Now, the only reason why I'm still going Now, the only reason why I'm still going to say to say to say yes, you still want to learn Vim is yes, you still want to learn Vim is yes, you still want to learn Vim is because because because if you want to become a DevOps engineer, if you want to become a DevOps engineer, if you want to become a DevOps engineer, then you're need to get if you would do then you're need to get if you would do then you're need to get if you would do it the right way, you're going to learn it the right way, you're going to learn it the right way, you're going to learn Kubernetes. And then you'll need to get Kubernetes. And then you'll need to get Kubernetes. And then you'll need to get the CKA, CKS, and CKAD exams. the CKA, CKS, and CKAD exams. the CKA, CKS, and CKAD exams. And as long as these exam environments And as long as these exam environments And as long as these exam environments require you to do hands-on work, which require you to do hands-on work, which require you to do hands-on work, which is time limited within 2 hours, is time limited within 2 hours, is time limited within 2 hours, if as long as they if as long as they if as long as they um require you to do that, um require you to do that, um require you to do that, then I believe that it's still useful then I believe that it's still useful then I believe that it's still useful and required to learn Vim because these and required to learn Vim because these and required to learn Vim because these exams are so freaking hard and you have exams are so freaking hard and you have exams are so freaking hard and you have to do it with high time pressure and any to do it with high time pressure and any to do it with high time pressure and any advantage you can get by doing the work advantage you can get by doing the work advantage you can get by doing the work faster and solving more questions during faster and solving more questions during faster and solving more questions during these exams these exams these exams are going is going to be an advantage are going is going to be an advantage are going is going to be an advantage that's going to help you in your career.

that's going to help you in your career. that's going to help you in your career. So as long as those exams still require So as long as those exams still require So as long as those exams still require you to do the hands-on work in these you to do the hands-on work in these you to do the hands-on work in these environments, I still think every DevOps environments, I still think every DevOps environments, I still think every DevOps engineer who is learning DevOps needs to engineer who is learning DevOps needs to engineer who is learning DevOps needs to spend time in Vim. And this is what we spend time in Vim. And this is what we spend time in Vim. And this is what we do in CubeCraft. Like one of the first do in CubeCraft. Like one of the first do in CubeCraft. Like one of the first things I will show you is my my Linux things I will show you is my my Linux things I will show you is my my Linux course where we're going to go deep into course where we're going to go deep into course where we're going to go deep into uh Linux on the command line. I'm going uh Linux on the command line. I'm going uh Linux on the command line. I'm going to teach you Vim, the basics, why you to teach you Vim, the basics, why you to teach you Vim, the basics, why you should do it, how to learn it, should do it, how to learn it, should do it, how to learn it, etc. Later I'm going to set you up with etc. Later I'm going to set you up with etc. Later I'm going to set you up with dot files and how to manage everything dot files and how to manage everything dot files and how to manage everything like I do. And I still think this is a like I do. And I still think this is a like I do. And I still think this is a good thing to do if you're going to be good thing to do if you're going to be good thing to do if you're going to be learn be becoming a DevOps engineer. learn be becoming a DevOps engineer. learn be becoming a DevOps engineer. But that I have to end with saying that But that I have to end with saying that But that I have to end with saying that I I I like this whole application that I like this whole application that I like this whole application that I built, CubeCraft AI, where I where you built, CubeCraft AI, where I where you built, CubeCraft AI, where I where you can query all of the knowledge which is can query all of the knowledge which is can query all of the knowledge which is looking running locally on a Kubernetes looking running locally on a Kubernetes looking running locally on a Kubernetes cluster cluster cluster uh that I own with my own GPU.

uh that I own with my own GPU. uh that I own with my own GPU. Like if I I can ask it questions, it's Like if I I can ask it questions, it's Like if I I can ask it questions, it's going to query going to query going to query 2 years of um coaching calls that I have 2 years of um coaching calls that I have 2 years of um coaching calls that I have done. done. done. I built all of this by dictating to the I built all of this by dictating to the I built all of this by dictating to the AI and I have not been using Vim to AI and I have not been using Vim to AI and I have not been using Vim to interact with it at all. interact with it at all. interact with it at all. So So So that is definitely the state where that is definitely the state where that is definitely the state where things are things are things are in that sense. So in that sense. So in that sense. So yeah, that is my current stance on it yeah, that is my current stance on it yeah, that is my current stance on it and with the world of AI, so much is and with the world of AI, so much is and with the world of AI, so much is changing. So this is what I'm currently changing. So this is what I'm currently changing. So this is what I'm currently This is my current opinion and yeah, This is my current opinion and yeah, This is my current opinion and yeah, we'll see where it is in the 6 months we'll see where it is in the 6 months we'll see where it is in the 6 months from now. from now. from now. Okay, so that's the That's the video recorded and That's the video recorded and let's see. That should have let's see. That should have let's see. That should have recorded to Is that to video? Videos?

Videos? No. Output? No. Output? No. Output? Where did that record to, OBS? Where did that record to, OBS? Where did that record to, OBS? Settings? Recording. Recording. Oh, this is just straight to my home Oh, this is just straight to my home Oh, this is just straight to my home directory. Okay. directory. Okay. directory. Okay. So Misha So Misha So Misha modified. Uh yeah, there we go. There it is. Uh yeah, there we go. There it is. Today. Today. Today. Cool. So that was recorded. Cool. So that was recorded. Cool. So that was recorded. Then I will upload that to Then I will upload that to Then I will upload that to Loom. Uh Uh upload. And this is how I help people in the And this is how I help people in the career accelerator. career accelerator. career accelerator. When people have questions, I will When people have questions, I will When people have questions, I will answer them by answer them by answer them by um of course by um of course by um of course by writing the posts, but sometimes or very writing the posts, but sometimes or very writing the posts, but sometimes or very often I will record custom videos like often I will record custom videos like often I will record custom videos like this.

this. this. And I'm not doing it on stream right And I'm not doing it on stream right And I'm not doing it on stream right now, so you can see now, so you can see now, so you can see um what it looks like inside of um what it looks like inside of um what it looks like inside of CubeCraft. But this is how I help people every But this is how I help people every single day. single day. single day. And this is why we're landing jobs And this is why we're landing jobs And this is why we're landing jobs once a week. once a week. once a week. We got a really cool job landing just We got a really cool job landing just We got a really cool job landing just now now now as well. as well. as well. Where Austin here Where Austin here Where Austin here now got an AI job at a Fortune 500 in now got an AI job at a Fortune 500 in now got an AI job at a Fortune 500 in the US, so that's really cool. the US, so that's really cool. the US, so that's really cool. They it was just landed this week. They it was just landed this week. They it was just landed this week. So, always happy to see that happening. The sooner you learn Vim, the more you The sooner you learn Vim, the more you profit. Yeah, I I I still think so. Hi profit. Yeah, I I I still think so. Hi profit. Yeah, I I I still think so. Hi Rob. Rob. Rob. Yeah, I I definitely think it is um Yeah, I I definitely think it is um Yeah, I I definitely think it is um still worth it. still worth it. still worth it. And yeah. All right, so how many people do we have All right, so how many people do we have live now?

live now? live now? 36, cool. 36, cool. 36, cool. Um Um Um now that we've recorded the video, I am now that we've recorded the video, I am now that we've recorded the video, I am going to continue with going to continue with going to continue with the following. Namely, in the previous the following. Namely, in the previous the following. Namely, in the previous stream, we were stream, we were stream, we were um I I set up the infrastructure so we um I I set up the infrastructure so we um I I set up the infrastructure so we could switch the models easier. could switch the models easier. could switch the models easier. And I have And I have And I have basically finished finished that. basically finished finished that. basically finished finished that. And then we also decided that we were And then we also decided that we were And then we also decided that we were going to going to going to um that we were going to stick to the that we were going to stick to the current Gemma deployment current Gemma deployment current Gemma deployment for now. for now. for now. But because on the Kubernetes cluster, But because on the Kubernetes cluster, But because on the Kubernetes cluster, I am limited to only one I am limited to only one I am limited to only one um um um GPU. GPU. GPU. So, if I open up the cluster here, No.

No. If I open up the cluster, technically I don't need K9s just to technically I don't need K9s just to show you the nodes, but why not nodes? show you the nodes, but why not nodes? show you the nodes, but why not nodes? So, my current cluster, it is built up So, my current cluster, it is built up So, my current cluster, it is built up it is running locally. it is running locally. it is running locally. And my GPU worker is actually located in And my GPU worker is actually located in And my GPU worker is actually located in my office. my office. my office. But the control plane is actually a But the control plane is actually a But the control plane is actually a Hetzner VM. Hetzner VM. Hetzner VM. So, I only have two nodes, and one of So, I only have two nodes, and one of So, I only have two nodes, and one of them has a GPU. So, currently I'm only them has a GPU. So, currently I'm only them has a GPU. So, currently I'm only able to deploy one model at a time able to deploy one model at a time able to deploy one model at a time because the way KServe works needs um because the way KServe works needs um because the way KServe works needs um means that it can only means that it can only means that it can only um um um run one pod serving one model at a time. run one pod serving one model at a time. run one pod serving one model at a time. So, I'm a bit limited to that. So, I'm a bit limited to that. So, I'm a bit limited to that. And if I want to try out And if I want to try out And if I want to try out locally, if I want to try out my locally, if I want to try out my locally, if I want to try out my application with a different model, cuz application with a different model, cuz application with a different model, cuz I really want to check out Qwen 27B for I really want to check out Qwen 27B for I really want to check out Qwen 27B for example.

And And or or 35B for that matter. I just wanted or or 35B for that matter. I just wanted or or 35B for that matter. I just wanted to to to to yeah, check out how it runs. to yeah, check out how it runs. to yeah, check out how it runs. How my app runs with that application How my app runs with that application How my app runs with that application with that um model. Now, I do have a Framework model. Now, I do have a Framework desktop which has a lot of shared which has a lot of shared uh VRAM. Actually, uh VRAM. Actually, uh VRAM. Actually, I have not been optimizing my desktop I have not been optimizing my desktop I have not been optimizing my desktop for um for um for um running AI yet. running AI yet. running AI yet. But I do have this one. I have the 128 But I do have this one. I have the 128 But I do have this one. I have the 128 GB GB GB setup. So, actually what I should do is setup. So, actually what I should do is setup. So, actually what I should do is I should just set up I should just set up I should just set up the application so that I can just run the application so that I can just run the application so that I can just run Qwen Qwen Qwen 35B locally. 35B locally. 35B locally. Run it, and then Run it, and then Run it, and then point the application to that. point the application to that. point the application to that. So, So, So, that is going to be fun. that is going to be fun. that is going to be fun. And I think that's what I'm going to do And I think that's what I'm going to do And I think that's what I'm going to do now because I have maybe 1 1/2 hours.

now because I have maybe 1 1/2 hours. now because I have maybe 1 1/2 hours. Yeah, about about an hour, and I think I Yeah, about about an hour, and I think I Yeah, about about an hour, and I think I would be able to get that up and running would be able to get that up and running would be able to get that up and running in an hour, and it will be fun to be in an hour, and it will be fun to be in an hour, and it will be fun to be able to actually run it locally able to actually run it locally able to actually run it locally with a different model. with a different model. with a different model. Okay. Okay. Okay. So, this can be cleared. So, this can be cleared. So, this can be cleared. And And And I just got to check something if Yes, this is running on the host Yes, this is running on the host network. So, network. So, network. So, my dev container should be able to run my dev container should be able to run my dev container should be able to run or or reach the AI model. And And yeah, I'm basically going to just start yeah, I'm basically going to just start yeah, I'm basically going to just start working on that. So, here case in point, working on that. So, here case in point, working on that. So, here case in point, right? We started the stream by right? We started the stream by right? We started the stream by talking about talking about talking about Vim and whether Vim and whether Vim and whether you should still learn Vim.

you should still learn Vim. you should still learn Vim. Oh, is my camera not working? Oh, is my camera not working? Oh, is my camera not working? What the heck? How long has my camera been frozen, How long has my camera been frozen, guys? The camera Oh, wait, here it is. The camera Oh, wait, here it is. Is it back again? I think it's back again. Can you please I think it's back again. Can you please confirm if you see me moving again? confirm if you see me moving again? confirm if you see me moving again? >> [laughter] >> Hey Casper from NL.

>> Hey Casper from NL. Netherlands. It is back. Thank you so Netherlands. It is back. Thank you so Netherlands. It is back. Thank you so much for confirming that. Appreciate it. Oh, you live in Taiwan. Oh, your NL, Oh, you live in Taiwan. Oh, your NL, that made me think that you are from the that made me think that you are from the that made me think that you are from the Netherlands. Thanks, guys. Appreciate you confirming Thanks, guys. Appreciate you confirming that. that. that. Um Um Um So, yes, we were talking We began the So, yes, we were talking We began the So, yes, we were talking We began the stream by talking about should you learn stream by talking about should you learn stream by talking about should you learn Vim? And well, the case in point here is Vim? And well, the case in point here is Vim? And well, the case in point here is that what I'm going to show you, that what I'm going to show you, that what I'm going to show you, we're going to do brainstorming. we're going to do brainstorming. we're going to do brainstorming. And And And um um um I will now then dictate to the AI what I I will now then dictate to the AI what I I will now then dictate to the AI what I want to do. want to do. want to do. And I I've already created an issue for And I I've already created an issue for And I I've already created an issue for this one. this one. this one. Local AI dev stack. Okay? Local AI dev stack. Okay? Local AI dev stack. Okay? So, here I'll say, I want to work on issue 140.

I want to work on issue 140. And And And the the the the setup is as follows. Like you are the setup is as follows. Like you are the setup is as follows. Like you are currently running inside of a dev currently running inside of a dev currently running inside of a dev container. container. container. I always do everything inside of dev I always do everything inside of dev I always do everything inside of dev containers to scope everything nicely containers to scope everything nicely containers to scope everything nicely and to make sure that my AI agents can and to make sure that my AI agents can and to make sure that my AI agents can run with full permissions. run with full permissions. run with full permissions. Now, for the current project, I want to Now, for the current project, I want to Now, for the current project, I want to have a stack that I can run locally in have a stack that I can run locally in have a stack that I can run locally in Podman. Podman. Podman. And I am using a Fedora Atomic desktop And I am using a Fedora Atomic desktop And I am using a Fedora Atomic desktop spin. spin. spin. Sp- the Sway one specifically. Sp- the Sway one specifically. Sp- the Sway one specifically. So, I am going to be using um a So, I am going to be using um a So, I am going to be using um a containerized workflow, and I'm also containerized workflow, and I'm also containerized workflow, and I'm also going to be running my going to be running my going to be running my AI model AI model AI model uh inside of a toolbox. uh inside of a toolbox. uh inside of a toolbox. I will share the link, and I want you to I will share the link, and I want you to I will share the link, and I want you to clone that repo so that you understand clone that repo so that you understand clone that repo so that you understand how these toolboxes are deployed on my how these toolboxes are deployed on my how these toolboxes are deployed on my Fedora machine.

Fedora machine. Fedora machine. But But But this is how I will this is how I will this is how I will um um um be making the model available. be making the model available. be making the model available. So, right now I'm thinking that we are So, right now I'm thinking that we are So, right now I'm thinking that we are going to spin up the application outside going to spin up the application outside going to spin up the application outside of the dev container, and I will just of the dev container, and I will just of the dev container, and I will just run Podman or Podman compose run Podman or Podman compose run Podman or Podman compose uh locally. uh locally. uh locally. And then the idea is that it will And then the idea is that it will And then the idea is that it will deploy if the front end and back end deploy if the front end and back end deploy if the front end and back end containers, which are then configured containers, which are then configured containers, which are then configured through environment variables to talk to through environment variables to talk to through environment variables to talk to the model that's running locally in a the model that's running locally in a the model that's running locally in a separate toolbox. separate toolbox. separate toolbox. So, this is um So, this is um So, this is um what I have in mind. what I have in mind. what I have in mind. I have no experience with Podman I have no experience with Podman I have no experience with Podman compose, only Docker compose. compose, only Docker compose. compose, only Docker compose. So, So, So, my system is running Podman in rootless my system is running Podman in rootless my system is running Podman in rootless mode. We are not running rootful, but mode. We are not running rootful, but mode. We are not running rootful, but rootless. So, we have to take that into rootless. So, we have to take that into rootless. So, we have to take that into account when building this feature.

All right, so that is how I would then All right, so that is how I would then scope out a prompt like this. And And this is no typing involved. this is no typing involved. this is no typing involved. And I I believe that like talking out And I I believe that like talking out And I I believe that like talking out loud like this is has a very different loud like this is has a very different loud like this is has a very different uh oops, I have to uh oops, I have to uh oops, I have to actually actually actually um share the um share the um share the toolbox link uh toolbox toolbox link uh toolbox toolbox link uh toolbox toolbox images. This is the Yeah, so I'm I'm like when you're Yeah, so I'm I'm like when you're talking to a um AI like this in natural language, I AI like this in natural language, I think the thinking process is also think the thinking process is also think the thinking process is also different. Like right now I'm like different. Like right now I'm like different. Like right now I'm like standing here walking, I'm able to talk. standing here walking, I'm able to talk. standing here walking, I'm able to talk. Like I think this this Like I think this this Like I think this this it it definitely helps with the the it it definitely helps with the the it it definitely helps with the the actual thoughts that you create, the actual thoughts that you create, the actual thoughts that you create, the creative thinking that you can do, creative thinking that you can do, creative thinking that you can do, and yeah, the way how deep you think.

and yeah, the way how deep you think. and yeah, the way how deep you think. So, what's going to be really cool is So, what's going to be really cool is So, what's going to be really cool is that that that in a week from now, I'm actually going in a week from now, I'm actually going in a week from now, I'm actually going to take my camper van to take my camper van to take my camper van and I'm going to go on a trip for 2 and I'm going to go on a trip for 2 and I'm going to go on a trip for 2 weeks and I'm going to be weeks and I'm going to be weeks and I'm going to be staying at campsites. I'm going to be staying at campsites. I'm going to be staying at campsites. I'm going to be driving through Germany and France. driving through Germany and France. driving through Germany and France. Not much traveling, but like finding a Not much traveling, but like finding a Not much traveling, but like finding a nice site and then staying there for 7 nice site and then staying there for 7 nice site and then staying there for 7 days or something. days or something. days or something. And I'm going to try out to like do like And I'm going to try out to like do like And I'm going to try out to like do like agentic engineering like I'm doing now, agentic engineering like I'm doing now, agentic engineering like I'm doing now, but then like but then like but then like in a nice environment sitting in in the in a nice environment sitting in in the in a nice environment sitting in in the sun sun sun or while on a mountain hike even and or while on a mountain hike even and or while on a mountain hike even and prompting them from my phone. prompting them from my phone. prompting them from my phone. And the idea is that I'm going to be And the idea is that I'm going to be And the idea is that I'm going to be streaming a lot of that as well. streaming a lot of that as well. streaming a lot of that as well. So, I So, I So, I I want to do a lot more live streaming I want to do a lot more live streaming I want to do a lot more live streaming um um um in the future. in the future. in the future. I'm My plan is to start streaming like I'm My plan is to start streaming like I'm My plan is to start streaming like almost every day while I'm building this almost every day while I'm building this almost every day while I'm building this application application application and yeah, showcasing my my AI skills and and yeah, showcasing my my AI skills and and yeah, showcasing my my AI skills and the things I'm learning, the things I'm the things I'm learning, the things I'm the things I'm learning, the things I'm building. And I want to take you guys building. And I want to take you guys building. And I want to take you guys along with that.

along with that. along with that. And show you how I do this basically. The issue text assumes in media CDI. The issue text assumes in media CDI. Yeah. So, the issues VLM recommendation So, the issues VLM recommendation doesn't apply. The model server will be doesn't apply. The model server will be doesn't apply. The model server will be llama server running inside a toolbox. llama server running inside a toolbox. llama server running inside a toolbox. Um yes. Um yes. Um yes. That is actually a good point. Rag back end today needs two open AI Rag back end today needs two open AI compatible end points for LM. compatible end points for LM. compatible end points for LM. Now we can embed. Now we can embed. Now we can embed. For the local stack, how do you want to For the local stack, how do you want to For the local stack, how do you want to provide both?

So, first of all, So, first of all, yes, it will be on a yes, it will be on a yes, it will be on a on the framework desktop Um Um I won't be doing I won't be doing I won't be doing any embeddings uh right now. uh right now. So, we can leave that out of scope So, we can leave that out of scope So, we can leave that out of scope for now. for now. for now. But do create an issue to But do create an issue to But do create an issue to uh consider this uh consider this uh consider this for the local stack for the local stack for the local stack in the future. Now I'm just about in the future. Now I'm just about in the future. Now I'm just about uh uh uh testing inference with a different testing inference with a different testing inference with a different model. model. model. And actually I'm not noticing that I'm And actually I'm not noticing that I'm And actually I'm not noticing that I'm not I'm typing right now and I'm not not I'm typing right now and I'm not not I'm typing right now and I'm not text to speech. text to speech. text to speech. Uh skip retrieval locally point rag back Uh skip retrieval locally point rag back Uh skip retrieval locally point rag back end only at LM gate embeddings RPC and end only at LM gate embeddings RPC and end only at LM gate embeddings RPC and retrieval disabled true no embedding retrieval disabled true no embedding retrieval disabled true no embedding model needed.

Yes, so it's basically Yes, so it's basically option C. Yes, so it's basically option C. Um we don't need embeddings. I I'm Um we don't need embeddings. I I'm Um we don't need embeddings. I I'm literally just The only thing I'm literally just The only thing I'm literally just The only thing I'm interested in right now is to test my interested in right now is to test my interested in right now is to test my front and back end front and back end front and back end while they talk to a different model while they talk to a different model while they talk to a different model because it's problematic with just one because it's problematic with just one because it's problematic with just one GPU GPU GPU to switch. So, I want to have a place to switch. So, I want to have a place to switch. So, I want to have a place where I can experiment with these models where I can experiment with these models where I can experiment with these models and have it run locally. And because of this, I don't think we And because of this, I don't think we need to need to need to um adhere to VLLM for in that particular um adhere to VLLM for in that particular um adhere to VLLM for in that particular case. case. case. Everything on the Kubernetes cluster Everything on the Kubernetes cluster Everything on the Kubernetes cluster should be using VLLM, should be using VLLM, should be using VLLM, but locally it's fine to use llama.cpp but locally it's fine to use llama.cpp but locally it's fine to use llama.cpp because in the end it's just an open AI because in the end it's just an open AI because in the end it's just an open AI end point that we need, right?

Wow, it is thinking a long time. There's Wow, it is thinking a long time. There's probably a lot of people probably a lot of people probably a lot of people playing with um Claude right now. Usually when things are very slow, it Usually when things are very slow, it will um like X will be all over it. like X will be all over it. Does anyone actually run 20 agents in Does anyone actually run 20 agents in Does anyone actually run 20 agents in parallel? I I love that this is going on parallel? I I love that this is going on parallel? I I love that this is going on actually. actually. actually. I run them perpendicular. Yes, four back I run them perpendicular. Yes, four back I run them perpendicular. Yes, four back end projects, each one codex. It has end projects, each one codex. It has end projects, each one codex. It has three sub agents. three sub agents. three sub agents. Three UI explorations, one LM studio for Three UI explorations, one LM studio for Three UI explorations, one LM studio for summaries. Okay. Okay. But they're mostly idle, yeah. But they're mostly idle, yeah. But they're mostly idle, yeah. >> [laughter] But this is the thing. Like you There But this is the thing. Like you There are so many people saying or claiming are so many people saying or claiming are so many people saying or claiming that they're running these AI storms and that they're running these AI storms and that they're running these AI storms and AI agent swarms.

AI agent swarms. AI agent swarms. But when you look at it, like most But when you look at it, like most But when you look at it, like most people are don't. And actually I have people are don't. And actually I have people are don't. And actually I have found myself that found myself that found myself that at most it is basically I'm able to get at most it is basically I'm able to get at most it is basically I'm able to get keep one or two running like constantly. keep one or two running like constantly. keep one or two running like constantly. But at that point the cognitive overload But at that point the cognitive overload But at that point the cognitive overload becomes too much. Let's see. Build mode for chat UI and Let's see. Build mode for chat UI and rag back end when podman compose up rag back end when podman compose up rag back end when podman compose up runs. Where do the two app containers runs. Where do the two app containers runs. Where do the two app containers come from? come from? come from? Uh Uh Uh built from source. Recommendation is A, build pre-built. Recommendation is A, build pre-built. No, I want to I want B. This is such a cool project. Please keep This is such a cool project. Please keep streaming every day and maybe experiment streaming every day and maybe experiment streaming every day and maybe experiment with other local agents.

with other local agents. with other local agents. Yes. Yes. Yes. Well, I'm glad you're saying that. Well, I'm glad you're saying that. Well, I'm glad you're saying that. This is actually actually what I'm This is actually actually what I'm This is actually actually what I'm planning to do. planning to do. planning to do. Um Um Um I want to show you this process of me I want to show you this process of me I want to show you this process of me building this app and this is going to building this app and this is going to building this app and this is going to be my first app be my first app be my first app or the first real one that I'm having or the first real one that I'm having or the first real one that I'm having like client I now have 30 users using like client I now have 30 users using like client I now have 30 users using the application right now. the application right now. the application right now. And I'm going to And I'm going to And I'm going to uh keep building more of these as I get uh keep building more of these as I get uh keep building more of these as I get better, too. But I'm now focusing on better, too. But I'm now focusing on better, too. But I'm now focusing on getting this one right first. getting this one right first. getting this one right first. Um repo root compose.yaml Um repo root compose.yaml Um repo root compose.yaml local zero to zero to yeah. Local AI building K3S and build store Local AI building K3S and build store and deploy images. Yeah, cool man. Nice and deploy images. Yeah, cool man. Nice and deploy images. Yeah, cool man. Nice work.

work. work. How do containers reach the host running How do containers reach the host running How do containers reach the host running llama server? This is where the rootless llama server? This is where the rootless llama server? This is where the rootless podman has a few quirks. podman has a few quirks. podman has a few quirks. Um Um podman has built-in DNS for the host Um podman has built-in DNS for the host gateway with pasta. gateway with pasta. gateway with pasta. This resolves automatically inside This resolves automatically inside This resolves automatically inside containers. No compose side gymnastics. containers. No compose side gymnastics. containers. No compose side gymnastics. Okay, I like that better actually. Yeah. Okay, I like that better actually. Yeah. Okay, I like that better actually. Yeah. Aim. Okay, retrieval of mechanism is in rag Okay, retrieval of mechanism is in rag backend.

backend. backend. New field, retrieval disabled New field, retrieval disabled New field, retrieval disabled in settings. in settings. in settings. Make Superbase URL optional. Make Superbase URL optional. Make Superbase URL optional. Um Well, I actually want the local stuff to Well, I actually want the local stuff to be able to talk to Superbase. be able to talk to Superbase. be able to talk to Superbase. And actually, can you please remind me And actually, can you please remind me And actually, can you please remind me do we have do we have do we have uh a separate Superbase staging instance uh a separate Superbase staging instance uh a separate Superbase staging instance or are we still talking to the same one? In any case, I want to use the same In any case, I want to use the same thing that the current staging cloud thing that the current staging cloud thing that the current staging cloud environment that's running on the environment that's running on the environment that's running on the cluster is using. Sway is like i3 for Wayland. Yes, that's Sway is like i3 for Wayland. Yes, that's exactly what it is. exactly what it is. exactly what it is. And yes, I am on a treadmill. God wears And yes, I am on a treadmill. God wears And yes, I am on a treadmill. God wears Gucci.

Actually, I did a 2-hour stream earlier Actually, I did a 2-hour stream earlier today today today where I was where I was where I was um Yeah, I was walking constantly. In the Yeah, I was walking constantly. In the morning, I've been working for a couple morning, I've been working for a couple morning, I've been working for a couple of hours while walking. of hours while walking. of hours while walking. And right now I'm I will also be walking And right now I'm I will also be walking And right now I'm I will also be walking for maybe 1 and 1/2 hours. So, that's for maybe 1 and 1/2 hours. So, that's for maybe 1 and 1/2 hours. So, that's like a solid 5 hours of walking I'm like a solid 5 hours of walking I'm like a solid 5 hours of walking I'm doing doing doing um while working, which is great. um while working, which is great. um while working, which is great. Staging is a separate Superbase project. Staging is a separate Superbase project. Staging is a separate Superbase project. It's my staging. It's my staging. It's my staging. Okay. Magic link, yes. Rag backend Superbase Magic link, yes. Rag backend Superbase PG vectors. PG vectors. PG vectors. Reading the same project vectors tables. Reading the same project vectors tables. Reading the same project vectors tables. Use Superbase URL Superbase key. Use Superbase URL Superbase key. Use Superbase URL Superbase key. Um To enable retrieval, you also need the To enable retrieval, you also need the embedding model to run locally. Ah, so the staging backend is talking to Ah, so the staging backend is talking to the the production Superbase then, the the production Superbase then, the the production Superbase then, right?

right? right? The staging The staging The staging Superbase is only for authentication. The full stack is uh Python The full stack is uh Python for the backend. for the backend. for the backend. Frontend is Next GS. Yes.

Okay, so Okay, so if I open Superbase then Why is my staging unhealthy? Why is my staging unhealthy? Um database Um database Um database schema schema schema message But But so we didn't duplicate all of the data so we didn't duplicate all of the data so we didn't duplicate all of the data to the staging Superbase then. I'm to the staging Superbase then. I'm to the staging Superbase then. I'm looking at it right now, but does the looking at it right now, but does the looking at it right now, but does the staging have staging have staging have uh any of the Q&A calls or YouTube uh any of the Q&A calls or YouTube uh any of the Q&A calls or YouTube transcripts ready in it? I I don't think transcripts ready in it? I I don't think transcripts ready in it? I I don't think so actually.

transcripts transcripts QA transcript chunks. Yeah. Well, you actually have all of the Well, you actually have all of the credentials to talk to Superbase. And credentials to talk to Superbase. And credentials to talk to Superbase. And I think we will need to migrate I think we will need to migrate I think we will need to migrate um the transcripts and the Q&A and all of the transcripts and the Q&A and all of the embeddings etc. the embeddings etc. the embeddings etc. We have to migrate all of that to We have to migrate all of that to We have to migrate all of that to staging the staging Superbase staging the staging Superbase staging the staging Superbase so that we have a data set to work with. so that we have a data set to work with. so that we have a data set to work with. And we can just query the staging And we can just query the staging And we can just query the staging database locally. Before jumping into building apps and Before jumping into building apps and SaaS with AI MCPs, do you recommend SaaS with AI MCPs, do you recommend SaaS with AI MCPs, do you recommend learning the languages manually for how learning the languages manually for how learning the languages manually for how long?

long? long? Yeah. Yeah. Yeah. I I I I definitely recommend learning the I definitely recommend learning the I definitely recommend learning the fundamentals of programming for sure. fundamentals of programming for sure. fundamentals of programming for sure. How long? This is difficult to to answer, but I This is difficult to to answer, but I wouldn't advise doing what I'm doing wouldn't advise doing what I'm doing wouldn't advise doing what I'm doing here if you don't know like programming here if you don't know like programming here if you don't know like programming fundamentals and if you haven't built fundamentals and if you haven't built fundamentals and if you haven't built some projects yourself. Well, actually you're wrong here because Well, actually you're wrong here because um um um there's no point in I cannot test the there's no point in I cannot test the there's no point in I cannot test the inference locally if we're not able to inference locally if we're not able to inference locally if we're not able to retrieve anything. retrieve anything. retrieve anything. So, both need to be done at this They So, both need to be done at this They So, both need to be done at this They both need to be done for the local both need to be done for the local both need to be done for the local development setup to work.

I'll be fine doing both of them I'll be fine doing both of them in this session. in this session. in this session. It's not that big of a deal It's not that big of a deal It's not that big of a deal to port something over to staging.

Uh, Uh, let's see. But why why do we need embedding? But why why do we need embedding? All of the embeddings All of the embeddings All of the embeddings are still are still are still in are in super base. in are in super base. in are in super base. Prod, we don't need to create new Prod, we don't need to create new Prod, we don't need to create new embeddings locally embeddings locally embeddings locally at this stage. at this stage. at this stage. We don't need the embedding We don't need the embedding We don't need the embedding model model model for retrieval, for retrieval, for retrieval, do we? Like maybe we do. I I Like maybe we do. I I but I'm pretty sure that we are not but I'm pretty sure that we are not but I'm pretty sure that we are not >> [snorts] >> [snorts] >> [snorts] >> k n l m demo >> k n l m demo >> k n l m demo k get pods k get pods k get pods I'm pretty sure we're not running an I'm pretty sure we're not running an I'm pretty sure we're not running an embeddings model right now.

Oh, we are running Oh, that's Oh, we are running Oh, that's interesting. interesting. interesting. I am running the nomic embeds running I am running the nomic embeds running I am running the nomic embeds running model. Retrieval needs to query the embedded Retrieval needs to query the embedded too on every single request. too on every single request. too on every single request. Interesting. Interesting. Interesting. Okay. Local [clears throat] llama server Local [clears throat] llama server toolbox. We could point it to the server, that's We could point it to the server, that's true. true. true. Pragmatic shortcut that requires case of Pragmatic shortcut that requires case of Pragmatic shortcut that requires case of API key in your local env. Yes, okay. Well, then then Yes, okay. Well, then then um then I would say then I would say we will run it in two toolboxes.

we will run it in two toolboxes. we will run it in two toolboxes. In that case In that case In that case a separate will run will run Let's see. Router mode. One toolbox, one Let's see. Router mode. One toolbox, one Let's see. Router mode. One toolbox, one process serving both the LLM and a nomic process serving both the LLM and a nomic process serving both the LLM and a nomic embed embed embed routed by the model field. Cleanest routed by the model field. Cleanest routed by the model field. Cleanest Okay. Okay. Okay. Then we'll do this. I've started learning DevOps 1 year ago I've started learning DevOps 1 year ago and I finished two courses, but I can't and I finished two courses, but I can't and I finished two courses, but I can't start to work because I feel I don't start to work because I feel I don't start to work because I feel I don't know enough. Can you give some advice? know enough. Can you give some advice? know enough. Can you give some advice? Um yes. So Um yes. So Um yes. So if you want to land a DevOps job if you want to land a DevOps job if you want to land a DevOps job then you need to invest in some proper then you need to invest in some proper then you need to invest in some proper mentoring and get get someone to help mentoring and get get someone to help mentoring and get get someone to help you. Now, this is what I do for a you. Now, this is what I do for a you. Now, this is what I do for a living. living. living. I can you There is Kubecraft. This is I can you There is Kubecraft. This is I can you There is Kubecraft. This is where people join where people join where people join and they and they and they follow an 80-hour course. You build a follow an 80-hour course. You build a follow an 80-hour course. You build a production grade portfolio production grade portfolio production grade portfolio all guided by a senior DevOps engineer, all guided by a senior DevOps engineer, all guided by a senior DevOps engineer, me. I'm ask I'm answering questions here me. I'm ask I'm answering questions here me. I'm ask I'm answering questions here day in day out.

day in day out. day in day out. And um And um And um if you want to land a DevOps job, this if you want to land a DevOps job, this if you want to land a DevOps job, this is where you need to go. I'll share the is where you need to go. I'll share the is where you need to go. I'll share the link. link. link. Uh kubecraft.dev. I'll share the link in the chat for you. I'll share the link in the chat for you. You can check that out if you want. You can check that out if you want. You can check that out if you want. But that is basically what I'm uh my advice. If you want to land a DevOps my advice. If you want to land a DevOps job, go job, go job, go there. there. there. Um I'm reading O'Reilly books Exercism I'm reading O'Reilly books Exercism start with Bash. Any specific resources start with Bash. Any specific resources start with Bash. Any specific resources recommend to speed up? recommend to speed up? recommend to speed up? Oh, that sounds pretty good. Oh, that sounds pretty good. Oh, that sounds pretty good. Exercism is great. Exercism is great. Exercism is great. Bash is also good. Bash is also good. Bash is also good. And then I would move to Python if I And then I would move to Python if I And then I would move to Python if I were you. were you. were you. Locked with path one router mode llama Locked with path one router mode llama Locked with path one router mode llama server. Here's something worth flagging server. Here's something worth flagging server. Here's something worth flagging up front. Rack backend needs zero code up front. Rack backend needs zero code up front. Rack backend needs zero code changes. changes. changes. That's good. That's exactly what we were That's good. That's exactly what we were That's good. That's exactly what we were um um um expecting. expecting. expecting. Nothing to ship in backend. Great.

Nothing to ship in backend. Great. Nothing to ship in backend. Great. I have all the answers I need to present I have all the answers I need to present I have all the answers I need to present the local stack design. the local stack design. the local stack design. Uh Uh Uh local architect Yeah, I love this. I'm local architect Yeah, I love this. I'm local architect Yeah, I love this. I'm using superpowers by the way. using superpowers by the way. using superpowers by the way. And I'm loving it how this brainstorming And I'm loving it how this brainstorming And I'm loving it how this brainstorming is taking you through it step by step is taking you through it step by step is taking you through it step by step instead of just boom going straight into instead of just boom going straight into instead of just boom going straight into the the building. the the building. the the building. And these are like the the the tricks And these are like the the the tricks And these are like the the the tricks that that that that that that we you need to learn as someone who is we you need to learn as someone who is we you need to learn as someone who is doing agentic engineering, right? doing agentic engineering, right? doing agentic engineering, right? So, let's see. Fedora host rootless So, let's see. Fedora host rootless So, let's see. Fedora host rootless toolbox. Yep, llama server. Quinn coder nomic embed. Yep, host Quinn coder nomic embed. Yep, host containers podman compose network k c i. containers podman compose network k c i. containers podman compose network k c i. Chat UI inter service rack backend. Chat UI inter service rack backend. Chat UI inter service rack backend. Yes. Yes. Yes. Yes. Yes. Yes. Yes. Yes. Yes. Superbase staging cloud. Superbase staging cloud. Superbase staging cloud. Yes. Compose file shape build context Compose file shape build context env file env local. Yep.

No health check, that's fine. No health check, that's fine. No ports No ports No ports reachable from chat UI. Yes. reachable from chat UI. Yes. reachable from chat UI. Yes. Network implicit default bridge Network implicit default bridge Network implicit default bridge or whatever podman. Yep. or whatever podman. Yep. or whatever podman. Yep. No model server service No model server service No model server service outside compose. Local env example env local. Yes.

Local env example env local. Yes. Sounds good. Run book. Run book. Contents are defined here as section Contents are defined here as section Contents are defined here as section headings. headings. headings. Actual prose land string. Uh Uh we don't we don't we don't um Let's not do the run book Let's not do the run book right now. Create issue to do so.

right now. Create issue to do so. right now. Create issue to do so. We'll probably We'll probably We'll probably need some debugging until need some debugging until need some debugging until it all works. So, soon it should start should be able So, soon it should start should be able to start cooking. Thanks, Misha. I really appreciate Thanks, Misha. I really appreciate everything you've put out so far for us. everything you've put out so far for us. everything you've put out so far for us. I'm benefiting a lot from your content. I'm benefiting a lot from your content. I'm benefiting a lot from your content. Thanks for Thanks for Thanks for saying so.

saying so. saying so. Glad that my content is helping people Glad that my content is helping people Glad that my content is helping people around the world. So, the idea here is that it's going to So, the idea here is that it's going to be able to be able to be able to um um um write the spec. write the spec. write the spec. And after that, it's going to be able to And after that, it's going to be able to And after that, it's going to be able to just just just um cook for a while on its own. um cook for a while on its own. um cook for a while on its own. And as it's doing that, then I'll be And as it's doing that, then I'll be And as it's doing that, then I'll be then I'm going to set up the toolbox. So, that we can actually try to connect So, that we can actually try to connect to the model. to the model. to the model. But, it shouldn't be a it shouldn't like But, it shouldn't be a it shouldn't like But, it shouldn't be a it shouldn't like it's a little bit overkill for it to it's a little bit overkill for it to it's a little bit overkill for it to like write a whole spec for this, but I like write a whole spec for this, but I like write a whole spec for this, but I do appreciate do appreciate do appreciate that we went through this because I that we went through this because I that we went through this because I actually didn't realize that the actually didn't realize that the actually didn't realize that the embedding model needs to be running.

Okay, so it's now going to write the Okay, so it's now going to write the spec. That's fine. That's fine. Let's see. Let's see. Let's see. Toolbox list. list. So, I do have a strix ROCm toolbox So, I do have a strix ROCm toolbox So, I do have a strix ROCm toolbox running already. running already. running already. Toolbox enter strix ROCm. Yeah, I did

Yeah, I did um Let's see if it's now actually Let's see if it's now actually working. Okay, so it's creating the design spec.

Okay, so it's creating the design spec. That's good. So, I was running a local So, I was running a local transcription uh toolbox here, uh toolbox here, and I started running whisper in there. and I started running whisper in there. and I started running whisper in there. But, maybe can I do toolbox help? help? Can I rename a toolbox? I kind of want to just I kind of want to just start over with a fresh one start over with a fresh one start over with a fresh one because I believe I do have it because I believe I do have it because I believe I do have it documented.

Okay, so I will just toolbox Okay, so I will just toolbox RM RM RM strix ROCm. Uh oh, yeah.

Uh oh, yeah. Toolbox enter. Let's see. What OS should you recommend Let's see. What OS should you recommend for a home lab Kubernetes cluster? for a home lab Kubernetes cluster? for a home lab Kubernetes cluster? Yeah. Yeah. Yeah. Theo, this is the kind of stuff that you Theo, this is the kind of stuff that you Theo, this is the kind of stuff that you need to join CubeCraft for. Like this is need to join CubeCraft for. Like this is need to join CubeCraft for. Like this is I have it all specked out and all all I have it all specked out and all all I have it all specked out and all all helping it I'm all helping people answer helping it I'm all helping people answer helping it I'm all helping people answer these questions. these questions. these questions. Um check out cubecraft.dev. The link is Um check out cubecraft.dev. The link is Um check out cubecraft.dev. The link is actually below up here, but I will show actually below up here, but I will show actually below up here, but I will show you. you. you. Um Like this is what I do for my main Like this is what I do for my main um work is I help people with home labs, um work is I help people with home labs, um work is I help people with home labs, landing DevOps jobs, etc. So, all those landing DevOps jobs, etc. So, all those landing DevOps jobs, etc. So, all those questions are answered there. questions are answered there. questions are answered there. And what operating system is Misha And what operating system is Misha And what operating system is Misha running? I'm using Fedora for running? I'm using Fedora for running? I'm using Fedora for for sway atomic, exactly. Yeah.

Okay, so now we see that I have my GPU Okay, so now we see that I have my GPU available inside of this toolbox, which available inside of this toolbox, which available inside of this toolbox, which is what we wanted. is what we wanted. is what we wanted. Spec written and committed. Spec written and committed. Spec written and committed. Uh please review the spec. Uh please review the spec. Uh please review the spec. Two natural steps once you approve. Two natural steps once you approve. Two natural steps once you approve. It's a prerequisite. I think I think we can skip the whole spec flow we can skip the whole spec flow we can skip the whole spec flow for the migration. It's straightforward. for the migration. It's straightforward. for the migration. It's straightforward. Just do it. No, I do never I never planned to No, I do never I never planned to release those courses independently. Uh yes, I use Uh yes, I use um RPM-OS 3. Yep. And then I just have um RPM-OS 3. Yep. And then I just have um RPM-OS 3. Yep. And then I just have everything inside of containers, so everything inside of containers, so everything inside of containers, so everything is defined as code anyway.

>> Guys, it's now going to create a plan to >> Guys, it's now going to create a plan to just Um, like I have all of this data in just Um, like I have all of this data in just Um, like I have all of this data in my production super base. my production super base. my production super base. But, I want to have that in my staging But, I want to have that in my staging But, I want to have that in my staging super base. So, the local cluster the super base. So, the local cluster the super base. So, the local cluster the local development is talking against local development is talking against local development is talking against that one. that one. that one. That's just good practice to do. Even though there's literally zero risk Even though there's literally zero risk for for for um There's zero risk for There's zero risk for any issues with production. And in fact, any issues with production. And in fact, any issues with production. And in fact, it's not actually production, but I it's it's not actually production, but I it's it's not actually production, but I it's just good to have it just good to have it just good to have it separated. separated. separated. Pull Postgres UI from chat UI env. Pull Postgres UI from chat UI env. Pull Postgres UI from chat UI env. Both are session pool area. Install.

Actually Actually um Uh, let's see. Where is it? Uh, let's see. Where is it? Settings. Settings. Settings. I connect. That's it. And And copy prompt. copy prompt. copy prompt. Can we use Superbase instead? Uh, marketplace. Uh, marketplace. Superbase. I actually don't like using MCPs. I actually don't like using MCPs. Let's just see what it does here.

Yeah, Fedora is not you definitely not Yeah, Fedora is not you definitely not for Kubernetes use. Don't like for Kubernetes use. Don't like for Kubernetes use. Don't like Fedora like my I'm using this as my Fedora like my I'm using this as my Fedora like my I'm using this as my workstation. I wouldn't run Kubernetes clusters on I wouldn't run Kubernetes clusters on Fedora. So, that's just llama server. Okay. So, that's just llama server. Okay. That sounds good.

Uh, Claude. Uh, Claude. Can you figure out is available is available to us? to us? to us? Oh, lol. It's literally saying that Oh, lol. It's literally saying that Oh, lol. It's literally saying that here. here. here. So, why is there only VRAM available? VRAM available? My machine is supposed to have 128. Um, the stuff you shared is Superbase is Um, the stuff you shared is Superbase is built for application.

Lightwell is another immutable base OS Lightwell is another immutable base OS for running Docker containers out of the for running Docker containers out of the for running Docker containers out of the box. That's an interesting idea. box. That's an interesting idea. box. That's an interesting idea. Lights well. Make Linux servers Make Linux servers fun again. I don't like that it's that it's focused I don't like that it's that it's focused on Docker, though. I would prefer to use Podman at this I would prefer to use Podman at this stage. That's also why I like Fedora Atomic. That's also why I like Fedora Atomic. But, it's basically the same, it seems. Seems like the same idea Seems like the same idea than Fedora Atomic. Flatcar Container Linux. Open source Flatcar Container Linux. Open source community powered innovation.

community powered innovation. community powered innovation. There is just so much There is just so much There is just so much so many different Linux distributions so many different Linux distributions so many different Linux distributions out there. Like out there. Like out there. Like Wow. The container infrastructure OS. Flatcar The container infrastructure OS. Flatcar Container Linux is designed from the Container Linux is designed from the Container Linux is designed from the ground up for container running ground up for container running ground up for container running workloads. workloads. workloads. Cool. Yeah, immutable Linux OS designed to Yeah, immutable Linux OS designed to scale. my my tables.

Yeah, Q&A's calls. Yeah, Q&A's calls. Cool. Cool. Cool. It now has the transcripts in here. So, It now has the transcripts in here. So, It now has the transcripts in here. So, it should be able to it should be able to it should be able to um um um embed them. I like using Ubuntu for AI workloads on I like using Ubuntu for AI workloads on Kubernetes. Easier to connect to the Kubernetes. Easier to connect to the Kubernetes. Easier to connect to the Kubernetes clusters to the video GPU Kubernetes clusters to the video GPU Kubernetes clusters to the video GPU using Ubuntu as base OS. using Ubuntu as base OS. using Ubuntu as base OS. Yeah, Ubuntu is solid. Yeah, Ubuntu is solid. Yeah, Ubuntu is solid. For sure. Okay, so we have migrated the data. So, Okay, so we have migrated the data. So, that's good. that's good. that's good. My calls, my videos. My calls, my videos. My calls, my videos. Read only. Okay. Um Um Now, it wants me to review the docs. Now, it wants me to review the docs. Now, it wants me to review the docs. >> [snorts] >> Podman >> Podman specs.

specs. specs. Specs for issue, yeah. Goals. One command Podman compose up. Goals. One command Podman compose up. Full Superbase staging parity. Architecture. Architecture. Yeah. Yeah. Yeah. Look at how good this documentation is. Look at how good this documentation is. Look at how good this documentation is. Like Like Like This is how you this is how you prompt This is how you this is how you prompt This is how you this is how you prompt AI models guys. This is how you do it. AI models guys. This is how you do it. AI models guys. This is how you do it. You give it you plan it out beforehand. You give it you plan it out beforehand. You give it you plan it out beforehand. Like I've been spending a full hour Like I've been spending a full hour Like I've been spending a full hour planning this. But when you do that, planning this. But when you do that, planning this. But when you do that, then the result is going to be so much then the result is going to be so much then the result is going to be so much better. better. better. This is how you will one shot things. Okay. Okay. Yep. Yep. Yep. Yep. Yep. Yep. Okay. So I'll have to

Okay. So I'll have to Um Um Um yeah. yeah. yeah. I have to go into my BIOS and increase I have to go into my BIOS and increase I have to go into my BIOS and increase the VRAM. Well, that explains it. So I have to Well, that explains it. So I have to reboot it after the stream. reboot it after the stream. reboot it after the stream. And up my VRAM. Direction goal of development where

Direction goal of development where application coding application coding application coding is toward our prompts docs and PRDs is toward our prompts docs and PRDs is toward our prompts docs and PRDs becoming the primary source of truth becoming the primary source of truth becoming the primary source of truth committed to get. Yes, exactly Barco. committed to get. Yes, exactly Barco. committed to get. Yes, exactly Barco. This is so well put because This is so well put because This is so well put because Um let me start the sub agent Um let me start the sub agent Um let me start the sub agent development. Cool. development. Cool. development. Cool. Uh this is actually so right because the Uh this is actually so right because the Uh this is actually so right because the thing is the code is not the thing is the code is not the thing is the code is not the the the the result anymore, right? the the the result anymore, right? the the the result anymore, right? So what you are So what you are So what you are what you are building is the specs. And what you are building is the specs. And what you are building is the specs. And then if you have the right specs, then if you have the right specs, then if you have the right specs, then with those specs, you can just then with those specs, you can just then with those specs, you can just regenerate the code. And you can even do regenerate the code. And you can even do regenerate the code. And you can even do it in any language, right? it in any language, right? it in any language, right? So So So this is the the the new way of creating this is the the the new way of creating this is the the the new way of creating software and applications software and applications software and applications is to spend so much time on specs. Like is to spend so much time on specs. Like is to spend so much time on specs. Like I'm I've now spent an hour on a spec on I'm I've now spent an hour on a spec on I'm I've now spent an hour on a spec on building a local Podman development building a local Podman development building a local Podman development setup.

setup. setup. Which Which Which perhaps might have took taken me less perhaps might have took taken me less perhaps might have took taken me less time if I had time if I had time if I had just written it myself. just written it myself. just written it myself. But of course I'm also just learning the But of course I'm also just learning the But of course I'm also just learning the agentic development workflows. agentic development workflows. agentic development workflows. So I choose to do it that way. But So I choose to do it that way. But So I choose to do it that way. But because I have created the specs as you because I have created the specs as you because I have created the specs as you see here, uh let's see. uh let's see. Where was it? Here. Look at how detailed this this Here. Look at how detailed this this spec is. spec is. spec is. Um oh, it's not rendering correctly Um oh, it's not rendering correctly Um oh, it's not rendering correctly because of the because of the because of the Here we go. Look at how detailed all of Here we go. Look at how detailed all of Here we go. Look at how detailed all of this is. this is. this is. Just for a local development setup, Just for a local development setup, Just for a local development setup, right? Like this is better than most right? Like this is better than most right? Like this is better than most documentation you will find in the wild. documentation you will find in the wild. documentation you will find in the wild. So So So because you commit you you create the because you commit you you create the because you commit you you create the documentation, you commit it, documentation, you commit it, documentation, you commit it, you'll always be able to reproduce what you'll always be able to reproduce what you'll always be able to reproduce what you have created you have created you have created even in different languages. even in different languages. even in different languages. So that is going to be the real work.

They're working towards reverse They're working towards reverse engineering the newer M4 M5 chips, too. engineering the newer M4 M5 chips, too. engineering the newer M4 M5 chips, too. But unfortunately still have M1 lying But unfortunately still have M1 lying But unfortunately still have M1 lying around and some other e-waste. around and some other e-waste. around and some other e-waste. And use it to learn devops. Yeah, that's And use it to learn devops. Yeah, that's And use it to learn devops. Yeah, that's cool. cool. cool. Hello JC. So what Misha's doing is spending time So what Misha's doing is spending time in taking great care of prompts and in taking great care of prompts and in taking great care of prompts and design phase seems like the better way design phase seems like the better way design phase seems like the better way forward. Yes, exactly. Are we actually live on Twitch right Are we actually live on Twitch right now? now? now? Yes, we are. And how many people are watching on And how many people are watching on Twitch?

Uh one one viewer on Twitch. Uh one one viewer on Twitch. One on Twitch. One on Twitch. One on Twitch. And we have 49 49 on YouTube, probably. on YouTube, probably. on YouTube, probably. Are we live on X, too? And I asked I asked Grok to And I asked I asked Grok to summarize it. Did it work? How is it going to for you today? Yes, How is it going to for you today? Yes, I'm doing good. How about yourself? Inference on Framework PC will be very Inference on Framework PC will be very slow. Okay. I will we're going to find slow. Okay. I will we're going to find slow. Okay. I will we're going to find out.

Like the inference on my 4090 Like the inference on my 4090 on the actual production app is really on the actual production app is really on the actual production app is really good. good. good. Uh what OS is Misha running right now? Doing of course Raspberry by Doing of course Raspberry by Okay, it's Arch Linux recovery trouble. Okay, it's Arch Linux recovery trouble. Okay, it's Arch Linux recovery trouble. Yeah, MacBook as a fallback. Yeah, MacBook as a fallback. Yeah, MacBook as a fallback. Um Let's see if it can find the latest Q&A Let's see if it can find the latest Q&A calls. Only use Q&A calls from Q&A calls from 2026. What distro is Mischa using? Okay, not explicitly named. Okay, not explicitly named. He did confirm that Linux distribution He did confirm that Linux distribution He did confirm that Linux distribution like Fedora is viable. like Fedora is viable. like Fedora is viable. Okay. So, we didn't discuss that during Okay. So, we didn't discuss that during Okay. So, we didn't discuss that during the Q&A calls that much.

the Q&A calls that much. the Q&A calls that much. But, if I just run, do you have any tips But, if I just run, do you have any tips But, if I just run, do you have any tips for the CKA? for the CKA? for the CKA? You see how fast this is, right? So, You see how fast this is, right? So, You see how fast this is, right? So, this is all running on Kubernetes. this is all running on Kubernetes. this is all running on Kubernetes. Um using Gemma Um using Gemma Um using Gemma in my local cluster on a 4090. in my local cluster on a 4090. in my local cluster on a 4090. So, that's working well. For the past 4 months, I built an agent For the past 4 months, I built an agent orchestrator. My project is getting orchestrator. My project is getting orchestrator. My project is getting solid. Okay. solid. Okay. solid. Okay. So, how many agents are you running and So, how many agents are you running and So, how many agents are you running and what are you building with them? what are you building with them? what are you building with them? Because we were just looking at a tweet Because we were just looking at a tweet Because we were just looking at a tweet uh just now. Yeah, here, this one. Does uh just now. Yeah, here, this one. Does uh just now. Yeah, here, this one. Does anyone actually run 20 agents in anyone actually run 20 agents in anyone actually run 20 agents in parallel? Right?

parallel? Right? parallel? Right? And And And someone is saying yes. someone is saying yes. someone is saying yes. But, But, But, not 20, but I've definitely run five 10 not 20, but I've definitely run five 10 not 20, but I've definitely run five 10 before for complex tasks. before for complex tasks. before for complex tasks. But, But, But, yeah, I mean based on the reactions, based on the reactions, no one no one no one is actually running 10 plus agents and is actually running 10 plus agents and is actually running 10 plus agents and doing any doing any doing any meaningful work. meaningful work. meaningful work. >> [clears throat] >> Okay, so it seems like it is now done. >> Okay, so it seems like it is now done. So, compose the YAML. compose the YAML. Okay, so if I go Podman compose pose executable file not Podman compose pose executable file not found in path.

found in path. found in path. Aha. Okay. Okay. So, So, So, Fedora Fedora Fedora atomic atomic atomic podman compose. Fedora sway atomic. Fedora sway atomic. Um It's referring the entire documentation

It's referring the entire documentation here. here. here. Cuz why is podman compose Cuz why is podman compose Cuz why is podman compose not installed not installed not installed on Fedora atomic? In order for podman compose to work with In order for podman compose to work with his default configuration, you must like his default configuration, you must like his default configuration, you must like What? Why would I need docker compose?

Let's see. Podman documentation. This is Let's see. Podman documentation. This is why I'm I'm learning podman, right? why I'm I'm learning podman, right? why I'm I'm learning podman, right? Because Podman remote. Podman remote. Compose. Podman compose. Podman compose. Is a thin wrapper around external Is a thin wrapper around external Is a thin wrapper around external compose provider such as docker compose compose provider such as docker compose compose provider such as docker compose or podman compose. or podman compose. or podman compose. So, you probably is executing another So, you probably is executing another So, you probably is executing another tool that implements the compose tool that implements the compose tool that implements the compose functionality. But, why is But, why is podman com which podman Why is podman compose not installed on Why is podman compose not installed on this system? My project different. I have a chat My project different. I have a chat layer only talk delegate task agents layer only talk delegate task agents layer only talk delegate task agents building a backlog and common board building a backlog and common board building a backlog and common board assigned. It's pretty solid.

assigned. It's pretty solid. assigned. It's pretty solid. Yeah, okay, that that sounds cool. But, Yeah, okay, that that sounds cool. But, Yeah, okay, that that sounds cool. But, then again, like what are you building, then again, like what are you building, then again, like what are you building, right? Like, Like, I'm able to run I'm able to run I'm able to run just many agents using tmux here. So, I just many agents using tmux here. So, I just many agents using tmux here. So, I just do it this way just do it this way just do it this way and it's totally fine. They work all the and it's totally fine. They work all the and it's totally fine. They work all the time. But, the thing is like I'm limited by But, the thing is like I'm limited by the ideas that I have and the ability to the ideas that I have and the ability to the ideas that I have and the ability to just keep the agents running. We are AI native, so we're going to ask We are AI native, so we're going to ask Claude. I use Fedora atomic sway. Claude. I use Fedora atomic sway. Claude. I use Fedora atomic sway. But, podman compose does not But, podman compose does not But, podman compose does not work work work and podman compose and podman compose and podman compose is not installed.

is not installed. is not installed. What do? Because I can't just install podman Because I can't just install podman compose either. I might have to layer compose either. I might have to layer compose either. I might have to layer that in that in that in in the RPM-OSTree. Do you plan to upgrade the newer Do you plan to upgrade the newer Framework laptop or upgrade strictly the Framework laptop or upgrade strictly the Framework laptop or upgrade strictly the motherboard? Yeah, it's a bit motherboard? Yeah, it's a bit motherboard? Yeah, it's a bit unfortunate. I've been using my one for unfortunate. I've been using my one for unfortunate. I've been using my one for three four months now and now there is a three four months now and now there is a three four months now and now there is a new one. I would rather have the new new one. I would rather have the new new one. I would rather have the new one, but I'm actually pretty satisfied one, but I'm actually pretty satisfied one, but I'm actually pretty satisfied with my current one. with my current one. with my current one. The only thing I don't really like is The only thing I don't really like is The only thing I don't really like is the the the um The only thing I don't really like is The only thing I don't really like is the screen for working outside.

the screen for working outside. the screen for working outside. So, yeah, maybe I do have to So, yeah, maybe I do have to So, yeah, maybe I do have to fix it. See, on atomic, the clean fixes See, on atomic, the clean fixes installed as a user level Python tool installed as a user level Python tool installed as a user level Python tool the rather than layering it. the rather than layering it. the rather than layering it. Podman compose is just a wrapper that Podman compose is just a wrapper that Podman compose is just a wrapper that shells to an external compose provider. shells to an external compose provider. shells to an external compose provider. It'll find podman compose on your path It'll find podman compose on your path It'll find podman compose on your path and work transparently once it's there. Yeah, exactly. I don't want to do it Yeah, exactly. I don't want to do it inside of a toolbox because then it inside of a toolbox because then it inside of a toolbox because then it becomes very awkward. becomes very awkward. becomes very awkward. But, is there Mies? Um Mies Um Mies s- s- s- search search search podman. Damn it, it's not in podman either. Mm, Damn it, it's not in podman either. Mm, downloads. or download and install Podman compose or download and install Podman compose yourself.

yourself. yourself. Here we go. Oh yeah, Podman compose containers. Oh yeah, Podman compose containers. Okay. Okay. Okay. An implementation of compose back with An implementation of compose back with An implementation of compose back with Podman back end. This script will download the repo This script will download the repo generate the binary and place the binary generate the binary and place the binary generate the binary and place the binary in the directory where you call the in the directory where you call the in the directory where you call the script. That's interesting. And this will be user local bin. And this will be user local bin. Okay, so we'll just curl warning failed to open the file warning failed to open the file permission denied user local bin home Misha bin

user local bin home Misha bin Okay, so we'll just curl it into Okay. Okay. Then CH mod Then CH mod Then CH mod +X executing external compose, yes.

executing external compose, yes. At least it's running. At least it's running. At least it's running. But now we're having an issue with the But now we're having an issue with the But now we're having an issue with the .env. So it's a binary, but it doesn't have So it's a binary, but it doesn't have the required the required the required uh uh uh Why? Okay, maybe if I do it Okay, maybe if I do it um Let's see. How was it again with pip? Um Let's see. How was it again with pip? Um mise pip Wait, which UV? Is UV installed?

Wait, which UV? Is UV installed? No UV. If you have UV installed, UV tool If you have UV installed, UV tool install Uh this becomes tricky because I'm Uh this becomes tricky because I'm installing it on my local system, but I installing it on my local system, but I installing it on my local system, but I think we'll just have to mise use UV. um um UV pip install Podman compose. What's UV pip install Podman compose. What's UV pip install Podman compose. What's this then?

Yeah, okay. So it's failing to create Yeah, okay. So it's failing to create the directory because the directory because the directory because LS user local LS user local LS user local lib to install it there. Hmm. to install it there. Hmm. Cuz user is also read-only. The thing is I'm the what I'm trying to The thing is I'm the what I'm trying to install right now is to run Podman install right now is to run Podman install right now is to run Podman compose. compose. compose. That's why I'm trying to use um I need That's why I'm trying to use um I need That's why I'm trying to use um I need to install these lib system-wide. to install these lib system-wide. to install these lib system-wide. Because apparently the Even if I get the binary for Podman Even if I get the binary for Podman compose, it wasn't working.

compose, it wasn't working. compose, it wasn't working. Which is really strange. So I will have to I will probably have So I will have to I will probably have to layer this into my to layer this into my to layer this into my um um um RPM-OSTree instead. And that means that I will have to And that means that I will have to um um um stop the stream and it's actually 8:00 stop the stream and it's actually 8:00 stop the stream and it's actually 8:00 p.m. So p.m. So p.m. So hold on. I'll be back in a second.

Yeah, I just checked on my dinner which Yeah, I just checked on my dinner which is cooking which is going to be is cooking which is going to be is cooking which is going to be uh going to be ready in going to be ready in 12 12 minutes from now. 12 12 minutes from now. 12 12 minutes from now. So So So what I actually need to do is I need to what I actually need to do is I need to what I actually need to do is I need to sudo or a RPM >> What's for dinner? It's uh chicken >> What's for dinner? It's uh chicken drumsticks drumsticks drumsticks and broccoli. Yeah, so here you will see that it has Yeah, so here you will see that it has now installed Podman compose now installed Podman compose now installed Podman compose python.env exactly what was needed. The python.env exactly what was needed. The python.env exactly what was needed. The the issue that we were running into.

the issue that we were running into. the issue that we were running into. So let me see. Is it uninstall? Yeah, UV pip Is it uninstall? Yeah, UV pip uninstall Podman compose system system Okay. system Okay. So we haven't installed it and mise So we haven't installed it and mise So we haven't installed it and mise uninstall uninstall uninstall UV meese up meese up Just do it. Up.

What is it? meese purge? What was it What is it? meese purge? What was it again? This is where I drop into plan mode and This is where I drop into plan mode and ask Claude for my options. ask Claude for my options. ask Claude for my options. Yeah. Yeah. Yeah. Yeah, that's the thing. Um Yeah, that's the thing. Um Yeah, that's the thing. Um The thing is like The thing is like The thing is like I'm able to just easily navigate all of I'm able to just easily navigate all of I'm able to just easily navigate all of these paths and all of the like these paths and all of the like these paths and all of the like binaries. Like I know exactly what this binaries. Like I know exactly what this binaries. Like I know exactly what this means. Like I'm curling the binary into means. Like I'm curling the binary into means. Like I'm curling the binary into a directory and now I I could instantly a directory and now I I could instantly a directory and now I I could instantly read read read that because that because that because um um um here here here No. No. No. Here. Here. Here. Like Like Like Here I was trying like, "Okay, I will Here I was trying like, "Okay, I will Here I was trying like, "Okay, I will install I'll install the binary.

I'll install the binary. And then I see it's modded. And then I see it's modded. And then I see it's modded. And then And then And then the binary was there, but I could the binary was there, but I could the binary was there, but I could instantly see, "Okay, we are missing the instantly see, "Okay, we are missing the instantly see, "Okay, we are missing the uh some uh some uh some dependencies for the dev dependencies." dependencies for the dev dependencies." dependencies for the dev dependencies." Which is annoying. Like if it had been a Which is annoying. Like if it had been a Which is annoying. Like if it had been a go binary, then all of this would have go binary, then all of this would have go binary, then all of this would have not been a problem, right? not been a problem, right? not been a problem, right? So, I'm not understanding why Podman So, I'm not understanding why Podman So, I'm not understanding why Podman compose couldn't just be a single compose couldn't just be a single compose couldn't just be a single binary. So, then I just tried meese and binary. So, then I just tried meese and binary. So, then I just tried meese and then I I instantly recognized, "Oh, then I I instantly recognized, "Oh, then I I instantly recognized, "Oh, yeah, okay, there's still no dot env." yeah, okay, there's still no dot env." yeah, okay, there's still no dot env." And I cannot write to it because it is And I cannot write to it because it is And I cannot write to it because it is um part of the read-only file system. um part of the read-only file system. um part of the read-only file system. So, So, So, um um um I would I just In in the end, I just I would I just In in the end, I just I would I just In in the end, I just layer it instead. So, if I do rpm ostree layer it instead. So, if I do rpm ostree layer it instead. So, if I do rpm ostree status status status then then then we see now there a third deployment is we see now there a third deployment is we see now there a third deployment is added. added. added. And And And if I will now reboot my system, then if I will now reboot my system, then if I will now reboot my system, then Podman compose is going to be installed Podman compose is going to be installed Podman compose is going to be installed in my system and I'm going to be able to in my system and I'm going to be able to in my system and I'm going to be able to run it.

run it. run it. Um Um Um So, that's it basically. So, that's it basically. So, that's it basically. And I I can like I I don't need Claude And I I can like I I don't need Claude And I I can like I I don't need Claude for that because I've done this so much for that because I've done this so much for that because I've done this so much in my years as a DevOps engineer setting in my years as a DevOps engineer setting in my years as a DevOps engineer setting up and building environments that this up and building environments that this up and building environments that this stuff is second nature to me. stuff is second nature to me. stuff is second nature to me. Um system-wide versus a local sandbox. Um system-wide versus a local sandbox. Um system-wide versus a local sandbox. Yeah, I'm Well, I'm very particular with Yeah, I'm Well, I'm very particular with Yeah, I'm Well, I'm very particular with this stuff. Like everything is running this stuff. Like everything is running this stuff. Like everything is running in containers. in containers. in containers. Everything is um Everything is um Everything is um uh uh uh something that I um um I'm very particular about. You know you I'm very particular about. You know you I'm very particular about. You know you can just ask Claude to install it for can just ask Claude to install it for can just ask Claude to install it for you, right? Well, you, right? Well, you, right? Well, that is a good point, but then like I'm that is a good point, but then like I'm that is a good point, but then like I'm running this on my local system, right? running this on my local system, right? running this on my local system, right? And I'm not allowed to run Claude And I'm not allowed to run Claude And I'm not allowed to run Claude directly in my CLI. I will need to run directly in my CLI. I will need to run directly in my CLI. I will need to run Claude from within a toolbox in that Claude from within a toolbox in that Claude from within a toolbox in that case. case. case. And it could probably do that. And it could probably do that. And it could probably do that. But then it's installing But then it's installing But then it's installing Well, let's see. Toolbox.

Well, let's see. Toolbox. Well, let's see. Toolbox. Enter. Enter. Enter. tools tools tools Claude Claude Claude If you were to install Podman compose If you were to install Podman compose If you were to install Podman compose on this Fedora Atomic Linux system on this Fedora Atomic Linux system on this Fedora Atomic Linux system how would you do it? Is Kubernetes good for building storage Is Kubernetes good for building storage systems? systems? systems? Yes, absolutely. Yes, absolutely. Yes, absolutely. If you use If you use If you use um Ceph Rook Yeah, exactly what Marco is um Ceph Rook Yeah, exactly what Marco is um Ceph Rook Yeah, exactly what Marco is saying. If you set up Kubernetes saying. If you set up Kubernetes saying. If you set up Kubernetes with Ceph Rook with Ceph Rook with Ceph Rook 100% um you can 100% um you can 100% um you can like you can definitely build your own like you can definitely build your own like you can definitely build your own storage OS basically, for sure. Issue is Ceph needs three plus nodes. Issue is Ceph needs three plus nodes. Well, I mean, duh. If you are Well, I mean, duh. If you are Well, I mean, duh. If you are if you want to build your own storage if you want to build your own storage if you want to build your own storage system, then you have more nodes, right?

system, then you have more nodes, right? system, then you have more nodes, right? Like if it's just one thing, then why do Like if it's just one thing, then why do Like if it's just one thing, then why do you need to build your own storage you need to build your own storage you need to build your own storage system? Uh okay. So, Uh okay. So, Claude was saying rpm-ostree. Yeah, Claude was saying rpm-ostree. Yeah, Claude was saying rpm-ostree. Yeah, okay. So, that's what I ended up at. okay. So, that's what I ended up at. okay. So, that's what I ended up at. pipx install I went to that through that pipx install I went to that through that pipx install I went to that through that through through through um um um Brew Brew Brew or um or um or um meese, right? meese, right? meese, right? So, So, So, we arrived at the same things. Would we arrived at the same things. Would we arrived at the same things. Would this have been faster? Probably. Yeah, this have been faster? Probably. Yeah, this have been faster? Probably. Yeah, you're right. you're right. you're right. But on the other hand, like But on the other hand, like But on the other hand, like Like I'm not a software engineer, right? Like I'm not a software engineer, right? Like I'm not a software engineer, right? I'm not a software engineer, so I am I'm not a software engineer, so I am I'm not a software engineer, so I am I am very I am very I am very comfortable with outsourcing that to the comfortable with outsourcing that to the comfortable with outsourcing that to the AI AI AI because first of all, that's what it's because first of all, that's what it's because first of all, that's what it's good at. good at. good at. But second of all, like I But second of all, like I But second of all, like I I have coding skills. I can I can write I have coding skills. I can I can write I have coding skills. I can I can write code from scratch, but I'm not a code from scratch, but I'm not a code from scratch, but I'm not a software engineer. software engineer. software engineer. And so, I don't really have like skills And so, I don't really have like skills And so, I don't really have like skills to maintain there. But like my sysadmin to maintain there. But like my sysadmin to maintain there. But like my sysadmin command-line skills Like this is what I command-line skills Like this is what I command-line skills Like this is what I what I'm good at. This is my profession what I'm good at. This is my profession what I'm good at. This is my profession and what I enjoy doing. And and what I enjoy doing. And and what I enjoy doing. And if you just always outsource everything if you just always outsource everything if you just always outsource everything to the AI, I have noticed in my own work to the AI, I have noticed in my own work to the AI, I have noticed in my own work that a lot of that stuff really quickly that a lot of that stuff really quickly that a lot of that stuff really quickly starts to atrophy. So, I would starts to atrophy. So, I would starts to atrophy. So, I would definitely recommend definitely recommend definitely recommend uh being selective with what you uh being selective with what you uh being selective with what you outsource to the AI outsource to the AI outsource to the AI and and also do some thinking yourself.

You You should never You You should never always keep thinking, okay? Don't just always keep thinking, okay? Don't just always keep thinking, okay? Don't just outsource all of your thinking to the outsource all of your thinking to the outsource all of your thinking to the AI. Be very careful. I'm very careful AI. Be very careful. I'm very careful AI. Be very careful. I'm very careful with this cuz I have really noticed some with this cuz I have really noticed some with this cuz I have really noticed some areas in my life areas in my life areas in my life and in my work where I outsourced it to and in my work where I outsourced it to and in my work where I outsourced it to the AI and my brain started rotting the AI and my brain started rotting the AI and my brain started rotting and I was not able to do it and I was not able to do it and I was not able to do it uh uh uh anymore. Like it's very quick how how anymore. Like it's very quick how how anymore. Like it's very quick how how easy and quickly you you lose these easy and quickly you you lose these easy and quickly you you lose these skills. skills. skills. Or lose these skills, I mean. Or lose these skills, I mean. Or lose these skills, I mean. So, okay, this is where the this stream So, okay, this is where the this stream So, okay, this is where the this stream has ended then. So, if we take a look at has ended then. So, if we take a look at has ended then. So, if we take a look at the actual result what it has built, it the actual result what it has built, it the actual result what it has built, it has built a compose.yaml. has built a compose.yaml. has built a compose.yaml. And it should be able to And it should be able to And it should be able to um build locally build and connect to my build locally build and connect to my staging Supabase environment. It has staging Supabase environment. It has staging Supabase environment. It has migrated all of the production data to migrated all of the production data to migrated all of the production data to my Supabase environment my Supabase environment my Supabase environment um to my staging environment. um to my staging environment. um to my staging environment. And then And then And then I will need to fill out the I will need to fill out the I will need to fill out the credentials. Actually, I should just ask credentials. Actually, I should just ask credentials. Actually, I should just ask it to do that.

it to do that. it to do that. Create the env file. And And so, when I reboot my system so, when I reboot my system so, when I reboot my system I should be able to then I should be able to then I should be able to then um um um have a local version that does pull in have a local version that does pull in have a local version that does pull in the production data from my staging the production data from my staging the production data from my staging environment. environment. environment. And And And I should then be able to I should then be able to I should then be able to connect to my different toolbox connect to my different toolbox connect to my different toolbox which is which is which is um um um in this case in this case in this case here uh here. llama.cpp list devices. So, uh here. llama.cpp list devices. So, here I have llama.cpp running inside of here I have llama.cpp running inside of here I have llama.cpp running inside of here. here. here. So, all I should be All I need to do So, all I should be All I need to do So, all I should be All I need to do here is to here is to here is to um pull the model.

um pull the model. um pull the model. And then um um it should be able to serve and my my it should be able to serve and my my it should be able to serve and my my local deployment should be able to talk local deployment should be able to talk local deployment should be able to talk to it. to it. to it. Now, there are two things that I need to Now, there are two things that I need to Now, there are two things that I need to do. do. do. I need to reboot. I'm writing in notes for myself. I'm writing in notes for myself. Reboot. And then for the for the Podman Reboot. And then for the for the Podman Reboot. And then for the for the Podman compose compose compose and I have to and I have to and I have to upgrade upgrade upgrade or configure So, yes, to be continued. I am now also So, yes, to be continued. I am now also It's now 8:00 p.m. I'm going to have It's now 8:00 p.m. I'm going to have It's now 8:00 p.m. I'm going to have dinner and relax for the rest of the dinner and relax for the rest of the dinner and relax for the rest of the night. night. night. Um how many people are joining watching Um how many people are joining watching Um how many people are joining watching right now, actually?

Uh Uh 43 people still. Very nice. 43 people still. Very nice. 43 people still. Very nice. So, my Yes, my chicken is cooking. I'm So, my Yes, my chicken is cooking. I'm So, my Yes, my chicken is cooking. I'm using an air fryer. So, it is now using an air fryer. So, it is now using an air fryer. So, it is now It It's It's actually beeping right now. It It's It's actually beeping right now. It It's It's actually beeping right now. Actually, that's That's really funny you Actually, that's That's really funny you Actually, that's That's really funny you say it. It's actually beeping now. say it. It's actually beeping now. say it. It's actually beeping now. So, my chicken is actually done right So, my chicken is actually done right So, my chicken is actually done right now. now. now. So, that means I'm going to end the So, that means I'm going to end the So, that means I'm going to end the stream. stream. stream. And then the next stream, I should have And then the next stream, I should have And then the next stream, I should have more VRAM. And then I have Podman more VRAM. And then I have Podman more VRAM. And then I have Podman Compose layered in. And I should be able Compose layered in. And I should be able Compose layered in. And I should be able to run our local our local development to run our local our local development to run our local our local development setup. setup. setup. Guys, thank you so much for joining. Guys, thank you so much for joining. Guys, thank you so much for joining. Uh, I want to mention that Uh, I want to mention that Uh, I want to mention that like the stuff that I'm doing here, this like the stuff that I'm doing here, this like the stuff that I'm doing here, this AI stuff, AI stuff, AI stuff, like like like many of you will know that I teach many of you will know that I teach many of you will know that I teach DevOps engineers, but I am DevOps engineers, but I am DevOps engineers, but I am thinking of start creating a thinking of start creating a thinking of start creating a um, second community. It's not going to um, second community. It's not going to um, second community. It's not going to be like a full career accelerator, be like a full career accelerator, be like a full career accelerator, but rather more like a community. but rather more like a community. but rather more like a community. And if you want to get on the founding And if you want to get on the founding And if you want to get on the founding members list, members list, members list, then you can check that out. I sent the then you can check that out. I sent the then you can check that out. I sent the the link in the stream here.

the link in the stream here. the link in the stream here. And this is going to be a And this is going to be a And this is going to be a um, um, um, community where I'm going to be sharing community where I'm going to be sharing community where I'm going to be sharing more knowledge about AI. And basically, more knowledge about AI. And basically, more knowledge about AI. And basically, if you want to learn AI from me, you can if you want to learn AI from me, you can if you want to learn AI from me, you can sign up from the wait on the waitlist sign up from the wait on the waitlist sign up from the wait on the waitlist there. It's for free. But if you sign there. It's for free. But if you sign there. It's for free. But if you sign up, you'll get a massive discount when I up, you'll get a massive discount when I up, you'll get a massive discount when I do launch it. I'm just trying to get a do launch it. I'm just trying to get a do launch it. I'm just trying to get a sense of how many people would be sense of how many people would be sense of how many people would be interested in it. So, check out the interested in it. So, check out the interested in it. So, check out the link. link. link. Thank you for tuning in to the stream. Thank you for tuning in to the stream. Thank you for tuning in to the stream. And guys, thank you so much. I'll see And guys, thank you so much. I'll see And guys, thank you so much. I'll see you in the next one. Have a good one.