This Is What Happens When You CRUSH An AI Video Model

Here's a thing that nobody tells you. If Here's a thing that nobody tells you. If you're running an AI video model you're running an AI video model you're running an AI video model locally, you're running it quantized. locally, you're running it quantized. locally, you're running it quantized. You just are. ComfyUI, probably the most You just are. ComfyUI, probably the most You just are. ComfyUI, probably the most popular tool for generating videos, it popular tool for generating videos, it popular tool for generating videos, it has you a GG UF version. And most has you a GG UF version. And most has you a GG UF version. And most workflows assume you grab the Q4 on day workflows assume you grab the Q4 on day workflows assume you grab the Q4 on day one and never looked back. Basically one and never looked back. Basically one and never looked back. Basically means the model weights are quantized means the model weights are quantized means the model weights are quantized down to four bits. But, nobody actually down to four bits. But, nobody actually down to four bits. But, nobody actually tells you what you give up to get this. tells you what you give up to get this. tells you what you give up to get this. So, I wanted to check it out. I took two So, I wanted to check it out. I took two So, I wanted to check it out. I took two video models and I ran each one all the video models and I ran each one all the video models and I ran each one all the way down an eight-step ladder from full way down an eight-step ladder from full way down an eight-step ladder from full precision at FP16 down all the way to precision at FP16 down all the way to precision at FP16 down all the way to two bits. Some of these look okay at two two bits. Some of these look okay at two two bits. Some of these look okay at two bits, actually. But, we'll go through bits, actually. But, we'll go through bits, actually. But, we'll go through that. WaN 2.2 14 billion parameters that. WaN 2.2 14 billion parameters that. WaN 2.2 14 billion parameters text-to-video and LTX 2.3 22 billion text-to-video and LTX 2.3 22 billion text-to-video and LTX 2.3 22 billion parameter model. This one is a bit parameter model. This one is a bit parameter model. This one is a bit spicier and that's because it generates spicier and that's because it generates spicier and that's because it generates the video and the audio together, which the video and the audio together, which the video and the audio together, which means now the picture and the sound can means now the picture and the sound can means now the picture and the sound can break separately. I ran the same prompt, break separately. I ran the same prompt, break separately. I ran the same prompt, the same seed, the same settings every the same seed, the same settings every the same seed, the same settings every single time. And as we walk down the single time. And as we walk down the single time. And as we walk down the ladder, only one thing actually changes, ladder, only one thing actually changes, ladder, only one thing actually changes, just the quantization. And all of this just the quantization. And all of this just the quantization. And all of this on a custom Linux box that I built on a custom Linux box that I built on a custom Linux box that I built running an RTX Pro 6000 Blackwell.

running an RTX Pro 6000 Blackwell. running an RTX Pro 6000 Blackwell. That's 96 gigs of RAM, which comfortably That's 96 gigs of RAM, which comfortably That's 96 gigs of RAM, which comfortably fit all the precision models that I fit all the precision models that I fit all the precision models that I have. Now, here's what I didn't expect. have. Now, here's what I didn't expect. have. Now, here's what I didn't expect. Two completely different architectures Two completely different architectures Two completely different architectures built by two completely different teams built by two completely different teams built by two completely different teams and they eventually kind of fall apart and they eventually kind of fall apart and they eventually kind of fall apart in the same way. Let's go through it. So, this is the reference. For WaN, I So, this is the reference. For WaN, I got FP16. got FP16. got FP16. >> [music] >> [music] >> [music] >> Looking good, looking good. For LTX, >> Looking good, looking good. For LTX, >> Looking good, looking good. For LTX, it's basically the same thing. It's it's basically the same thing. It's it's basically the same thing. It's BF16, just slightly different BF16, just slightly different BF16, just slightly different quantization, but they're both full quantization, but they're both full quantization, but they're both full format or full precision. And this is format or full precision. And this is format or full precision. And this is what everything else we're going to what everything else we're going to what everything else we're going to compare gets [music] compared against. compare gets [music] compared against. compare gets [music] compared against. For WaN, I ran five prompts. Some are For WaN, I ran five prompts. Some are For WaN, I ran five prompts. Some are simple objects, some are humans, some simple objects, some are humans, some simple objects, some are humans, some busy scenes. For LTX, I measured a busy scenes. For LTX, I measured a busy scenes. For LTX, I measured a couple more things, whether the words couple more things, whether the words couple more things, whether the words stayed correct stayed correct stayed correct >> Does this support the new Phantom 5090? >> Does this support the new Phantom 5090? >> Does this support the new Phantom 5090? >> and whether the audio still sounded >> and whether the audio still sounded >> and whether the audio still sounded intelligible. intelligible. intelligible. >> Did you try turning it off and on again? >> Did you try turning it off and on again? >> Did you try turning it off and on again? >> Then I scored the videos with some >> Then I scored the videos with some >> Then I scored the videos with some common benchmarks like SSIM, LPIPS, and common benchmarks like SSIM, LPIPS, and common benchmarks like SSIM, LPIPS, and prompt alignment tests. Plus the test of prompt alignment tests. Plus the test of prompt alignment tests. Plus the test of my actual eyeballs and ear holes. But, my actual eyeballs and ear holes. But, my actual eyeballs and ear holes. But, enough about my holes. Let's see the enough about my holes. Let's see the enough about my holes. Let's see the next level so we can actually get next level so we can actually get next level so we can actually get something to compare it against, shall something to compare it against, shall something to compare it against, shall we? So, here's when FP16, the whole we? So, here's when FP16, the whole we? So, here's when FP16, the whole model is 27 GB, quite chunky. And I do model is 27 GB, quite chunky. And I do model is 27 GB, quite chunky. And I do see pretty good character consistency.

see pretty good character consistency. see pretty good character consistency. Looks pretty good. Except the very Looks pretty good. Except the very Looks pretty good. Except the very beginning, two frames or so, where it beginning, two frames or so, where it beginning, two frames or so, where it looks like an old, I don't know, TV CRT looks like an old, I don't know, TV CRT looks like an old, I don't know, TV CRT screen. But, after that it balances out screen. But, after that it balances out screen. But, after that it balances out and the character remains pretty and the character remains pretty and the character remains pretty consistent throughout. The leaves look consistent throughout. The leaves look consistent throughout. The leaves look normal. Let's see LTX. normal. Let's see LTX. normal. Let's see LTX. >> Does this support the new Phantom 5090? >> Does this support the new Phantom 5090? >> Does this support the new Phantom 5090? >> Oh, yeah. 24 lanes. >> Oh, yeah. 24 lanes. >> Oh, yeah. 24 lanes. Wait. THEY LIED TO US AGAIN! THEY LIED TO US AGAIN! >> DOES THIS SUPPORT THE NEW PHANTOM 5090? >> DOES THIS SUPPORT THE NEW PHANTOM 5090? >> DOES THIS SUPPORT THE NEW PHANTOM 5090? >> That's crazy. Yeah, my prompt had him >> That's crazy. Yeah, my prompt had him >> That's crazy. Yeah, my prompt had him doing something funny, but uh that was doing something funny, but uh that was doing something funny, but uh that was just nuts. That's me and Dan at Micro just nuts. That's me and Dan at Micro just nuts. That's me and Dan at Micro Center, but our voices are just Center, but our voices are just Center, but our voices are just not at all the same. Although, not at all the same. Although, not at all the same. Although, the voices are very clear, very the voices are very clear, very the voices are very clear, very consistent. And after analyzing both the consistent. And after analyzing both the consistent. And after analyzing both the image quality and the audio quality on image quality and the audio quality on image quality and the audio quality on this one, it's kind of hard to see, but this one, it's kind of hard to see, but this one, it's kind of hard to see, but these are our baselines and I will go these are our baselines and I will go these are our baselines and I will go through these very soon here. through these very soon here. through these very soon here. >> This tiny thing has quietly become one >> This tiny thing has quietly become one >> This tiny thing has quietly become one of the most useful tools I carry. As of the most useful tools I carry. As of the most useful tools I carry. As someone who constantly is bouncing someone who constantly is bouncing someone who constantly is bouncing between meetings, conferences, and between meetings, conferences, and between meetings, conferences, and content, I need a better way to keep content, I need a better way to keep content, I need a better way to keep track of the details. The hard part for track of the details. The hard part for track of the details. The hard part for me is I cannot stay fully present, me is I cannot stay fully present, me is I cannot stay fully present, listen, and get footage, and take clean listen, and get footage, and take clean listen, and get footage, and take clean notes all at the same time. So, now I notes all at the same time. So, now I notes all at the same time. So, now I use PLAUD Note Pen S like a second brain use PLAUD Note Pen S like a second brain use PLAUD Note Pen S like a second brain for meetings, interviews, and event for meetings, interviews, and event for meetings, interviews, and event days. Most note-taking setups still days. Most note-taking setups still days. Most note-taking setups still leave me doing the worst part leave me doing the worst part leave me doing the worst part afterwards, sorting out who said what afterwards, sorting out who said what afterwards, sorting out who said what and what actually matters. This thing is and what actually matters. This thing is and what actually matters. This thing is actually really simple to use. One click actually really simple to use. One click actually really simple to use. One click starts the recording. The physical starts the recording. The physical starts the recording. The physical button helps mark key moments. And then button helps mark key moments. And then button helps mark key moments. And then it organizes everything for me after. It it organizes everything for me after. It it organizes everything for me after. It can capture up to 20 hours non-stop and

can capture up to 20 hours non-stop and can capture up to 20 hours non-stop and then turns that into transcripts with then turns that into transcripts with then turns that into transcripts with speaker labels, clean summaries, and speaker labels, clean summaries, and speaker labels, clean summaries, and actionable to-do lists instead of one actionable to-do lists instead of one actionable to-do lists instead of one giant recording that I never revisit. I giant recording that I never revisit. I giant recording that I never revisit. I also really like ask Plaud. It lets me also really like ask Plaud. It lets me also really like ask Plaud. It lets me pull up past information or think pull up past information or think pull up past information or think through next steps without digging through next steps without digging through next steps without digging through everything manually. The big win through everything manually. The big win through everything manually. The big win for me is less mental load and better for me is less mental load and better for me is less mental load and better follow through. And I'm saying that as follow through. And I'm saying that as follow through. And I'm saying that as someone who's been using Plaud since someone who's been using Plaud since someone who's been using Plaud since 2024. I went through all the different 2024. I went through all the different 2024. I went through all the different versions. Obviously, use it where versions. Obviously, use it where versions. Obviously, use it where recording is appropriate and always get recording is appropriate and always get recording is appropriate and always get permission, but as a workflow tool, this permission, but as a workflow tool, this permission, but as a workflow tool, this thing has really been useful for me. Use thing has really been useful for me. Use thing has really been useful for me. Use my code Alex15off for 15% off. There's my code Alex15off for 15% off. There's my code Alex15off for 15% off. There's also a Prime Day deal plus 30-day free also a Prime Day deal plus 30-day free also a Prime Day deal plus 30-day free return policy. Check the link in the return policy. Check the link in the return policy. Check the link in the description. Now we cut to half the bits and this is Now we cut to half the bits and this is where it gets weird. Eight bits per where it gets weird. Eight bits per where it gets weird. Eight bits per weight, but specifically the FP8 format, weight, but specifically the FP8 format, weight, but specifically the FP8 format, so floating points. Half the bytes of so floating points. Half the bytes of so floating points. Half the bytes of FP16. FP8 is natively supported on this FP16. FP8 is natively supported on this FP16. FP8 is natively supported on this GPU, so that should be a slam dunk. FP8 GPU, so that should be a slam dunk. FP8 GPU, so that should be a slam dunk. FP8 is 14 GB on disk. How's the quality? So is 14 GB on disk. How's the quality? So is 14 GB on disk. How's the quality? So here I started with the simplest thing here I started with the simplest thing here I started with the simplest thing that I rendered and this is a just glass that I rendered and this is a just glass that I rendered and this is a just glass of water just sitting there. Camera's of water just sitting there. Camera's of water just sitting there. Camera's moving around just a little bit.

moving around just a little bit. moving around just a little bit. Whatever changes you see here is just Whatever changes you see here is just Whatever changes you see here is just the model basically rendering things the model basically rendering things the model basically rendering things differently even though the seed is differently even though the seed is differently even though the seed is exactly the same. The FP16 version looks exactly the same. The FP16 version looks exactly the same. The FP16 version looks just a little bit more realistic to me just a little bit more realistic to me just a little bit more realistic to me except for what are these lines walking except for what are these lines walking except for what are these lines walking across the table? I don't know, it might across the table? I don't know, it might across the table? I don't know, it might be raining or something or something be raining or something or something be raining or something or something else is moving in the room. On the else is moving in the room. On the else is moving in the room. On the right, it's a little choppier and a right, it's a little choppier and a right, it's a little choppier and a little bit more blurry, but if it wasn't little bit more blurry, but if it wasn't little bit more blurry, but if it wasn't next to the FP16 image, it would just next to the FP16 image, it would just next to the FP16 image, it would just pass. Here's how things change. FP8 is pass. Here's how things change. FP8 is pass. Here's how things change. FP8 is already off the baseline here. This is already off the baseline here. This is already off the baseline here. This is the static detail video. 0.19 on LPIPS the static detail video. 0.19 on LPIPS the static detail video. 0.19 on LPIPS and LPIPS is basically like my eyeballs and LPIPS is basically like my eyeballs and LPIPS is basically like my eyeballs in software. If it was zero, that would in software. If it was zero, that would in software. If it was zero, that would mean it's identical and the higher the mean it's identical and the higher the mean it's identical and the higher the numbers, it means it's more different numbers, it means it's more different numbers, it means it's more different from the original. Now here's the from the original. Now here's the from the original. Now here's the surprise. The very next level down, it's surprise. The very next level down, it's surprise. The very next level down, it's not really down, it's parallel I'd say, not really down, it's parallel I'd say, not really down, it's parallel I'd say, it's Q80. So, it's still eight bits per it's Q80. So, it's still eight bits per it's Q80. So, it's still eight bits per weight, but in a different format. It's weight, but in a different format. It's weight, but in a different format. It's integer eight with K quant grouping. On integer eight with K quant grouping. On integer eight with K quant grouping. On the glass video, Q8 lands at 0.07. It's the glass video, Q8 lands at 0.07. It's the glass video, Q8 lands at 0.07. It's almost identical to full precision, almost identical to full precision, almost identical to full precision, while FP8 is about twice that much. And while FP8 is about twice that much. And while FP8 is about twice that much. And yeah, look at that. The glass actually yeah, look at that. The glass actually yeah, look at that. The glass actually looks almost identical to the FP16 in looks almost identical to the FP16 in looks almost identical to the FP16 in the video itself, according to my balls the video itself, according to my balls the video itself, according to my balls in the eye. Q8, pretty much the same in the eye. Q8, pretty much the same in the eye. Q8, pretty much the same disk size as FP8, but better fidelity.

disk size as FP8, but better fidelity. disk size as FP8, but better fidelity. And guess what? The average across all And guess what? The average across all And guess what? The average across all the five prompts here, FP8 is still the five prompts here, FP8 is still the five prompts here, FP8 is still about twice as far from baseline as Q8. about twice as far from baseline as Q8. about twice as far from baseline as Q8. So, FP8, the format with hardware So, FP8, the format with hardware So, FP8, the format with hardware acceleration that everybody said was the acceleration that everybody said was the acceleration that everybody said was the future, it drifts further and further future, it drifts further and further future, it drifts further and further from full precision than the older int from full precision than the older int from full precision than the older int eight approach. Same number of bits, the eight approach. Same number of bits, the eight approach. Same number of bits, the difference is just where it spends that difference is just where it spends that difference is just where it spends that precision. And here is the kicker, the precision. And here is the kicker, the precision. And here is the kicker, the red car, a simple motion prompt. There's red car, a simple motion prompt. There's red car, a simple motion prompt. There's FP16, at FP8, the car drives backwards. FP16, at FP8, the car drives backwards. FP16, at FP8, the car drives backwards. Not in any other quantization does this Not in any other quantization does this Not in any other quantization does this happen. Just FP8. And this time, it's happen. Just FP8. And this time, it's happen. Just FP8. And this time, it's not just slightly different rendering, not just slightly different rendering, not just slightly different rendering, it's just wrong. So, more bits is not it's just wrong. So, more bits is not it's just wrong. So, more bits is not the same as better. The format does more the same as better. The format does more the same as better. The format does more work than the bit count does. Remember work than the bit count does. Remember work than the bit count does. Remember that, because we're about to watch the that, because we're about to watch the that, because we're about to watch the same exact surprise happen in a same exact surprise happen in a same exact surprise happen in a completely different model. FP8 again, same hardware acceleration, FP8 again, same hardware acceleration, same expected slam dunk. same expected slam dunk. same expected slam dunk. >> Does this support the new Phantom 5090? >> Does this support the new Phantom 5090? >> Does this support the new Phantom 5090? >> Looks about the same, right? Watch the >> Looks about the same, right? Watch the >> Looks about the same, right? Watch the face and listen at the end.

face and listen at the end. face and listen at the end. >> Oh, yeah, 24 lanes. >> Oh, yeah, 24 lanes. >> Oh, yeah, 24 lanes. Wait. Wait. Wait. THEY LIED TO US AGAIN. THEY LIED TO US AGAIN. THEY LIED TO US AGAIN. >> [laughter] >> [laughter] >> [laughter] >> DID YOU CATCH THAT? >> DID YOU CATCH THAT? >> DID YOU CATCH THAT? WHISPER picks it up. Word error rate on WHISPER picks it up. Word error rate on WHISPER picks it up. Word error rate on FP8 is 0.18. And basically, word error FP8 is 0.18. And basically, word error FP8 is 0.18. And basically, word error rate is just how many words drifted from rate is just how many words drifted from rate is just how many words drifted from the baseline transcript. Zero is the baseline transcript. Zero is the baseline transcript. Zero is identical. This is the only quantization identical. This is the only quantization identical. This is the only quantization between BF16 and Q3KM with a non-zero between BF16 and Q3KM with a non-zero between BF16 and Q3KM with a non-zero WER, word error rate. The model added a WER, word error rate. The model added a WER, word error rate. The model added a laugh that does not exist in the laugh that does not exist in the laugh that does not exist in the baseline audio. SSIM point 87 LPIPS is baseline audio. SSIM point 87 LPIPS is baseline audio. SSIM point 87 LPIPS is at point 07, pretty good. By the way, at point 07, pretty good. By the way, at point 07, pretty good. By the way, the top three are for video quality the top three are for video quality the top three are for video quality comparisons, and the bottom two for LTX comparisons, and the bottom two for LTX comparisons, and the bottom two for LTX only are for audio. And here we have mel only are for audio. And here we have mel only are for audio. And here we have mel spectrogram MSE 26.7. spectrogram MSE 26.7. spectrogram MSE 26.7. And for video, by the way, the SSIM is And for video, by the way, the SSIM is And for video, by the way, the SSIM is the pixel match. So, one would be the pixel match. So, one would be the pixel match. So, one would be identical. Obviously, BF16 is identical identical. Obviously, BF16 is identical identical. Obviously, BF16 is identical to itself, so that's one. And here on to itself, so that's one. And here on to itself, so that's one. And here on FP8, we're sliding down to point 8. And FP8, we're sliding down to point 8. And FP8, we're sliding down to point 8. And mel MSC down here is the audio mel MSC down here is the audio mel MSC down here is the audio fingerprint. So, zero would be identical fingerprint. So, zero would be identical fingerprint. So, zero would be identical in this case. Anything higher than that in this case. Anything higher than that in this case. Anything higher than that is drift.

is drift. is drift. >> Does this support the new Phantom 5090? >> Does this support the new Phantom 5090? >> Does this support the new Phantom 5090? >> Oh, yeah, 24 lanes. >> Oh, yeah, 24 lanes. >> Oh, yeah, 24 lanes. Wait. Wait. Wait. Stealing Stealing Stealing our words again. our words again. our words again. >> So, compare that to Q80, >> So, compare that to Q80, >> So, compare that to Q80, one level down on the ladder, sort of one level down on the ladder, sort of one level down on the ladder, sort of more like more like more like horizontal. Same eight bits per weight, horizontal. Same eight bits per weight, horizontal. Same eight bits per weight, of course. Different format, though. of course. Different format, though. of course. Different format, though. Look at the difference between BF16 and Look at the difference between BF16 and Look at the difference between BF16 and Q80. There's a huge difference in the Q80. There's a huge difference in the Q80. There's a huge difference in the amount of space it takes. 46 GB versus amount of space it takes. 46 GB versus amount of space it takes. 46 GB versus 23, but they look identical. Even the 23, but they look identical. Even the 23, but they look identical. Even the motion blur in those exact moments, in motion blur in those exact moments, in motion blur in those exact moments, in those frames, is exactly the same. But those frames, is exactly the same. But those frames, is exactly the same. But FP8 FP8 FP8 not. And Q8 is better on every single not. And Q8 is better on every single not. And Q8 is better on every single metric, both video and audio. So, FP8 metric, both video and audio. So, FP8 metric, both video and audio. So, FP8 underperformed in one underperformed in one underperformed in one one one, I don't know how to say that one one, I don't know how to say that one one, I don't know how to say that properly. Don't ask me. And it also properly. Don't ask me. And it also properly. Don't ask me. And it also underperformed in LTX. Two different underperformed in LTX. Two different underperformed in LTX. Two different model architectures, two different model architectures, two different model architectures, two different teams, two different training runs. teams, two different training runs. teams, two different training runs. Weird, right? Maybe not. So, if you've Weird, right? Maybe not. So, if you've Weird, right? Maybe not. So, if you've got a choice between FP8 and Q8 for a got a choice between FP8 and Q8 for a got a choice between FP8 and Q8 for a video model right now, take Q8. I'm video model right now, take Q8. I'm video model right now, take Q8. I'm going to skip over levels between Q6 and going to skip over levels between Q6 and going to skip over levels between Q6 and Q5 because they do get a little bit Q5 because they do get a little bit Q5 because they do get a little bit worse, but there's no real big jumps worse, but there's no real big jumps worse, but there's no real big jumps here until you get to Q4. In fact, with here until you get to Q4. In fact, with here until you get to Q4. In fact, with these glasses, Q4 looks pretty good to these glasses, Q4 looks pretty good to these glasses, Q4 looks pretty good to me, too. Same thing with the car. I see me, too. Same thing with the car. I see me, too. Same thing with the car. I see slight differences with the detail of slight differences with the detail of slight differences with the detail of the car itself, but overall, the motion, the car itself, but overall, the motion, the car itself, but overall, the motion, the clarity looks pretty good. Let's

the clarity looks pretty good. Let's the clarity looks pretty good. Let's take a look at the lady here. Q6 and Q5 take a look at the lady here. Q6 and Q5 take a look at the lady here. Q6 and Q5 both 12 and 11 GB respectively on disk. both 12 and 11 GB respectively on disk. both 12 and 11 GB respectively on disk. It definitely looks like the same exact It definitely looks like the same exact It definitely looks like the same exact lady. So, what happens at Q4? Can we lady. So, what happens at Q4? Can we lady. So, what happens at Q4? Can we save more space and actually get away save more space and actually get away save more space and actually get away with it? with it? with it? >> [music] >> Q4, 9 GB on disk. This is the kind of >> Q4, 9 GB on disk. This is the kind of thing you'd run on a 24 GB consumer GPU. thing you'd run on a 24 GB consumer GPU. thing you'd run on a 24 GB consumer GPU. And this is where most of the community And this is where most of the community And this is where most of the community lives. And if you watch this in lives. And if you watch this in lives. And if you watch this in isolation, you'd probably think, "Huh, isolation, you'd probably think, "Huh, isolation, you'd probably think, "Huh, it's fine." Sure, the face is a little it's fine." Sure, the face is a little it's fine." Sure, the face is a little bit fuzzier and a little bit not as bit fuzzier and a little bit not as bit fuzzier and a little bit not as crisp, but it's passable. The forest crisp, but it's passable. The forest crisp, but it's passable. The forest still moves, but if you compare it, still moves, but if you compare it, still moves, but if you compare it, Elpips says, "We are Elpips says, "We are Elpips says, "We are at 0.34." That's further away than FP8. at 0.34." That's further away than FP8. at 0.34." That's further away than FP8. For complex scene, we're at 0.35. What For complex scene, we're at 0.35. What For complex scene, we're at 0.35. What about that glass of water? Not so bad about that glass of water? Not so bad about that glass of water? Not so bad here, 0.2. For Elpips, that character here, 0.2. For Elpips, that character here, 0.2. For Elpips, that character consistency is getting there. Let me consistency is getting there. Let me consistency is getting there. Let me show you. Notice anything different show you. Notice anything different show you. Notice anything different about her hair? And also, it kind of about her hair? And also, it kind of about her hair? And also, it kind of doesn't look like the same woman doesn't look like the same woman doesn't look like the same woman anymore. Here's the complex scene. It's anymore. Here's the complex scene. It's anymore. Here's the complex scene. It's a little bit harder to tell here. Tokyo a little bit harder to tell here. Tokyo a little bit harder to tell here. Tokyo night market, quarter disk size of FP16, night market, quarter disk size of FP16, night market, quarter disk size of FP16, but it still holds together. This is but it still holds together. This is but it still holds together. This is kind of like a sweet spot over here, and kind of like a sweet spot over here, and kind of like a sweet spot over here, and you can stop here if you don't have any you can stop here if you don't have any you can stop here if you don't have any reason to get any smaller. Red car looks reason to get any smaller. Red car looks reason to get any smaller. Red car looks fine, and do comment down below if you fine, and do comment down below if you fine, and do comment down below if you notice any weirdnesses that I didn't notice any weirdnesses that I didn't notice any weirdnesses that I didn't notice. We save a ton of space in LTX notice. We save a ton of space in LTX notice. We save a ton of space in LTX 2.3. The Q4 is only 14 GB here. There's 2.3. The Q4 is only 14 GB here. There's 2.3. The Q4 is only 14 GB here. There's some extra weirdness going on here with some extra weirdness going on here with some extra weirdness going on here with the face there and my eyes and Dan's

the face there and my eyes and Dan's the face there and my eyes and Dan's eyes, and the unexpected reaction makes eyes, and the unexpected reaction makes eyes, and the unexpected reaction makes its reappearance. its reappearance. its reappearance. But look at the mel spectrogram here. We But look at the mel spectrogram here. We But look at the mel spectrogram here. We just jumped to 46.9. just jumped to 46.9. just jumped to 46.9. One level up at Q5, it was just 9.9. So, One level up at Q5, it was just 9.9. So, One level up at Q5, it was just 9.9. So, the audio fidelity just dropped roughly the audio fidelity just dropped roughly the audio fidelity just dropped roughly five times in a single step. Now, check five times in a single step. Now, check five times in a single step. Now, check the top row, the video metrics. SSIM is the top row, the video metrics. SSIM is the top row, the video metrics. SSIM is .88, basically right where it was. LPIPS .88, basically right where it was. LPIPS .88, basically right where it was. LPIPS is .07, the picture is right about where is .07, the picture is right about where is .07, the picture is right about where it was for FP8. So, it's definitely it was for FP8. So, it's definitely it was for FP8. So, it's definitely gradually getting worse here. And Q4 gradually getting worse here. And Q4 gradually getting worse here. And Q4 shows a big change, at least in the shows a big change, at least in the shows a big change, at least in the objective measurements, but not as much objective measurements, but not as much objective measurements, but not as much as audio. So, that's finding number two, as audio. So, that's finding number two, as audio. So, that's finding number two, audio degrades before video here, even audio degrades before video here, even audio degrades before video here, even though it might not be perceived as such though it might not be perceived as such though it might not be perceived as such or might not be as noticeable as video. or might not be as noticeable as video. or might not be as noticeable as video. If you're only watching the picture, If you're only watching the picture, If you're only watching the picture, you're going to miss that. And why does you're going to miss that. And why does you're going to miss that. And why does this matter? The picture degrading is this matter? The picture degrading is this matter? The picture degrading is something you might catch on a rewatch, something you might catch on a rewatch, something you might catch on a rewatch, but the audio degrading is something the but the audio degrading is something the but the audio degrading is something the viewer hears immediately. You ever watch viewer hears immediately. You ever watch viewer hears immediately. You ever watch a YouTube video with terrible audio? I a YouTube video with terrible audio? I a YouTube video with terrible audio? I hope I hope my audio is actually decent hope I hope my audio is actually decent hope I hope my audio is actually decent here. Can't be perfect, but I try. But, here. Can't be perfect, but I try. But, here. Can't be perfect, but I try. But, you can put up with pretty bad video.

you can put up with pretty bad video. you can put up with pretty bad video. However, if you hear terrible audio, However, if you hear terrible audio, However, if you hear terrible audio, people just click away right away. Very people just click away right away. Very people just click away right away. Very different tolerance levels. So, models different tolerance levels. So, models different tolerance levels. So, models like LTX, the ones that support audio, like LTX, the ones that support audio, like LTX, the ones that support audio, have to take extra special care. have to take extra special care. have to take extra special care. >> [music] >> All right, two bits per weight. This is >> All right, two bits per weight. This is 5 GB for when. This is the bottom of the 5 GB for when. This is the bottom of the 5 GB for when. This is the bottom of the ladder and it shows. That glass is ladder and it shows. That glass is ladder and it shows. That glass is >> [laughter] >> [laughter] >> [laughter] >> uh pretty bad looking. There's even >> uh pretty bad looking. There's even >> uh pretty bad looking. There's even differences frame to frame. Look at the differences frame to frame. Look at the differences frame to frame. Look at the lady. Woo, lady. Woo, lady. Woo, that's terrible. And Q3 is very similar that's terrible. And Q3 is very similar that's terrible. And Q3 is very similar to Q4, where it does mess with the hair to Q4, where it does mess with the hair to Q4, where it does mess with the hair quite a bit from the original and quite a bit from the original and quite a bit from the original and doesn't look at all like the original. doesn't look at all like the original. doesn't look at all like the original. But, Q2 is a whole different level But, Q2 is a whole different level But, Q2 is a whole different level that's completely unusable here at this that's completely unusable here at this that's completely unusable here at this point. Look at this character point. Look at this character point. Look at this character consistency chart. L-Pips is 0.54. consistency chart. L-Pips is 0.54. consistency chart. L-Pips is 0.54. Tokyo market, the complex scene, we're Tokyo market, the complex scene, we're Tokyo market, the complex scene, we're at 0.57, even worse. Both up about 60% at 0.57, even worse. Both up about 60% at 0.57, even worse. Both up about 60% from Q4. Now, LTX at the same two bits from Q4. Now, LTX at the same two bits from Q4. Now, LTX at the same two bits does the exact same thing. Look at that does the exact same thing. Look at that does the exact same thing. Look at that first frame, looks pretty good, right?

first frame, looks pretty good, right? first frame, looks pretty good, right? This is image to video, by the way, in This is image to video, by the way, in This is image to video, by the way, in case you didn't know. Small model, 8.3 case you didn't know. Small model, 8.3 case you didn't know. Small model, 8.3 GB, GB, GB, but look what happens if I play this. but look what happens if I play this. but look what happens if I play this. >> Does this support the new Phantom 5090? >> Does this support the new Phantom 5090? >> Does this support the new Phantom 5090? Oh, yeah, 24 lanes. Oh, yeah, 24 lanes. Oh, yeah, 24 lanes. Wait. THEY LIED TO US [screaming] AGAIN! THEY LIED TO US [screaming] AGAIN! >> OH MY GOD, THAT ONE just like tugs at >> OH MY GOD, THAT ONE just like tugs at >> OH MY GOD, THAT ONE just like tugs at the heartstrings. The audio is very the heartstrings. The audio is very the heartstrings. The audio is very different. It sounds robotic until he different. It sounds robotic until he different. It sounds robotic until he screams, then it sounds like more of a screams, then it sounds like more of a screams, then it sounds like more of a somebody trying to get an Oscar. But, somebody trying to get an Oscar. But, somebody trying to get an Oscar. But, look how it terribly video quality is at look how it terribly video quality is at look how it terribly video quality is at every frame. Anything that's moving is every frame. Anything that's moving is every frame. Anything that's moving is basically completely destroyed and basically completely destroyed and basically completely destroyed and smashed. Whoa. Look at the transcript here. Word Whoa. Look at the transcript here. Word error rate here for Q3 and Q2, it went error rate here for Q3 and Q2, it went error rate here for Q3 and Q2, it went from they lied to us again to they lied from they lied to us again to they lied from they lied to us again to they lied to us again. Caps in the middle of the to us again. Caps in the middle of the to us again. Caps in the middle of the sentence. These are not the prompts, by sentence. These are not the prompts, by sentence. These are not the prompts, by the way, these are the transcripts for the way, these are the transcripts for the way, these are the transcripts for that uh whisper test that are generated that uh whisper test that are generated that uh whisper test that are generated from the video. And for LTX, it's not from the video. And for LTX, it's not from the video. And for LTX, it's not just this one skit. LTX again, this is just this one skit. LTX again, this is just this one skit. LTX again, this is BF16, FP8 looking like a very different BF16, FP8 looking like a very different BF16, FP8 looking like a very different person. Q8, Q6, Q5 person. Q8, Q6, Q5 person. Q8, Q6, Q5 look like BF16 pretty much, pretty much look like BF16 pretty much, pretty much look like BF16 pretty much, pretty much the same person. But, the leaves are the same person. But, the leaves are the same person. But, the leaves are wrong, even in BF16. Those are not maple wrong, even in BF16. Those are not maple wrong, even in BF16. Those are not maple leaves. I don't know what that is, like leaves. I don't know what that is, like leaves. I don't know what that is, like a starfish in the shape of a leaf or a a starfish in the shape of a leaf or a a starfish in the shape of a leaf or a leaf in the shape of a starfish, I leaf in the shape of a starfish, I leaf in the shape of a starfish, I guess. The shirt is also very difficult guess. The shirt is also very difficult guess. The shirt is also very difficult here. Every single one of these has a here. Every single one of these has a here. Every single one of these has a different shirt. Q5 and Q4 are kind of different shirt. Q5 and Q4 are kind of different shirt. Q5 and Q4 are kind of the same. And look what happens with Q3.

the same. And look what happens with Q3. the same. And look what happens with Q3. We have a different person here. >> Can you tell which one of me is the real >> Can you tell which one of me is the real one? one? one? >> She's not even looking at the camera. >> She's not even looking at the camera. >> She's not even looking at the camera. Forget about Q2. >> Can you tell which one of me is the real >> Can you tell which one of me is the real one? one? one? >> Yeah, it's not you. Actually, it's none >> Yeah, it's not you. Actually, it's none >> Yeah, it's not you. Actually, it's none of them, but some of these could pass of them, but some of these could pass of them, but some of these could pass for real one. >> Can you tell which one of me is the real >> Can you tell which one of me is the real one? one? one? >> Did you notice the audio difference >> Did you notice the audio difference >> Did you notice the audio difference between BF16 and Q2? Huge difference. between BF16 and Q2? Huge difference. between BF16 and Q2? Huge difference. This one sounds real. Here's Q2 again. This one sounds real. Here's Q2 again. This one sounds real. Here's Q2 again. >> Can you tell which one of me is the real >> Can you tell which one of me is the real >> Can you tell which one of me is the real one? one? one? >> Totally fake at this point, right? This >> Totally fake at this point, right? This >> Totally fake at this point, right? This is what her charts look like. Pretty is what her charts look like. Pretty is what her charts look like. Pretty gradual SSIM, L Pips also pretty gradual SSIM, L Pips also pretty gradual SSIM, L Pips also pretty gradual, but you know, it gets up there. gradual, but you know, it gets up there. gradual, but you know, it gets up there. They all have a little bump on FP8, They all have a little bump on FP8, They all have a little bump on FP8, which makes it equivalent to Q4/Q3. which makes it equivalent to Q4/Q3. which makes it equivalent to Q4/Q3. Let's check out the tech support scene. Let's check out the tech support scene. Let's check out the tech support scene. BF16, listen to the audio, too. BF16, listen to the audio, too. BF16, listen to the audio, too. >> Did you try turning it off and on again? >> NOT SURE WHAT'S GOING ON over there.

>> NOT SURE WHAT'S GOING ON over there. Something's going on. With FP8, the lady Something's going on. With FP8, the lady Something's going on. With FP8, the lady is totally different. is totally different. is totally different. >> Try turning it off and on again. >> [laughter] >> [laughter] >> LOOK AT HER. SHE'S NOT IMPRESSED. And >> LOOK AT HER. SHE'S NOT IMPRESSED. And >> LOOK AT HER. SHE'S NOT IMPRESSED. And the lady looks different in pretty much the lady looks different in pretty much the lady looks different in pretty much every single one of these. Her shirt every single one of these. Her shirt every single one of these. Her shirt color is different, the wall decorations color is different, the wall decorations color is different, the wall decorations are different, what's on the computer are different, what's on the computer are different, what's on the computer screen is different. But once we get to screen is different. But once we get to screen is different. But once we get to the 11 GB Q3, things just start melting the 11 GB Q3, things just start melting the 11 GB Q3, things just start melting down. Check this out. down. Check this out. down. Check this out. >> Did you try turning it off and on again? >> Did you try turning it off and on again? >> Did you try turning it off and on again? >> I AM THE IT DEPARTMENT. >> KIND OF LOOKS LIKE MR. Smith from The >> KIND OF LOOKS LIKE MR. Smith from The Matrix. You know, she's trying to push Matrix. You know, she's trying to push Matrix. You know, she's trying to push his buttons, and she's winning. Q2. his buttons, and she's winning. Q2. his buttons, and she's winning. Q2. >> Did you try turning it off and on again? >> Did you try turning it off and on again? >> Did you try turning it off and on again? >> I AM >> I AM >> I AM THE THE THE IT DEPARTMENT. IT DEPARTMENT. IT DEPARTMENT. >> DID YOU TRY >> DID YOU TRY >> DID YOU TRY >> WOW, that one is like way off the rails.

>> WOW, that one is like way off the rails. >> WOW, that one is like way off the rails. First of all, the lady sounds robotic. First of all, the lady sounds robotic. First of all, the lady sounds robotic. The video is just totally destroyed. So, faces are still the hardest part. So, faces are still the hardest part. Whisper for Q2 and Mel spectrogram for Whisper for Q2 and Mel spectrogram for Whisper for Q2 and Mel spectrogram for Q2 all just are off the chart terrible. Q2 all just are off the chart terrible. Q2 all just are off the chart terrible. So, by now it looks like two bits just So, by now it looks like two bits just So, by now it looks like two bits just wrecks everything. But, here's what I wrecks everything. But, here's what I wrecks everything. But, here's what I did not expect. The glass of water did not expect. The glass of water did not expect. The glass of water scene, SSIM is at .74 at two bits per scene, SSIM is at .74 at two bits per scene, SSIM is at .74 at two bits per weight. It doesn't look great when I'm weight. It doesn't look great when I'm weight. It doesn't look great when I'm examining with my balls of eye, but the examining with my balls of eye, but the examining with my balls of eye, but the numbers are saying it's okay. So, numbers are saying it's okay. So, numbers are saying it's okay. So, take these tests also with a grain of take these tests also with a grain of take these tests also with a grain of salt. I didn't come up with these tests. salt. I didn't come up with these tests. salt. I didn't come up with these tests. These are pretty much standard tests. These are pretty much standard tests. These are pretty much standard tests. So, So, So, yeah. Haven't shown you this one yet, yeah. Haven't shown you this one yet, yeah. Haven't shown you this one yet, the marble rolling down the ramp. And the marble rolling down the ramp. And the marble rolling down the ramp. And even though each one of these frames by even though each one of these frames by even though each one of these frames by themselves might look okay, the physics themselves might look okay, the physics themselves might look okay, the physics of this thing is just terrible at every of this thing is just terrible at every of this thing is just terrible at every single weight level. Even at full single weight level. Even at full single weight level. Even at full precision here, the ball just doesn't precision here, the ball just doesn't precision here, the ball just doesn't roll like a real ball. And then two roll like a real ball. And then two roll like a real ball. And then two balls get merged. That's just the model, balls get merged. That's just the model, balls get merged. That's just the model, not the quantization. It's just wrong.

not the quantization. It's just wrong. not the quantization. It's just wrong. Not necessarily worse. Although, at Q2, Not necessarily worse. Although, at Q2, Not necessarily worse. Although, at Q2, you could pretty much say it's worse. you could pretty much say it's worse. you could pretty much say it's worse. So, finding number three I'd say is that So, finding number three I'd say is that So, finding number three I'd say is that there is no universal cliff. And that's there is no universal cliff. And that's there is no universal cliff. And that's true in both models. If the thing you're true in both models. If the thing you're true in both models. If the thing you're making is a single hard object in making is a single hard object in making is a single hard object in motion, you can probably ride this all motion, you can probably ride this all motion, you can probably ride this all the way down. But, if you're making the way down. But, if you're making the way down. But, if you're making humans in it, the floor is a lot higher. humans in it, the floor is a lot higher. humans in it, the floor is a lot higher. Bottom line, run Q4 unless you've got a Bottom line, run Q4 unless you've got a Bottom line, run Q4 unless you've got a reason not to. And the sweet spot reason not to. And the sweet spot reason not to. And the sweet spot doesn't move when you switch model doesn't move when you switch model doesn't move when you switch model architectures. But, also if your output architectures. But, also if your output architectures. But, also if your output has audio, listen to it at Q4. The has audio, listen to it at Q4. The has audio, listen to it at Q4. The picture will trick you, but the voice picture will trick you, but the voice picture will trick you, but the voice will not. And if you remember one thing will not. And if you remember one thing will not. And if you remember one thing from this whole video, let it be this. from this whole video, let it be this. from this whole video, let it be this. Format matters more than bit count. The Format matters more than bit count. The Format matters more than bit count. The full ladders are linked down below, full ladders are linked down below, full ladders are linked down below, every clip and every chart. And thanks every clip and every chart. And thanks every clip and every chart. And thanks to this rig, I was able to generate to this rig, I was able to generate to this rig, I was able to generate these videos pretty fast. If you want to these videos pretty fast. If you want to these videos pretty fast. If you want to know whether this 96 GB RTX Pro 6000 rig know whether this 96 GB RTX Pro 6000 rig know whether this 96 GB RTX Pro 6000 rig was actually worth it, that video is was actually worth it, that video is was actually worth it, that video is over here. Thanks for watching, [music] over here. Thanks for watching, [music] over here. Thanks for watching, [music] and I'll see you next time.