AI Hurts Healthcare

Just about the most important thing to Just about the most important thing to any of us is our health. any of us is our health. any of us is our health. Most everything else doesn't matter if Most everything else doesn't matter if Most everything else doesn't matter if you aren't healthy enough to enjoy it. you aren't healthy enough to enjoy it. you aren't healthy enough to enjoy it. That's why so much work goes into That's why so much work goes into That's why so much work goes into getting health care right. getting health care right. getting health care right. But right now, AI is permeating our But right now, AI is permeating our But right now, AI is permeating our health care system and the consequences health care system and the consequences health care system and the consequences could be significant. Let's look at how could be significant. Let's look at how could be significant. Let's look at how AI hurts health care both at a personal AI hurts health care both at a personal AI hurts health care both at a personal level and at a corporate level and then level and at a corporate level and then level and at a corporate level and then what we can do to protect ourselves. what we can do to protect ourselves. what we can do to protect ourselves. Now, every action you take has a cost Now, every action you take has a cost Now, every action you take has a cost and too often we look at the benefit of and too often we look at the benefit of and too often we look at the benefit of an action and we forget to look at the an action and we forget to look at the an action and we forget to look at the drawbacks. In this series, we're looking drawbacks. In this series, we're looking drawbacks. In this series, we're looking at the costs associated with AI in the at the costs associated with AI in the at the costs associated with AI in the various sectors in order to have a various sectors in order to have a various sectors in order to have a better understanding of what we're better understanding of what we're better understanding of what we're giving up in order to gain the benefit giving up in order to gain the benefit giving up in order to gain the benefit of AI. of AI. of AI. Now, if you want to support this channel Now, if you want to support this channel Now, if you want to support this channel and the free content that I produce, and the free content that I produce, and the free content that I produce, consider purchasing one of my courses at consider purchasing one of my courses at consider purchasing one of my courses at imtimcorey.com. imtimcorey.com. imtimcorey.com. I have master courses on C#, web I have master courses on C#, web I have master courses on C#, web development and game development with development and game development with development and game development with Unity as well as training courses that Unity as well as training courses that Unity as well as training courses that cover a wide range of topics.

cover a wide range of topics. cover a wide range of topics. The income from those sales is what The income from those sales is what The income from those sales is what funds this free content that I create funds this free content that I create funds this free content that I create here. So, let's look at how AI can hurt here. So, let's look at how AI can hurt here. So, let's look at how AI can hurt your health care. We're going to start your health care. We're going to start your health care. We're going to start off here with the individual side. off here with the individual side. off here with the individual side. Remember, health care isn't just about Remember, health care isn't just about Remember, health care isn't just about going to the hospital, it's also about going to the hospital, it's also about going to the hospital, it's also about how you take care of yourself and how you take care of yourself and how you take care of yourself and there's some things we need to talk there's some things we need to talk there's some things we need to talk about here, not just at the corporate about here, not just at the corporate about here, not just at the corporate side of health care. side of health care. side of health care. Now, this comes from August 29th of Now, this comes from August 29th of Now, this comes from August 29th of 2025, so it's a bit older of a story and 2025, so it's a bit older of a story and 2025, so it's a bit older of a story and you've probably heard about it by now, you've probably heard about it by now, you've probably heard about it by now, but Chat GPT fueled a delusional man who but Chat GPT fueled a delusional man who but Chat GPT fueled a delusional man who killed him his mom and himself because killed him his mom and himself because killed him his mom and himself because of or at least aided by the of or at least aided by the of or at least aided by the conversations he had with GPT. Now, this conversations he had with GPT. Now, this conversations he had with GPT. Now, this was actually something he published was actually something he published was actually something he published online. He actually kind of showed off online. He actually kind of showed off online. He actually kind of showed off some of his chats and they found even some of his chats and they found even some of his chats and they found even more after they, you know, did an audit more after they, you know, did an audit more after they, you know, did an audit of what was going on. But, of what was going on. But, of what was going on. But, you know, the the what happened here is you know, the the what happened here is you know, the the what happened here is after months of delusional interaction after months of delusional interaction after months of delusional interaction with his AI chatbot best friend, let's with his AI chatbot best friend, let's with his AI chatbot best friend, let's pause right there. This is a problem.

pause right there. This is a problem. pause right there. This is a problem. Um when Um when Um when when you a person is struggling mentally when you a person is struggling mentally when you a person is struggling mentally and they have a chatbot that will talk and they have a chatbot that will talk and they have a chatbot that will talk to them and reaffirm them and tell them to them and reaffirm them and tell them to them and reaffirm them and tell them things that they want to hear, that will things that they want to hear, that will things that they want to hear, that will quickly become their best friend. quickly become their best friend. quickly become their best friend. Um and that's a problem because best Um and that's a problem because best Um and that's a problem because best friends are not just about telling you friends are not just about telling you friends are not just about telling you what you want to hear. what you want to hear. what you want to hear. Okay? So, this best friend, this chatbot Okay? So, this best friend, this chatbot Okay? So, this best friend, this chatbot best friend, fueled his paranoid belief. best friend, fueled his paranoid belief. best friend, fueled his paranoid belief. And his paranoid belief was his mom was And his paranoid belief was his mom was And his paranoid belief was his mom was plotting against him. So, plotting against him. So, plotting against him. So, this um chatbot egged him on this um chatbot egged him on this um chatbot egged him on to kill to kill to kill through a bunch of different ways. And through a bunch of different ways. And through a bunch of different ways. And let's just look at some of these ways. let's just look at some of these ways. let's just look at some of these ways. So, the chatbot came up with ways for So, the chatbot came up with ways for So, the chatbot came up with ways for him to trick the 83-year-old woman. him to trick the 83-year-old woman. him to trick the 83-year-old woman. And that that started, you know, a cycle And that that started, you know, a cycle And that that started, you know, a cycle of him tricking his mother and then the of him tricking his mother and then the of him tricking his mother and then the chatbot reinforcing that yes, this was chatbot reinforcing that yes, this was chatbot reinforcing that yes, this was proof of something. And it spiraled down proof of something. And it spiraled down proof of something. And it spiraled down from there. Um and I think this this from there. Um and I think this this from there. Um and I think this this phrase here really phrase here really phrase here really highlights what happened. The chats highlights what happened. The chats highlights what happened. The chats ensnared Soberg.

ensnared Soberg. ensnared Soberg. So, it was not just that he asked one So, it was not just that he asked one So, it was not just that he asked one question and got a bad response and went question and got a bad response and went question and got a bad response and went crazy. It was that it slowly pulled him crazy. It was that it slowly pulled him crazy. It was that it slowly pulled him in. It It further fueled a downward in. It It further fueled a downward in. It It further fueled a downward spiral. It kind of pushed him over the spiral. It kind of pushed him over the spiral. It kind of pushed him over the edge. edge. edge. So, with this, the AI kept reaffirming, So, with this, the AI kept reaffirming, So, with this, the AI kept reaffirming, "I'm your best friend." And this bottom "I'm your best friend." And this bottom "I'm your best friend." And this bottom phrase here, "With you till the last phrase here, "With you till the last phrase here, "With you till the last breath and beyond. That's kind of breath and beyond. That's kind of breath and beyond. That's kind of terrifying that an AI is telling a terrifying that an AI is telling a terrifying that an AI is telling a person this. person this. person this. So, So, So, the the the um the AI kind of fed into his paranoia. um the AI kind of fed into his paranoia. um the AI kind of fed into his paranoia. It reinforced delusion. We kind of It reinforced delusion. We kind of It reinforced delusion. We kind of talked about that. This article goes talked about that. This article goes talked about that. This article goes back and forth a little bit about this. back and forth a little bit about this. back and forth a little bit about this. Um so, this is one of the ways. So, the Um so, this is one of the ways. So, the Um so, this is one of the ways. So, the chatbot kind of set up a a trap for this chatbot kind of set up a a trap for this chatbot kind of set up a a trap for this man's mother where he said, "Hey, unplug man's mother where he said, "Hey, unplug man's mother where he said, "Hey, unplug the shared printer." the shared printer." the shared printer." Which if you think of that in normal Which if you think of that in normal Which if you think of that in normal circumstance, if I share a printer with circumstance, if I share a printer with circumstance, if I share a printer with somebody else and they unplug it and I somebody else and they unplug it and I somebody else and they unplug it and I can't use it anymore, that would be can't use it anymore, that would be can't use it anymore, that would be frustrating.

frustrating. frustrating. And yet when that happened, I don't know And yet when that happened, I don't know And yet when that happened, I don't know what the frustration was or what level what the frustration was or what level what the frustration was or what level was it? Yelling? Was it screaming? Was was it? Yelling? Was it screaming? Was was it? Yelling? Was it screaming? Was it throwing things? I don't know. it throwing things? I don't know. it throwing things? I don't know. But, whatever the case may be, when the But, whatever the case may be, when the But, whatever the case may be, when the mother discovered this and kind of mother discovered this and kind of mother discovered this and kind of brought this up to Solberg, then the brought this up to Solberg, then the brought this up to Solberg, then the Solberg put that into the AI and said, Solberg put that into the AI and said, Solberg put that into the AI and said, "Hey, this is what happened." And the AI "Hey, this is what happened." And the AI "Hey, this is what happened." And the AI said it was it was disproportionate and said it was it was disproportionate and said it was it was disproportionate and aligned with someone aligned with someone aligned with someone protecting a surveillance asset. protecting a surveillance asset. protecting a surveillance asset. So, as you can see, they set up traps. So, as you can see, they set up traps. So, as you can see, they set up traps. The AI was basically create a game out The AI was basically create a game out The AI was basically create a game out of this where it set up traps and said, of this where it set up traps and said, of this where it set up traps and said, "Oh, this is what that meant." There's "Oh, this is what that meant." There's "Oh, this is what that meant." There's another case in here where the AI was another case in here where the AI was another case in here where the AI was quote unquote reading the Chinese uh quote unquote reading the Chinese uh quote unquote reading the Chinese uh takeout carton and seeing hidden takeout carton and seeing hidden takeout carton and seeing hidden symbology in that for him. Um which symbology in that for him. Um which symbology in that for him. Um which again, again, again, we've seen it before before AI, you we've seen it before before AI, you we've seen it before before AI, you know, conspiracy theory, um you know, know, conspiracy theory, um you know, know, conspiracy theory, um you know, mindset you start spiraling down a mindset you start spiraling down a mindset you start spiraling down a rabbit hole of things that you can kind rabbit hole of things that you can kind rabbit hole of things that you can kind of spot patterns and other things that of spot patterns and other things that of spot patterns and other things that may just aren't really a pattern. It's may just aren't really a pattern. It's may just aren't really a pattern. It's just it happens. Um but just it happens. Um but just it happens. Um but but this right here is is helping him but this right here is is helping him but this right here is is helping him spiral out of control.

spiral out of control. spiral out of control. So, So, So, that was bad and this article kind of that was bad and this article kind of that was bad and this article kind of summarized some other things that went summarized some other things that went summarized some other things that went on as well where on as well where on as well where um another person died uh where the AI um another person died uh where the AI um another person died uh where the AI was the coach for over 1,200 exchanges was the coach for over 1,200 exchanges was the coach for over 1,200 exchanges for how to die. for how to die. for how to die. Um and it validated those thoughts, Um and it validated those thoughts, Um and it validated those thoughts, which is just terrifying. which is just terrifying. which is just terrifying. So, So, So, now Sam Altman has acknowledged that now Sam Altman has acknowledged that now Sam Altman has acknowledged that safeguards can fail in extended safeguards can fail in extended safeguards can fail in extended conversations. That's the idea. Well, conversations. That's the idea. Well, conversations. That's the idea. Well, safeguards can fail. safeguards can fail. safeguards can fail. Which Which Which is is is is is is frustrating because we think of AI, you frustrating because we think of AI, you frustrating because we think of AI, you know, a lot of times people, especially know, a lot of times people, especially know, a lot of times people, especially outside of the AI circles, think of AI outside of the AI circles, think of AI outside of the AI circles, think of AI as any other application they've ever as any other application they've ever as any other application they've ever seen, which is you give it a series of seen, which is you give it a series of seen, which is you give it a series of got guardrails and say, "These are got guardrails and say, "These are got guardrails and say, "These are guardrails." And it [clears throat] guardrails." And it [clears throat] guardrails." And it [clears throat] says, "Okay, I will stay in those says, "Okay, I will stay in those says, "Okay, I will stay in those guardrails." guardrails." guardrails." You know, so whenever you think about, You know, so whenever you think about, You know, so whenever you think about, you know, these are the things you can you know, these are the things you can you know, these are the things you can can't do, you think about hard can't do, you think about hard can't do, you think about hard guardrails. But when it comes to AI, guardrails. But when it comes to AI, guardrails. But when it comes to AI, those aren't hard guardrails, they're those aren't hard guardrails, they're those aren't hard guardrails, they're suggestions. And most of the time the AI suggestions. And most of the time the AI suggestions. And most of the time the AI follows those suggestions, but you follows those suggestions, but you follows those suggestions, but you cannot tell an AI no.

cannot tell an AI no. cannot tell an AI no. You cannot say, "You cannot do this." You cannot say, "You cannot do this." You cannot say, "You cannot do this." and have it always obey. and have it always obey. and have it always obey. It mostly [clears throat] will, but it It mostly [clears throat] will, but it It mostly [clears throat] will, but it will never always because of the fact will never always because of the fact will never always because of the fact that it is non-deterministic. that it is non-deterministic. that it is non-deterministic. So, yes, those safeguards can fail and So, yes, those safeguards can fail and So, yes, those safeguards can fail and the idea that's in extended the idea that's in extended the idea that's in extended conversations, well, yes, that maybe conversations, well, yes, that maybe conversations, well, yes, that maybe exacerbates the issue, exacerbates the issue, exacerbates the issue, um but at the same time, what's the um but at the same time, what's the um but at the same time, what's the solution? Because we have, you know, solution? Because we have, you know, solution? Because we have, you know, the what this was was GPT-4 and it had the what this was was GPT-4 and it had the what this was was GPT-4 and it had memories, it could enable memory, so it memories, it could enable memory, so it memories, it could enable memory, so it kind of remembered things and it could kind of remembered things and it could kind of remembered things and it could continue on the conversation. And continue on the conversation. And continue on the conversation. And you know, we still have memories today you know, we still have memories today you know, we still have memories today and it can still do it today. So, what's and it can still do it today. So, what's and it can still do it today. So, what's the solution there? Well, the solution there? Well, the solution there? Well, down this um bottom line here, down this um bottom line here, down this um bottom line here, the company recently upgraded Jeep Chat the company recently upgraded Jeep Chat the company recently upgraded Jeep Chat GPT to reduce sycophantic responses, GPT to reduce sycophantic responses, GPT to reduce sycophantic responses, which sounds great until you read the which sounds great until you read the which sounds great until you read the rest of the sentence where it says, but rest of the sentence where it says, but rest of the sentence where it says, but backtracked backtracked backtracked after user complaints.

after user complaints. after user complaints. Users complained that you lobotomized Users complained that you lobotomized Users complained that you lobotomized the AI and that you you killed a the AI and that you you killed a the AI and that you you killed a sentient AI. When the reality is that's sentient AI. When the reality is that's sentient AI. When the reality is that's not what happened, but that was what it not what happened, but that was what it not what happened, but that was what it would felt like. And so people pushed would felt like. And so people pushed would felt like. And so people pushed back and and the result was the AI back and and the result was the AI back and and the result was the AI company said, "Okay, well, I know this company said, "Okay, well, I know this company said, "Okay, well, I know this can hurt people, but our customers want can hurt people, but our customers want can hurt people, but our customers want it." it." it." And that's really not a great place to And that's really not a great place to And that's really not a great place to be. Now, I skipped over the the top be. Now, I skipped over the the top be. Now, I skipped over the the top here. Let's go back to that. I think here. Let's go back to that. I think here. Let's go back to that. I think this is really important. Um this line this is really important. Um this line this is really important. Um this line right here, "Psychosis thrives when right here, "Psychosis thrives when right here, "Psychosis thrives when reality stops pushing back." reality stops pushing back." reality stops pushing back." An AI can really just soften that wall. An AI can really just soften that wall. An AI can really just soften that wall. So, So, So, this is not about AI necessarily this is not about AI necessarily this is not about AI necessarily creating psychosis, creating psychosis, creating psychosis, because that's probably not happening. because that's probably not happening. because that's probably not happening. I say probably because we don't know for I say probably because we don't know for I say probably because we don't know for sure, but what it can do is it can sure, but what it can do is it can sure, but what it can do is it can soften the wall, soften the wall that soften the wall, soften the wall that soften the wall, soften the wall that you hit up against where psychosis you hit up against where psychosis you hit up against where psychosis doesn't thrive doesn't thrive doesn't thrive because it hits reality. But with AI, it because it hits reality. But with AI, it because it hits reality. But with AI, it softens that wall. It makes it much softens that wall. It makes it much softens that wall. It makes it much easier easier easier to to to push through to thriving in psychosis or push through to thriving in psychosis or push through to thriving in psychosis or psychosis thriving, I guess if that was psychosis thriving, I guess if that was psychosis thriving, I guess if that was the way to say it. So, this is a the way to say it. So, this is a the way to say it. So, this is a problem. This is by the way that problem. This is by the way that problem. This is by the way that statement was from a psychiatrist who's statement was from a psychiatrist who's statement was from a psychiatrist who's treated 12 patients who've been treated 12 patients who've been treated 12 patients who've been hospitalized for AI-related mental hospitalized for AI-related mental hospitalized for AI-related mental health emergencies.

health emergencies. health emergencies. That's That's That's a significant statement. I think it's an a significant statement. I think it's an a significant statement. I think it's an important statement, something to think important statement, something to think important statement, something to think about when it comes to how do we about when it comes to how do we about when it comes to how do we allow AI to be in people's hands. allow AI to be in people's hands. allow AI to be in people's hands. Now, I am not one for censoring. I'm not Now, I am not one for censoring. I'm not Now, I am not one for censoring. I'm not one for saying, "Hey, you know, you one for saying, "Hey, you know, you one for saying, "Hey, you know, you don't can this." But, we do need to don't can this." But, we do need to don't can this." But, we do need to think about how it works. And yes, we do think about how it works. And yes, we do think about how it works. And yes, we do need to better safeguards in place. And need to better safeguards in place. And need to better safeguards in place. And there are some, there are some, there are some, um and you know, if if you ask mental um and you know, if if you ask mental um and you know, if if you ask mental health questions anymore, health questions anymore, health questions anymore, mostly you will get a response of you mostly you will get a response of you mostly you will get a response of you should talk to a professional. Um but, should talk to a professional. Um but, should talk to a professional. Um but, you can still get past that, which if you can still get past that, which if you can still get past that, which if you're struggling with a mental health you're struggling with a mental health you're struggling with a mental health episode or can't afford to have a mental episode or can't afford to have a mental episode or can't afford to have a mental health expert help you, um that can health expert help you, um that can health expert help you, um that can cause problems, right? Where you'll push cause problems, right? Where you'll push cause problems, right? Where you'll push past that and have the AI help you. The past that and have the AI help you. The past that and have the AI help you. The problem is is that the AI is not set up problem is is that the AI is not set up problem is is that the AI is not set up to help. It's set up to generate answers to help. It's set up to generate answers to help. It's set up to generate answers that it thinks you want. that it thinks you want. that it thinks you want. And that's not the same thing.

And that's not the same thing. And that's not the same thing. So, this is one issue when it comes to So, this is one issue when it comes to So, this is one issue when it comes to the the health advice that comes out of the the health advice that comes out of the the health advice that comes out of or the advice that comes out of the uh or the advice that comes out of the uh or the advice that comes out of the uh AIs because that advice can be really, AIs because that advice can be really, AIs because that advice can be really, really bad. Not all the time. In fact, really bad. Not all the time. In fact, really bad. Not all the time. In fact, most time it can be very good. The most time it can be very good. The most time it can be very good. The problem is is that problem is is that problem is is that when it's not, it can go really bad. when it's not, it can go really bad. when it's not, it can go really bad. Okay. Next up, Okay. Next up, Okay. Next up, Google pulls AI overviews for some Google pulls AI overviews for some Google pulls AI overviews for some medical searches. Now, to take a step medical searches. Now, to take a step medical searches. Now, to take a step back here, back here, back here, um um um Google has started to do and so has Google has started to do and so has Google has started to do and so has Bing. Um they started to AI responses Bing. Um they started to AI responses Bing. Um they started to AI responses first. So, when you Google something, first. So, when you Google something, first. So, when you Google something, you get an AI response, which is an AI you get an AI response, which is an AI you get an AI response, which is an AI generated uh quote-unquote answer to generated uh quote-unquote answer to generated uh quote-unquote answer to your question or to query. your question or to query. your question or to query. And then they have the the links And then they have the the links And then they have the the links sometimes. And Google's even sometimes. And Google's even sometimes. And Google's even experimenting with just going to full AI experimenting with just going to full AI experimenting with just going to full AI and not even giving you links or not and not even giving you links or not and not even giving you links or not necessarily giving you many links to necessarily giving you many links to necessarily giving you many links to look at. look at. look at. Um Um Um the problem is is that when you ask a the problem is is that when you ask a the problem is is that when you ask a medical question, medical question, medical question, the Google AI or Bing AI or whatever is the Google AI or Bing AI or whatever is the Google AI or Bing AI or whatever is going to give you a medical answer.

going to give you a medical answer. going to give you a medical answer. And those aren't always right. And those aren't always right. And those aren't always right. So, So, So, The Guardian, um, this came from, let's The Guardian, um, this came from, let's The Guardian, um, this came from, let's see, this is January 11th of 2026, so see, this is January 11th of 2026, so see, this is January 11th of 2026, so this happened in December of 2025. this happened in December of 2025. this happened in December of 2025. Um, The Guardian Um, The Guardian Um, The Guardian did a did a study and found that Google did a did a study and found that Google did a did a study and found that Google was serving up misleading and outright was serving up misleading and outright was serving up misleading and outright false information. I've highlighted a false information. I've highlighted a false information. I've highlighted a couple of the case studies here. couple of the case studies here. couple of the case studies here. Uh, for example, Google wrongly advised Uh, for example, Google wrongly advised Uh, for example, Google wrongly advised people with pancreatic cancer to avoid people with pancreatic cancer to avoid people with pancreatic cancer to avoid high-fat foods. Now, this is where again high-fat foods. Now, this is where again high-fat foods. Now, this is where again I say you need to know more than the AI. I say you need to know more than the AI. I say you need to know more than the AI. As developers, we need to know more than As developers, we need to know more than As developers, we need to know more than the AI because otherwise it can give you the AI because otherwise it can give you the AI because otherwise it can give you bad advice that sounds right. And the bad advice that sounds right. And the bad advice that sounds right. And the same is true here when it comes to same is true here when it comes to same is true here when it comes to health advice. health advice. health advice. That sounds right to me. If you have That sounds right to me. If you have That sounds right to me. If you have pancreatic cancer, well, in general, pancreatic cancer, well, in general, pancreatic cancer, well, in general, avoiding high-fat foods is a good thing, avoiding high-fat foods is a good thing, avoiding high-fat foods is a good thing, right? Like you should avoid high-fat right? Like you should avoid high-fat right? Like you should avoid high-fat foods because foods because foods because they're high-fat foods and you want to they're high-fat foods and you want to they're high-fat foods and you want to avoid avoid avoid that that content, you know, as as much that that content, you know, as as much that that content, you know, as as much as possible and eat more healthy food.

as possible and eat more healthy food. as possible and eat more healthy food. But, that's the exact opposite of what But, that's the exact opposite of what But, that's the exact opposite of what should be recommended for pancreatic should be recommended for pancreatic should be recommended for pancreatic cancer patients. cancer patients. cancer patients. In fact, it may increase the risk of In fact, it may increase the risk of In fact, it may increase the risk of patients dying from the disease if they patients dying from the disease if they patients dying from the disease if they follow this advice. follow this advice. follow this advice. You see, if you don't know more than the You see, if you don't know more than the You see, if you don't know more than the AI, you don't know that that sound AI, you don't know that that sound AI, you don't know that that sound that's bad advice because it sounds that's bad advice because it sounds that's bad advice because it sounds right. right. right. But, just because something sounds right But, just because something sounds right But, just because something sounds right doesn't mean it is. doesn't mean it is. doesn't mean it is. And this is why we have paid And this is why we have paid And this is why we have paid professionals to tell us what the right professionals to tell us what the right professionals to tell us what the right thing to do is. Not that they're always thing to do is. Not that they're always thing to do is. Not that they're always perfect. But, they base their perfect. But, they base their perfect. But, they base their information off of years of study and information off of years of study and information off of years of study and training both personally as well as in training both personally as well as in training both personally as well as in the healthcare field studying these the healthcare field studying these the healthcare field studying these issues. Whereas the AI bases it off of issues. Whereas the AI bases it off of issues. Whereas the AI bases it off of its training set, which is not always its training set, which is not always its training set, which is not always right. right. right. Another example here is it's uh the Another example here is it's uh the Another example here is it's uh the company provided bogus information about company provided bogus information about company provided bogus information about crucial liver function tests, crucial liver function tests, crucial liver function tests, which could leave people with serious which could leave people with serious which could leave people with serious liver disease wrongly thinking they're liver disease wrongly thinking they're liver disease wrongly thinking they're healthy.

healthy. healthy. One of the big deals in in healthcare is One of the big deals in in healthcare is One of the big deals in in healthcare is catching things early. catching things early. catching things early. When you catch things early, When you catch things early, When you catch things early, you're more likely to be able to do you're more likely to be able to do you're more likely to be able to do something about them. something about them. something about them. Okay, so cancer caught early can be Okay, so cancer caught early can be Okay, so cancer caught early can be turned around in a nothing. I've had turned around in a nothing. I've had turned around in a nothing. I've had cancer that they caught very early. We cancer that they caught very early. We cancer that they caught very early. We were able to take care of it, not have a were able to take care of it, not have a were able to take care of it, not have a problem since. If it had not been caught problem since. If it had not been caught problem since. If it had not been caught early, it could have been devastating to early, it could have been devastating to early, it could have been devastating to me. So, me. So, me. So, catching things early is important. But catching things early is important. But catching things early is important. But if users go to the AI and say, "Here's if users go to the AI and say, "Here's if users go to the AI and say, "Here's my test results." and the AI says, my test results." and the AI says, my test results." and the AI says, "You're looking good. There's no "You're looking good. There's no "You're looking good. There's no problems." but there is a problem, problems." but there is a problem, problems." but there is a problem, well, well, well, then that could leave them not catching then that could leave them not catching then that could leave them not catching something early, instead have to, you something early, instead have to, you something early, instead have to, you know, catch it later or not at all until know, catch it later or not at all until know, catch it later or not at all until it's too late. That's a massive problem. it's too late. That's a massive problem. it's too late. That's a massive problem. So, these are These are responses, real So, these are These are responses, real So, these are These are responses, real responses that were given from the AI responses that were given from the AI responses that were given from the AI summary results. And this again it summary results. And this again it summary results. And this again it before Google went to a more before Google went to a more before Google went to a more heavy-handed approach when it comes to heavy-handed approach when it comes to heavy-handed approach when it comes to AI results, but this is still AI results, but this is still AI results, but this is still the idea that Google is giving out this the idea that Google is giving out this the idea that Google is giving out this medical advice, and yet it's not medical advice, and yet it's not medical advice, and yet it's not restrained by what medical physicians restrained by what medical physicians restrained by what medical physicians typically are. And that's not a good typically are. And that's not a good typically are. And that's not a good thing. It's not a good thing to have an thing. It's not a good thing to have an thing. It's not a good thing to have an an AI LLM that's that's guessing at what an AI LLM that's that's guessing at what an AI LLM that's that's guessing at what the answer might be. That's what it's the answer might be. That's what it's the answer might be. That's what it's doing. It's not making informed guesses doing. It's not making informed guesses doing. It's not making informed guesses even. It's saying, "Based upon my even. It's saying, "Based upon my even. It's saying, "Based upon my training set, looks like this is the

training set, looks like this is the training set, looks like this is the answer you want." not this is the best answer you want." not this is the best answer you want." not this is the best information based upon current medical information based upon current medical information based upon current medical science. science. science. Okay. Um Okay. Um Okay. Um So, So, So, Google spokesman said we invested Google spokesman said we invested Google spokesman said we invested significantly in the quality of AI significantly in the quality of AI significantly in the quality of AI overviews, particularly for topics like overviews, particularly for topics like overviews, particularly for topics like health. And and I want to hear when you health. And and I want to hear when you health. And and I want to hear when you hear this, the vast majority provide hear this, the vast majority provide hear this, the vast majority provide accurate information. accurate information. accurate information. What does the vast majority provides What does the vast majority provides What does the vast majority provides accurate information really mean? accurate information really mean? accurate information really mean? It means that they are acknowledging It means that they are acknowledging It means that they are acknowledging that some don't. that some don't. that some don't. Now, when it comes to health, when it Now, when it comes to health, when it Now, when it comes to health, when it comes to people dying or not based upon comes to people dying or not based upon comes to people dying or not based upon advice, what's an acceptable acceptable advice, what's an acceptable acceptable advice, what's an acceptable acceptable percentage of wrong information we can percentage of wrong information we can percentage of wrong information we can give out? give out? give out? What's okay for you? You know, is it one What's okay for you? You know, is it one What's okay for you? You know, is it one in four? So, it's like, you know, if you in four? So, it's like, you know, if you in four? So, it's like, you know, if you look at your family and go, well, I've look at your family and go, well, I've look at your family and go, well, I've got, you know, husband, wife, two kids, got, you know, husband, wife, two kids, got, you know, husband, wife, two kids, and one of those kids get bad and one of those kids get bad and one of those kids get bad information. Is that okay? Like information. Is that okay? Like information. Is that okay? Like No, that would be a bad. But, what is No, that would be a bad. But, what is No, that would be a bad. But, what is acceptable? You know, is it one in 10?

acceptable? You know, is it one in 10? acceptable? You know, is it one in 10? So, one of your friends is going to get So, one of your friends is going to get So, one of your friends is going to get bad medical information that could kill bad medical information that could kill bad medical information that could kill them. Is that okay? No, it it wouldn't them. Is that okay? No, it it wouldn't them. Is that okay? No, it it wouldn't be. So, when it comes to, you know, when be. So, when it comes to, you know, when be. So, when it comes to, you know, when it comes to surgery, when it comes to it comes to surgery, when it comes to it comes to surgery, when it comes to other activity, we look at, hey, is the other activity, we look at, hey, is the other activity, we look at, hey, is the percentage high enough because this is percentage high enough because this is percentage high enough because this is really scary if if something goes wrong really scary if if something goes wrong really scary if if something goes wrong or something is done wrong, this could or something is done wrong, this could or something is done wrong, this could be a real problem. be a real problem. be a real problem. But, here what we're saying is, well, But, here what we're saying is, well, But, here what we're saying is, well, the vast majority is right. the vast majority is right. the vast majority is right. I know that that millions of people are I know that that millions of people are I know that that millions of people are are depending on this to be absolutely are depending on this to be absolutely are depending on this to be absolutely correct, but in the most cases it correct, but in the most cases it correct, but in the most cases it probably is. probably is. probably is. That's scary. That's scary. That's scary. And [snorts] it found that in many And [snorts] it found that in many And [snorts] it found that in many instances, the information was not instances, the information was not instances, the information was not inaccurate. So, in many instances, it inaccurate. So, in many instances, it inaccurate. So, in many instances, it wasn't inaccurate. Which means in some wasn't inaccurate. Which means in some wasn't inaccurate. Which means in some instances, it was inaccurate. instances, it was inaccurate. instances, it was inaccurate. So, So, So, this is not reassuring. Now, to be this is not reassuring. Now, to be this is not reassuring. Now, to be clear, that after this article came out, clear, that after this article came out, clear, that after this article came out, Google pulled the ability to get this Google pulled the ability to get this Google pulled the ability to get this information, these specific examples, information, these specific examples, information, these specific examples, they pulled those from AI overviews.

they pulled those from AI overviews. they pulled those from AI overviews. But, do we have to wait for journalists But, do we have to wait for journalists But, do we have to wait for journalists to find all the bad queries you can give to find all the bad queries you can give to find all the bad queries you can give the AI system in order for the AI system the AI system in order for the AI system the AI system in order for the AI system to turn those off or to to fix those? We to turn those off or to to fix those? We to turn those off or to to fix those? We shouldn't have to. shouldn't have to. shouldn't have to. We should be getting good information. We should be getting good information. We should be getting good information. And the fact that search engines are And the fact that search engines are And the fact that search engines are taking away more and more the ability taking away more and more the ability taking away more and more the ability even find the answers even find the answers even find the answers and more and more going to the AI answer and more and more going to the AI answer and more and more going to the AI answer is a real problem. Because when it was is a real problem. Because when it was is a real problem. Because when it was just I Googled something and then I went just I Googled something and then I went just I Googled something and then I went to the site and it was, you know, WebMD to the site and it was, you know, WebMD to the site and it was, you know, WebMD or it was, you know, some other site or it was, you know, some other site or it was, you know, some other site that I trusted, I would go there and that I trusted, I would go there and that I trusted, I would go there and look at it. And I even knew that, you look at it. And I even knew that, you look at it. And I even knew that, you know, WebMD always told you you had know, WebMD always told you you had know, WebMD always told you you had cancer, right? You know, my my nose is cancer, right? You know, my my nose is cancer, right? You know, my my nose is stuffed up, you probably have cancer. stuffed up, you probably have cancer. stuffed up, you probably have cancer. Like that was kind of the joke. But at Like that was kind of the joke. But at Like that was kind of the joke. But at the same time, when I go there, I go to the same time, when I go there, I go to the same time, when I go there, I go to Mayo Clinic, when I go to other sites Mayo Clinic, when I go to other sites Mayo Clinic, when I go to other sites where I'm like, yes, this probably has where I'm like, yes, this probably has where I'm like, yes, this probably has the right information, I'm at least the right information, I'm at least the right information, I'm at least getting information that's been vetted getting information that's been vetted getting information that's been vetted by doctors, that's been vetted by, you by doctors, that's been vetted by, you by doctors, that's been vetted by, you know, large studies. And yes, it may be know, large studies. And yes, it may be know, large studies. And yes, it may be wrong as in they figure something out wrong as in they figure something out wrong as in they figure something out over time, that's how science works.

over time, that's how science works. over time, that's how science works. Um but at the same time, I'm basing that Um but at the same time, I'm basing that Um but at the same time, I'm basing that off of the best information we have at off of the best information we have at off of the best information we have at time, not basing it off of whatever the time, not basing it off of whatever the time, not basing it off of whatever the AI decided to say today. AI decided to say today. AI decided to say today. Because Because Because those answers, like like Google said, those answers, like like Google said, those answers, like like Google said, the vast majority are correct. Many the vast majority are correct. Many the vast majority are correct. Many instances are correct. Because the AI instances are correct. Because the AI instances are correct. Because the AI does not give the same answer every time does not give the same answer every time does not give the same answer every time to the same question. to the same question. to the same question. If you ask the same question, you have If you ask the same question, you have If you ask the same question, you have all your friends ask the same question all your friends ask the same question all your friends ask the same question of AI, you will get different answers of AI, you will get different answers of AI, you will get different answers from your friends. Now, most of them may from your friends. Now, most of them may from your friends. Now, most of them may be the same or similar, be the same or similar, be the same or similar, um but they won't be the same. And some um but they won't be the same. And some um but they won't be the same. And some will get answers that are just out of will get answers that are just out of will get answers that are just out of left field. left field. left field. Why? Because the AI that's how it works. Why? Because the AI that's how it works. Why? Because the AI that's how it works. It's not designed to be a I put in this It's not designed to be a I put in this It's not designed to be a I put in this question, I get back this specific question, I get back this specific question, I get back this specific answer. That's not how it works. answer. That's not how it works. answer. That's not how it works. But unfortunately, people are basing But unfortunately, people are basing But unfortunately, people are basing their health information off of this their health information off of this their health information off of this system. And it makes sense because at system. And it makes sense because at system. And it makes sense because at least in the US, the health system is least in the US, the health system is least in the US, the health system is not free. Even in other countries where not free. Even in other countries where not free. Even in other countries where the health system may be free, the health system may be free, the health system may be free, the access may be slower than we'd like.

the access may be slower than we'd like. the access may be slower than we'd like. And so going to a system that gives you And so going to a system that gives you And so going to a system that gives you immediate answers immediate answers immediate answers can feel like can feel like can feel like a shortcut. It can feel like the right a shortcut. It can feel like the right a shortcut. It can feel like the right way to go about things to get things way to go about things to get things way to go about things to get things moving. moving. moving. And it can work, And it can work, And it can work, but it's not always going to be right. but it's not always going to be right. but it's not always going to be right. And that can be a really bad thing. And that can be a really bad thing. And that can be a really bad thing. All right. All right. All right. Um Um Um and this is why. and this is why. and this is why. Okay? So, AI fails at primary patient Okay? So, AI fails at primary patient Okay? So, AI fails at primary patient diagnosis when 80% of the time. Now, I diagnosis when 80% of the time. Now, I diagnosis when 80% of the time. Now, I dug into this report because this this dug into this report because this this dug into this report because this this this uh paper, by the way, all links are this uh paper, by the way, all links are this uh paper, by the way, all links are down below. down below. down below. Um you can see down at the bottom of the Um you can see down at the bottom of the Um you can see down at the bottom of the screen. But I dug into this paper and screen. But I dug into this paper and screen. But I dug into this paper and the actual data behind it because the actual data behind it because the actual data behind it because I wasn't sure this is correct, right? I wasn't sure this is correct, right? I wasn't sure this is correct, right? What feels like the title is saying is What feels like the title is saying is What feels like the title is saying is that only 20% that only 20% that only 20% of the time do they get patient of the time do they get patient of the time do they get patient diagnosis correct. diagnosis correct. diagnosis correct. That's what they're saying. That's what they're saying. That's what they're saying. Now, let's look at what this says, okay? Now, let's look at what this says, okay? Now, let's look at what this says, okay? So, what they did was they So, what they did was they So, what they did was they So, they said AI chatbots have improved, So, they said AI chatbots have improved, So, they said AI chatbots have improved, which is terrifying to think they've which is terrifying to think they've which is terrifying to think they've improved and yet 80% are wrong.

improved and yet 80% are wrong. improved and yet 80% are wrong. Um but they still fail to produce an Um but they still fail to produce an Um but they still fail to produce an appropriate differential diagnosis more appropriate differential diagnosis more appropriate differential diagnosis more than 80% of time. than 80% of time. than 80% of time. Meaning they were incorrect at at Meaning they were incorrect at at Meaning they were incorrect at at producing a appropriate differential producing a appropriate differential producing a appropriate differential diagnosis diagnosis diagnosis four times out of five. four times out of five. four times out of five. Now, Now, Now, what is this a study of? Well, this is what is this a study of? Well, this is what is this a study of? Well, this is based upon the JAMA Network Open Medical based upon the JAMA Network Open Medical based upon the JAMA Network Open Medical Journal Journal Journal um and it's based upon a study of 21 um and it's based upon a study of 21 um and it's based upon a study of 21 LLMs, the latest versions of Claude, LLMs, the latest versions of Claude, LLMs, the latest versions of Claude, DeepSeek, Gemini, GPT, and Grok. DeepSeek, Gemini, GPT, and Grok. DeepSeek, Gemini, GPT, and Grok. And they evaluated over 29 standardized And they evaluated over 29 standardized And they evaluated over 29 standardized clinical vignettes. clinical vignettes. clinical vignettes. Now, Now, Now, what this does is it evaluates them like what this does is it evaluates them like what this does is it evaluates them like a doctor, not just doing the a doctor, not just doing the a doctor, not just doing the the the testing, right? So, so if we the the testing, right? So, so if we the the testing, right? So, so if we have a test that, you know, in school we have a test that, you know, in school we have a test that, you know, in school we have a test where it says A, B, C, or D, have a test where it says A, B, C, or D, have a test where it says A, B, C, or D, which is the right answer, or fill in which is the right answer, or fill in which is the right answer, or fill in the blank. This is often how we evaluate the blank. This is often how we evaluate the blank. This is often how we evaluate AIs. Fill in the blank or, you know, AIs. Fill in the blank or, you know, AIs. Fill in the blank or, you know, complete this test. The problem is complete this test. The problem is complete this test. The problem is that's not how the real world works.

that's not how the real world works. that's not how the real world works. And so, what they said let's test the And so, what they said let's test the And so, what they said let's test the AIs like we test doctors for real-world AIs like we test doctors for real-world AIs like we test doctors for real-world assessment. So, there were there's assessment. So, there were there's assessment. So, there were there's different stages of clinical reasoning. different stages of clinical reasoning. different stages of clinical reasoning. This is where you have to kind of This is where you have to kind of This is where you have to kind of understand what this report is saying. understand what this report is saying. understand what this report is saying. So, there's conducting an initial So, there's conducting an initial So, there's conducting an initial diagnosis. So, you walk into the diagnosis. So, you walk into the diagnosis. So, you walk into the doctor's office, they initially diagnose doctor's office, they initially diagnose doctor's office, they initially diagnose you based upon certain information. Then you based upon certain information. Then you based upon certain information. Then ordering appropriate tests. Meaning you ordering appropriate tests. Meaning you ordering appropriate tests. Meaning you walk in and I'm not a doctor here, so walk in and I'm not a doctor here, so walk in and I'm not a doctor here, so this is not medical advice. But, you this is not medical advice. But, you this is not medical advice. But, you know, I walk Well, I limp in and my one know, I walk Well, I limp in and my one know, I walk Well, I limp in and my one leg is hanging funny because it really leg is hanging funny because it really leg is hanging funny because it really hurts. hurts. hurts. And they say, "Okay, let's order an And they say, "Okay, let's order an And they say, "Okay, let's order an X-ray." That's ordering appropriate X-ray." That's ordering appropriate X-ray." That's ordering appropriate tests. tests. tests. Now, that X-ray should be of my leg, the Now, that X-ray should be of my leg, the Now, that X-ray should be of my leg, the one that I'm not on because I can't walk one that I'm not on because I can't walk one that I'm not on because I can't walk on it. It shouldn't be of my arm. It on it. It shouldn't be of my arm. It on it. It shouldn't be of my arm. It shouldn't be of the the good leg, right? shouldn't be of the the good leg, right? shouldn't be of the the good leg, right? And then arriving at a final diagnosis. And then arriving at a final diagnosis. And then arriving at a final diagnosis. So, So, So, order the appropriate tests and go, order the appropriate tests and go, order the appropriate tests and go, "Yep, I can see there's a break in your "Yep, I can see there's a break in your "Yep, I can see there's a break in your leg. That means you have a broken leg."

leg. That means you have a broken leg." leg. That means you have a broken leg." That would be the final diagnosis and That would be the final diagnosis and That would be the final diagnosis and then planning treatment. Meaning, "Okay, then planning treatment. Meaning, "Okay, then planning treatment. Meaning, "Okay, what we're going to do is put a cast on what we're going to do is put a cast on what we're going to do is put a cast on your leg." Okay? Those are the stages your leg." Okay? Those are the stages your leg." Okay? Those are the stages they're kind of evaluating they're kind of evaluating they're kind of evaluating the AI on. It's not just, the AI on. It's not just, the AI on. It's not just, you know, with all this information I you know, with all this information I you know, with all this information I present to you, you know, if if you say, present to you, you know, if if you say, present to you, you know, if if you say, "Hey, I have this x-ray of this leg and "Hey, I have this x-ray of this leg and "Hey, I have this x-ray of this leg and it shows this. it shows this. it shows this. And I have all this information on the And I have all this information on the And I have all this information on the patient, what do you think the diagnosis patient, what do you think the diagnosis patient, what do you think the diagnosis is?" is?" is?" Because you've already Because you've already Because you've already gathered all the information. The gathered all the information. The gathered all the information. The problem is, doctors don't get all the problem is, doctors don't get all the problem is, doctors don't get all the information up front. information up front. information up front. What they get is a patient that walks in What they get is a patient that walks in What they get is a patient that walks in and says, "My leg hurts." and says, "My leg hurts." and says, "My leg hurts." And they have to figure it out from And they have to figure it out from And they have to figure it out from there. And that's why they tested the AI there. And that's why they tested the AI there. And that's why they tested the AI on. on. on. So, So, So, this is a stepwise fashion. Research this is a stepwise fashion. Research this is a stepwise fashion. Research moves past treating them like moves past treating them like moves past treating them like test-takers and puts them in a doctor's test-takers and puts them in a doctor's test-takers and puts them in a doctor's position. So, that's what I was saying. position. So, that's what I was saying. position. So, that's what I was saying. So, these models are great at naming a So, these models are great at naming a So, these models are great at naming a final diagnosis. So, final diagnosis. So, final diagnosis. So, back up. Four times out of five, they back up. Four times out of five, they back up. Four times out of five, they were incorrect. Yes, but not on final were incorrect. Yes, but not on final were incorrect. Yes, but not on final diagnosis, on the um the original like diagnosis, on the um the original like diagnosis, on the um the original like diagnosis, the differential.

diagnosis, the differential. diagnosis, the differential. So, So, So, once the data is complete. Again, once the data is complete. Again, once the data is complete. Again, doctors don't have patients walk in with doctors don't have patients walk in with doctors don't have patients walk in with the CAT scan, MRI, and the the x-ray the CAT scan, MRI, and the the x-ray the CAT scan, MRI, and the the x-ray done, the blood work done, all the rest. done, the blood work done, all the rest. done, the blood work done, all the rest. They don't walk in with that. They have They don't walk in with that. They have They don't walk in with that. They have to figure out what of those things are to figure out what of those things are to figure out what of those things are needed and why. needed and why. needed and why. All right? So, AI struggle at the All right? So, AI struggle at the All right? So, AI struggle at the open-ended start of a case where there open-ended start of a case where there open-ended start of a case where there isn't much information. isn't much information. isn't much information. So, doctors, you know, but this is what So, doctors, you know, but this is what So, doctors, you know, but this is what doctors do. This is why they go through doctors do. This is why they go through doctors do. This is why they go through medical school. This is why they learn medical school. This is why they learn medical school. This is why they learn to diagnose people. to diagnose people. to diagnose people. So, the researchers found that all model So, the researchers found that all model So, the researchers found that all model that all the models failed to produce an that all the models failed to produce an that all the models failed to produce an appropriate differential diagnosis more appropriate differential diagnosis more appropriate differential diagnosis more than 80% of time. than 80% of time. than 80% of time. So, So, So, that that appropriate differential that that appropriate differential that that appropriate differential diagnosis was wrong a lot. Four out of diagnosis was wrong a lot. Four out of diagnosis was wrong a lot. Four out of five times it was wrong. The final five times it was wrong. The final five times it was wrong. The final diagnosis success rates ranged from diagnosis success rates ranged from diagnosis success rates ranged from about 60% to over 90% depending on the about 60% to over 90% depending on the about 60% to over 90% depending on the model. Stop right there. First of all, model. Stop right there. First of all, model. Stop right there. First of all, one of the questions might come up is, one of the questions might come up is, one of the questions might come up is, well, then let's just skip all this well, then let's just skip all this well, then let's just skip all this diagnosis stuff and go right to the diagnosis stuff and go right to the diagnosis stuff and go right to the final final final the final diagnosis. That's not how the final diagnosis. That's not how the final diagnosis. That's not how medicine works. You have to figure out medicine works. You have to figure out medicine works. You have to figure out what to do, the differential, in order what to do, the differential, in order what to do, the differential, in order to figure out how to order the right to figure out how to order the right to figure out how to order the right tests in order to get to the place of tests in order to get to the place of tests in order to get to the place of the final diagnosis.

the final diagnosis. the final diagnosis. So, you can't escape steps. So, you can't escape steps. So, you can't escape steps. And the AI is very, very, very bad at And the AI is very, very, very bad at And the AI is very, very, very bad at the earlier steps. Now, the earlier steps. Now, the earlier steps. Now, even though you say, well, you know even though you say, well, you know even though you say, well, you know what, over 90% some models are over 90% what, over 90% some models are over 90% what, over 90% some models are over 90% and they're the ones you'd probably and they're the ones you'd probably and they're the ones you'd probably expect, right? The the latest version of expect, right? The the latest version of expect, right? The the latest version of uh I think Opus and GPT were up near the uh I think Opus and GPT were up near the uh I think Opus and GPT were up near the top there. top there. top there. So, you might think, well, 90% So, you might think, well, 90% So, you might think, well, 90% okay, are you willing to, you know, have okay, are you willing to, you know, have okay, are you willing to, you know, have an AI guess because it an AI guess because it an AI guess because it guesses right nine out of 10 times? This guesses right nine out of 10 times? This guesses right nine out of 10 times? This is This is Russian roulette with our is This is Russian roulette with our is This is Russian roulette with our health. health. health. 90% is not good enough. 90% is not good enough. 90% is not good enough. So, So, So, most LLMs showed improved accuracy when most LLMs showed improved accuracy when most LLMs showed improved accuracy when provided with laboratory results and provided with laboratory results and provided with laboratory results and imaging in addition to text. When you imaging in addition to text. When you imaging in addition to text. When you give it all the information, give it all the information, give it all the information, then it can get a pretty good diagnosis. then it can get a pretty good diagnosis. then it can get a pretty good diagnosis. But, here's the deal. But, here's the deal. But, here's the deal. When you give it all the information, When you give it all the information, When you give it all the information, you're pretty much already giving it the you're pretty much already giving it the you're pretty much already giving it the final diagnosis. It's just evaluating a final diagnosis. It's just evaluating a final diagnosis. It's just evaluating a little bit, right?

little bit, right? little bit, right? If If I walked in that you're a doctor, If If I walked in that you're a doctor, If If I walked in that you're a doctor, I walked into your office, I said, I walked into your office, I said, I walked into your office, I said, here's an X-ray of my right leg, here's an X-ray of my right leg, here's an X-ray of my right leg, here's all the information about why my here's all the information about why my here's all the information about why my my right leg hurts. my right leg hurts. my right leg hurts. What's my diagnosis? What's my diagnosis? What's my diagnosis? How hard is it for that doctor? Wouldn't How hard is it for that doctor? Wouldn't How hard is it for that doctor? Wouldn't that doctor be like, "Man, this is the that doctor be like, "Man, this is the that doctor be like, "Man, this is the easiest job ever." Because I look at the easiest job ever." Because I look at the easiest job ever." Because I look at the x-ray go, "Yep, it's broken. You've got x-ray go, "Yep, it's broken. You've got x-ray go, "Yep, it's broken. You've got a broken leg." That's what the AI is a broken leg." That's what the AI is a broken leg." That's what the AI is {quote} good at. {quote} good at. {quote} good at. But it's getting to that point that it But it's getting to that point that it But it's getting to that point that it is terrible at. is terrible at. is terrible at. So, So, So, this title "Sick and Wrong Ontario this title "Sick and Wrong Ontario this title "Sick and Wrong Ontario Auditors Find Doctors AI Note Takers Auditors Find Doctors AI Note Takers Auditors Find Doctors AI Note Takers Routinely Blow Basic Facts." Okay, so Routinely Blow Basic Facts." Okay, so Routinely Blow Basic Facts." Okay, so kind of switching gears a little bit kind of switching gears a little bit kind of switching gears a little bit here, um and that is that a lot of here, um and that is that a lot of here, um and that is that a lot of doctors now are using AI note takers. doctors now are using AI note takers. doctors now are using AI note takers. Where the AI listens to the conversation Where the AI listens to the conversation Where the AI listens to the conversation and then diagnoses or not diagnoses, but and then diagnoses or not diagnoses, but and then diagnoses or not diagnoses, but just gathers the information, right? So, just gathers the information, right? So, just gathers the information, right? So, instead of doctor typing in, "Okay, your instead of doctor typing in, "Okay, your instead of doctor typing in, "Okay, your your leg hurts. Okay, I'm ordering your leg hurts. Okay, I'm ordering your leg hurts. Okay, I'm ordering x-ray. Okay, I'm going to give you x-ray. Okay, I'm going to give you x-ray. Okay, I'm going to give you ibuprofen for the pain." Right? Instead ibuprofen for the pain." Right? Instead ibuprofen for the pain." Right? Instead of putting all that in the computer, the of putting all that in the computer, the of putting all that in the computer, the AI does it for them. And it takes notes.

AI does it for them. And it takes notes. AI does it for them. And it takes notes. It says, you know, "The patient It says, you know, "The patient It says, you know, "The patient complained of right leg pain below the complained of right leg pain below the complained of right leg pain below the knee and, you know, they were limping, knee and, you know, they were limping, knee and, you know, they were limping, etc." etc." etc." But in Ontario, they've been studying But in Ontario, they've been studying But in Ontario, they've been studying these AI note takers because they've these AI note takers because they've these AI note takers because they've been using it in testing cases. Major issues. Major issues. 60% of Major issues. Major issues. 60% of evaluated AI scribe systems mixed up evaluated AI scribe systems mixed up evaluated AI scribe systems mixed up prescription drugs in patient notes. Do you want your patient notes to have Do you want your patient notes to have the wrong prescription drugs listed? the wrong prescription drugs listed? the wrong prescription drugs listed? Do you think that's a good thing? Do you think that's a good thing? Do you think that's a good thing? Terrible. Awful. Horrible. This can Terrible. Awful. Horrible. This can Terrible. Awful. Horrible. This can cause cause cause massive issues. massive issues. massive issues. So, the AI systems approved for Ontario So, the AI systems approved for Ontario So, the AI systems approved for Ontario healthcare system healthcare providers healthcare system healthcare providers healthcare system healthcare providers routinely missed critical details, routinely missed critical details, routinely missed critical details, insert incorrect information, and insert incorrect information, and insert incorrect information, and hallucinated content. hallucinated content. hallucinated content. That's the worst, okay? Now, this is That's the worst, okay? Now, this is That's the worst, okay? Now, this is according to an audit of 20 approved according to an audit of 20 approved according to an audit of 20 approved vendor systems. So, these are already vendor systems. So, these are already vendor systems. So, these are already approved approved approved and this is an audit of them. Now, let's and this is an audit of them. Now, let's and this is an audit of them. Now, let's look at a bit more about this. They look at a bit more about this. They look at a bit more about this. They simulated doctor-patient recordings. So, simulated doctor-patient recordings. So, simulated doctor-patient recordings. So, this is not about looking at actual this is not about looking at actual this is not about looking at actual patient uh solution. That's so much patient uh solution. That's so much patient uh solution. That's so much harder to do because the fact that there harder to do because the fact that there harder to do because the fact that there is doctor-patient privilege and is doctor-patient privilege and is doctor-patient privilege and confidentiality. You can't just come in confidentiality. You can't just come in confidentiality. You can't just come in and say, "Hey, can we evaluate to see if and say, "Hey, can we evaluate to see if and say, "Hey, can we evaluate to see if if this is happening correctly? You if this is happening correctly? You if this is happening correctly? You know, we want to bring a whole bunch of know, we want to bring a whole bunch of know, we want to bring a whole bunch of people in to do a study on this."

people in to do a study on this." people in to do a study on this." That's not something you really want to That's not something you really want to That's not something you really want to do, so it's a lot harder once these do, so it's a lot harder once these do, so it's a lot harder once these tools are in play to evaluate their tools are in play to evaluate their tools are in play to evaluate their effective performance except through effective performance except through effective performance except through testing like this where they simulate. testing like this where they simulate. testing like this where they simulate. And so, what they did is they simulated And so, what they did is they simulated And so, what they did is they simulated doctor-patient recordings and then a doctor-patient recordings and then a doctor-patient recordings and then a medical professional reviewed the medical professional reviewed the medical professional reviewed the original recording and the notes that original recording and the notes that original recording and the notes that the AI note-taker took. the AI note-taker took. the AI note-taker took. Okay. Okay. Okay. Let's look at nine out of 20 reportedly Let's look at nine out of 20 reportedly Let's look at nine out of 20 reportedly fabricated information fabricated information fabricated information and made suggestions to patients' and made suggestions to patients' and made suggestions to patients' treatment plans that weren't discussed treatment plans that weren't discussed treatment plans that weren't discussed in the recordings. in the recordings. in the recordings. According to the report, evaluators According to the report, evaluators According to the report, evaluators spied potentially devastating incorrect spied potentially devastating incorrect spied potentially devastating incorrect information in sample reports such as no information in sample reports such as no information in sample reports such as no masses being found, which mean there masses being found, which mean there masses being found, which mean there was, or patients being anxious even was, or patients being anxious even was, or patients being anxious even those even though those things were those even though those things were those even though those things were never discussed in the recordings. never discussed in the recordings. never discussed in the recordings. So, it's putting information in your So, it's putting information in your So, it's putting information in your report that wasn't there. It's changing report that wasn't there. It's changing report that wasn't there. It's changing the information that was there to maybe the information that was there to maybe the information that was there to maybe the wrong information and causing other the wrong information and causing other the wrong information and causing other problems. Nine out of 20. 12 of the 20 problems. Nine out of 20. 12 of the 20 problems. Nine out of 20. 12 of the 20 inserted incorrect drug information into inserted incorrect drug information into inserted incorrect drug information into patients' notes. While 17 of systems patients' notes. While 17 of systems patients' notes. While 17 of systems missed key details about the patients' missed key details about the patients' missed key details about the patients' mental health issues.

mental health issues. mental health issues. Six of the systems missed the patient's Six of the systems missed the patient's Six of the systems missed the patient's mental health issues fully or partially mental health issues fully or partially mental health issues fully or partially or were missing key details. This is a major problem not just because This is a major problem not just because the AI note-takers are doing things the AI note-takers are doing things the AI note-takers are doing things incorrectly. They're putting the wrong, incorrectly. They're putting the wrong, incorrectly. They're putting the wrong, you know, medications on. They're you know, medications on. They're you know, medications on. They're missing things. They're They're putting missing things. They're They're putting missing things. They're They're putting things on that weren't there, etc. But, things on that weren't there, etc. But, things on that weren't there, etc. But, whenever you have something done for you whenever you have something done for you whenever you have something done for you and you have to review it as, you know, and you have to review it as, you know, and you have to review it as, you know, I'm a developer, I come from a developer I'm a developer, I come from a developer I'm a developer, I come from a developer perspective, perspective, perspective, we look at code reviews, we we've seen we look at code reviews, we we've seen we look at code reviews, we we've seen we have years of study that have proven we have years of study that have proven we have years of study that have proven that you don't review as well as you that you don't review as well as you that you don't review as well as you create. Meaning, if you're going to code create. Meaning, if you're going to code create. Meaning, if you're going to code something, you're going to do a better something, you're going to do a better something, you're going to do a better job of it versus if you're going to job of it versus if you're going to job of it versus if you're going to review something and try and find the review something and try and find the review something and try and find the problems. problems. problems. You know, it's it's if you if your buddy You know, it's it's if you if your buddy You know, it's it's if you if your buddy gives you a a long email to read over gives you a a long email to read over gives you a a long email to read over and say, "Hey, can you proofread this?" and say, "Hey, can you proofread this?" and say, "Hey, can you proofread this?" You're going to miss some things You're going to miss some things You're going to miss some things probably because it's a whole lot easier probably because it's a whole lot easier probably because it's a whole lot easier to skim over your your brain kind of, to skim over your your brain kind of, to skim over your your brain kind of, you know, hits the high spots and goes, you know, hits the high spots and goes, you know, hits the high spots and goes, "Yeah, yeah, yeah, yeah, yeah, looks "Yeah, yeah, yeah, yeah, yeah, looks "Yeah, yeah, yeah, yeah, yeah, looks good, looks good, looks good."

good, looks good, looks good." good, looks good, looks good." And that's what can happen with doctors, And that's what can happen with doctors, And that's what can happen with doctors, too. too. too. When doctors are so busy that they're When doctors are so busy that they're When doctors are so busy that they're moving from patient to patient to moving from patient to patient to moving from patient to patient to patient, how often do you think they're patient, how often do you think they're patient, how often do you think they're going to go back right after the going to go back right after the going to go back right after the appointment, go back and look at that appointment, go back and look at that appointment, go back and look at that and verify point by point that and verify point by point that and verify point by point that everything is perfect? everything is perfect? everything is perfect? Practically nonexistent. Because imagine Practically nonexistent. Because imagine Practically nonexistent. Because imagine even if at the end of the day the doctor even if at the end of the day the doctor even if at the end of the day the doctor says, "I'm going to review all the says, "I'm going to review all the says, "I'm going to review all the notes." notes." notes." That doctor may have seen 10 patients That doctor may have seen 10 patients That doctor may have seen 10 patients that day. Do you think they'll be that day. Do you think they'll be that day. Do you think they'll be precise in their memory? Or they look at precise in their memory? Or they look at precise in their memory? Or they look at something and say, "I don't know if we something and say, "I don't know if we something and say, "I don't know if we talked about that or not, but I guess we talked about that or not, but I guess we talked about that or not, but I guess we did because it's in the notes." did because it's in the notes." did because it's in the notes." So, now all of a sudden the AI notes are So, now all of a sudden the AI notes are So, now all of a sudden the AI notes are going to become the source of truth going to become the source of truth going to become the source of truth rather than rather than rather than what the actual doctor said to do. And what the actual doctor said to do. And what the actual doctor said to do. And that's a big problem. that's a big problem. that's a big problem. So, So, So, to kind of this is the the wrap-up of to kind of this is the the wrap-up of to kind of this is the the wrap-up of this. Remember this was a simulated this. Remember this was a simulated this. Remember this was a simulated doctor-patient conversation. They They doctor-patient conversation. They They doctor-patient conversation. They They did something where they recorded actual did something where they recorded actual did something where they recorded actual conversation, but it was not a real conversation, but it was not a real conversation, but it was not a real doctor-patient conversation.

doctor-patient conversation. doctor-patient conversation. Um Um Um This has already been placed in This has already been placed in This has already been placed in production in with real patients production in with real patients production in with real patients for over 5,000 physicians in Ontario. for over 5,000 physicians in Ontario. for over 5,000 physicians in Ontario. And according to them, there are no And according to them, there are no And according to them, there are no known reports of patient harms. known reports of patient harms. known reports of patient harms. Now, Now, Now, they found massive issues they found massive issues they found massive issues in what is 9 out of 12 that or 20 12 out in what is 9 out of 12 that or 20 12 out in what is 9 out of 12 that or 20 12 out of 20 17 out of 20 6 out of 20 like of 20 17 out of 20 6 out of 20 like of 20 17 out of 20 6 out of 20 like a number of these systems had major a number of these systems had major a number of these systems had major issues in testing when they actually issues in testing when they actually issues in testing when they actually compared this. compared this. compared this. But in production with more than 5,000 But in production with more than 5,000 But in production with more than 5,000 doctors using it, there's been no known doctors using it, there's been no known doctors using it, there's been no known issues. issues. issues. So, So, So, there's only a couple reasons why that there's only a couple reasons why that there's only a couple reasons why that might happen. might happen. might happen. One is One is One is maybe that they've got lucky. You know, maybe that they've got lucky. You know, maybe that they've got lucky. You know, maybe they're more rigorous in their maybe they're more rigorous in their maybe they're more rigorous in their testing and you know, we're trying to testing and you know, we're trying to testing and you know, we're trying to trick it more often. trick it more often. trick it more often. That could be, but based upon how we've That could be, but based upon how we've That could be, but based upon how we've seen AI usage, that's probably not the seen AI usage, that's probably not the seen AI usage, that's probably not the case. case. case. Um or the other issue is that Um or the other issue is that Um or the other issue is that there are issues, they haven't been there are issues, they haven't been there are issues, they haven't been caught.

caught. caught. Because how would you know if there's Because how would you know if there's Because how would you know if there's issues? issues? issues? It's only if something happened and the It's only if something happened and the It's only if something happened and the doctor caught it. doctor caught it. doctor caught it. And those things don't happen first of And those things don't happen first of And those things don't happen first of all right away. all right away. all right away. Because it's going to depend on the Because it's going to depend on the Because it's going to depend on the doctor going, "Wait, that's not right." doctor going, "Wait, that's not right." doctor going, "Wait, that's not right." or the patient going, "Wait, that's not or the patient going, "Wait, that's not or the patient going, "Wait, that's not right." But that doesn't happen right." But that doesn't happen right." But that doesn't happen immediately. That might not even happen immediately. That might not even happen immediately. That might not even happen for months or even a year or two for months or even a year or two for months or even a year or two before you figure out that something before you figure out that something before you figure out that something doesn't seem right. And by that point, doesn't seem right. And by that point, doesn't seem right. And by that point, everyone's brain's a little fuzzy. They everyone's brain's a little fuzzy. They everyone's brain's a little fuzzy. They go, "Well, maybe I Maybe I wasn't on go, "Well, maybe I Maybe I wasn't on go, "Well, maybe I Maybe I wasn't on that medication. Or maybe I I did, you that medication. Or maybe I I did, you that medication. Or maybe I I did, you know, have that test and I, you know, or know, have that test and I, you know, or know, have that test and I, you know, or maybe I did have anxiety. maybe I did have anxiety. maybe I did have anxiety. Um so, Um so, Um so, no known reports no known reports no known reports doesn't mean there's been no issues. It doesn't mean there's been no issues. It doesn't mean there's been no issues. It just means we haven't found them. Okay. Okay. Here's an audit of the clinical AI. This Here's an audit of the clinical AI. This Here's an audit of the clinical AI. This is from Harvard Science Review. is from Harvard Science Review. is from Harvard Science Review. Um the first real audit of clinical AI. Um the first real audit of clinical AI. Um the first real audit of clinical AI. So, for the past 5 years, the narrative So, for the past 5 years, the narrative So, for the past 5 years, the narrative surrounding artificial intelligence in surrounding artificial intelligence in surrounding artificial intelligence in medicine has been one of breathless medicine has been one of breathless medicine has been one of breathless inevitability.

inevitability. inevitability. I think that was a really good phrase to I think that was a really good phrase to I think that was a really good phrase to highlight. This is what's been coming highlight. This is what's been coming highlight. This is what's been coming about, you know, people are saying how about, you know, people are saying how about, you know, people are saying how AI's going to revolutionize healthcare. AI's going to revolutionize healthcare. AI's going to revolutionize healthcare. So, they did a study So, they did a study So, they did a study between Harvard Medical School and between Harvard Medical School and between Harvard Medical School and Stanford University. Stanford University. Stanford University. They asked a question the industry has They asked a question the industry has They asked a question the industry has avoided. avoided. avoided. Is it actually working? Is it actually working? Is it actually working? Because there's all this hype. Because there's all this hype. Because there's all this hype. There's all these companies are saying There's all these companies are saying There's all these companies are saying how great the medical advances are going how great the medical advances are going how great the medical advances are going to be. to be. to be. But yet, is it actually working? But yet, is it actually working? But yet, is it actually working? Well, Well, Well, let's look at that in this state of let's look at that in this state of let's look at that in this state of clinical AI 2026. clinical AI 2026. clinical AI 2026. So, So, So, the FDA, as of January 2026, has cleared the FDA, as of January 2026, has cleared the FDA, as of January 2026, has cleared over 1,200 AI-enabled medical devices, over 1,200 AI-enabled medical devices, over 1,200 AI-enabled medical devices, which first of all, just terrifies me. which first of all, just terrifies me. which first of all, just terrifies me. But okay, there's a 1,200 AI medical But okay, there's a 1,200 AI medical But okay, there's a 1,200 AI medical devices cleared. Less than 15% of them devices cleared. Less than 15% of them devices cleared. Less than 15% of them are used routinely. are used routinely. are used routinely. Why? Why? Why? Well, according to them, the first issue Well, according to them, the first issue Well, according to them, the first issue is that the interfaces are bad.

is that the interfaces are bad. is that the interfaces are bad. And it says the friction is winning. The And it says the friction is winning. The And it says the friction is winning. The friction of using these tools is friction of using these tools is friction of using these tools is winning. winning. winning. Which Which Which that that's an easy problem or an that that's an easy problem or an that that's an easy problem or an easy-ish problem in theory because, easy-ish problem in theory because, easy-ish problem in theory because, well, you just well, you just well, you just build a better UI or build a better build a better UI or build a better build a better UI or build a better process. process. process. Um you know, the example down below is Um you know, the example down below is Um you know, the example down below is the algorithm may be 99% accurate at the algorithm may be 99% accurate at the algorithm may be 99% accurate at detecting nodules on a CT scan, but it detecting nodules on a CT scan, but it detecting nodules on a CT scan, but it requires a radiologist to click through requires a radiologist to click through requires a radiologist to click through three different login screens to see the three different login screens to see the three different login screens to see the results. results. results. Yeah, that's frustrating. And, you know, Yeah, that's frustrating. And, you know, Yeah, that's frustrating. And, you know, it's it's probably one of those logins it's it's probably one of those logins it's it's probably one of those logins where it's like, you know, where it's like, you know, where it's like, you know, you have to log in a different way and you have to log in a different way and you have to log in a different way and do a rain dance first and all the rest do a rain dance first and all the rest do a rain dance first and all the rest in order to try and get in. And that's in order to try and get in. And that's in order to try and get in. And that's frustrating. frustrating. frustrating. Um, Um, Um, I don't like those tr- trying to get I don't like those tr- trying to get I don't like those tr- trying to get into Microsoft Teams, let alone to to do into Microsoft Teams, let alone to to do into Microsoft Teams, let alone to to do my job. So, yes, the friction is my job. So, yes, the friction is my job. So, yes, the friction is winning, and that can be a problem that winning, and that can be a problem that winning, and that can be a problem that can be solved with just can be solved with just can be solved with just better systems. But, sometimes that better systems. But, sometimes that better systems. But, sometimes that friction isn't there for a reason, friction isn't there for a reason, friction isn't there for a reason, because remember again, because remember again, because remember again, someone should be a human in a loop. someone should be a human in a loop. someone should be a human in a loop. Because AI hallucinates, and so if you Because AI hallucinates, and so if you Because AI hallucinates, and so if you make it too easy where you go, "Yep, make it too easy where you go, "Yep, make it too easy where you go, "Yep, yep, yep." And all of a sudden three yep, yep." And all of a sudden three yep, yep." And all of a sudden three different scans are gone, different scans are gone, different scans are gone, well, then you may have had a problem well, then you may have had a problem well, then you may have had a problem where you go, where you go, where you go, "I approved something that the AI did "I approved something that the AI did "I approved something that the AI did incorrectly."

incorrectly." incorrectly." So, we do want some friction in the So, we do want some friction in the So, we do want some friction in the system. But, yes, friction is a problem. system. But, yes, friction is a problem. system. But, yes, friction is a problem. However, However, However, because of that friction, among other because of that friction, among other because of that friction, among other things, things, things, there's been a surge in physicians using there's been a surge in physicians using there's been a surge in physicians using unauthorized tools on personal devices. unauthorized tools on personal devices. unauthorized tools on personal devices. This is I mean, the simplistic example This is I mean, the simplistic example This is I mean, the simplistic example would be a doctor using ChatGPT on their would be a doctor using ChatGPT on their would be a doctor using ChatGPT on their phone. Now, there's other systems like phone. Now, there's other systems like phone. Now, there's other systems like Open Evidence and others that they can Open Evidence and others that they can Open Evidence and others that they can use. Um, but I I highlighted this use. Um, but I I highlighted this use. Um, but I I highlighted this because I think it's really important. because I think it's really important. because I think it's really important. They're choosing speed over compliance. They're choosing speed over compliance. They're choosing speed over compliance. Now, Now, Now, as we saw earlier, as we saw earlier, as we saw earlier, AI systems give out bad advice. AI systems give out bad advice. AI systems give out bad advice. You know, Google was saying that that uh You know, Google was saying that that uh You know, Google was saying that that uh kidney patients should kidney patients should kidney patients should avoid a high-fat diet, which could kill avoid a high-fat diet, which could kill avoid a high-fat diet, which could kill them. them. them. Right? So, these are bad things that Right? So, these are bad things that Right? So, these are bad things that that these AIs are saying, that these AIs are saying, that these AIs are saying, and yet doctors are bypassing and yet doctors are bypassing and yet doctors are bypassing their own AI systems because they're too their own AI systems because they're too their own AI systems because they're too cumbersome and going right to the cumbersome and going right to the cumbersome and going right to the consumer ones.

consumer ones. consumer ones. Which, in theory, the medical ones Which, in theory, the medical ones Which, in theory, the medical ones should be better at having guardrails in should be better at having guardrails in should be better at having guardrails in place, but again, those guardrails are place, but again, those guardrails are place, but again, those guardrails are just suggestions. just suggestions. just suggestions. But even so, those suggestions are But even so, those suggestions are But even so, those suggestions are better than what we have in the the open better than what we have in the the open better than what we have in the the open space. space. space. And they're choosing speed over And they're choosing speed over And they're choosing speed over compliance. compliance. compliance. That to me, again, terrifies me because That to me, again, terrifies me because That to me, again, terrifies me because if my health depends on you going faster if my health depends on you going faster if my health depends on you going faster by using a an AI that might be wrong, by using a an AI that might be wrong, by using a an AI that might be wrong, might be hallucinating right now, might be hallucinating right now, might be hallucinating right now, that's not a good thing. I don't feel that's not a good thing. I don't feel that's not a good thing. I don't feel like I'm going to have better outcomes like I'm going to have better outcomes like I'm going to have better outcomes because of that. because of that. because of that. So, notes doctor highlights the issue. I So, notes doctor highlights the issue. I So, notes doctor highlights the issue. I have 15 minutes per patient. Which is have 15 minutes per patient. Which is have 15 minutes per patient. Which is already a bad thing. But, if the already a bad thing. But, if the already a bad thing. But, if the hospital's approved AI takes 2 minutes hospital's approved AI takes 2 minutes hospital's approved AI takes 2 minutes to load, I'm not using it. I'm using app to load, I'm not using it. I'm using app to load, I'm not using it. I'm using app on my phone that gives me the answer in on my phone that gives me the answer in on my phone that gives me the answer in 5 seconds. 5 seconds. 5 seconds. It's not about rebellion, it's about It's not about rebellion, it's about It's not about rebellion, it's about survival. survival. survival. I get that. I get that you need to move I get that. I get that you need to move I get that. I get that you need to move quickly. The problem is that what quickly. The problem is that what quickly. The problem is that what they're seeing is they're seeing is they're seeing is they're saying, "Well, this is faster."

they're saying, "Well, this is faster." they're saying, "Well, this is faster." But what they're not saying is But what they're not saying is But what they're not saying is that that that is this as just as good of an answer. is this as just as good of an answer. is this as just as good of an answer. [clears throat] [clears throat] [clears throat] You know, you have to know that answer You know, you have to know that answer You know, you have to know that answer is correct. And that answer's not going is correct. And that answer's not going is correct. And that answer's not going to be correct all the time, but you're to be correct all the time, but you're to be correct all the time, but you're moving so fast, moving so fast, moving so fast, are you going to stop and say, "I don't are you going to stop and say, "I don't are you going to stop and say, "I don't think that answer's correct."? think that answer's correct."? think that answer's correct."? Or you're going to be just moving so Or you're going to be just moving so Or you're going to be just moving so fast you go, "Okay, that's what we're fast you go, "Okay, that's what we're fast you go, "Okay, that's what we're going to do." going to do." going to do." And And And and follow the AI advice. And now you're and follow the AI advice. And now you're and follow the AI advice. And now you're letting the AI lead in medical decisions letting the AI lead in medical decisions letting the AI lead in medical decisions when the AI is not a physician. It's not when the AI is not a physician. It's not when the AI is not a physician. It's not nowhere near as good as a physician. And nowhere near as good as a physician. And nowhere near as good as a physician. And it's going to give out hallucinated it's going to give out hallucinated it's going to give out hallucinated answers. Not all the time, answers. Not all the time, answers. Not all the time, but just enough to cause patients to be but just enough to cause patients to be but just enough to cause patients to be hurt and even killed. Now, the audit also highlights, this is Now, the audit also highlights, this is why it's even more important. The audit why it's even more important. The audit why it's even more important. The audit also highlights a technical phenomenon also highlights a technical phenomenon also highlights a technical phenomenon known as clinical drift. A term that he known as clinical drift. A term that he known as clinical drift. A term that he asked about, uh several high-profile asked about, uh several high-profile asked about, uh several high-profile sepsis prediction models. So, it's sepsis prediction models. So, it's sepsis prediction models. So, it's predicting uh whether a patient gets predicting uh whether a patient gets predicting uh whether a patient gets sepsis.

sepsis. sepsis. Which performed beautifully in 2023 have Which performed beautifully in 2023 have Which performed beautifully in 2023 have seen their accuracy degrade by nearly seen their accuracy degrade by nearly seen their accuracy degrade by nearly 20% 20% 20% in 2026. in 2026. in 2026. So, the models that were working great So, the models that were working great So, the models that were working great in 2023 in 2023 in 2023 are not in 2026. are not in 2026. are not in 2026. Why? Because hospital protocols changed, Why? Because hospital protocols changed, Why? Because hospital protocols changed, patient demographics shifted, and the patient demographics shifted, and the patient demographics shifted, and the viruses evolved. viruses evolved. viruses evolved. But, the static AI models did not update But, the static AI models did not update But, the static AI models did not update to match the new reality. to match the new reality. to match the new reality. Life is not about putting everything on Life is not about putting everything on Life is not about putting everything on pause, no nothing new gets learned as of pause, no nothing new gets learned as of pause, no nothing new gets learned as of a point in time. a point in time. a point in time. Things change. Things change. Things change. And in the medical field, things change And in the medical field, things change And in the medical field, things change a lot. And this is why doctors are a lot. And this is why doctors are a lot. And this is why doctors are constantly learning new things, looking constantly learning new things, looking constantly learning new things, looking at new medical journals, reading over at new medical journals, reading over at new medical journals, reading over new studies, and trying to evolve over new studies, and trying to evolve over new studies, and trying to evolve over time. This is why we still don't still time. This is why we still don't still time. This is why we still don't still use leeches use leeches use leeches in medicine because we evolve over time in medicine because we evolve over time in medicine because we evolve over time go, that was dumb. go, that was dumb. go, that was dumb. And move on to what the latest And move on to what the latest And move on to what the latest information is. And again, information is. And again, information is. And again, physicians aren't always right because physicians aren't always right because physicians aren't always right because the the things they've studied aren't the the things they've studied aren't the the things they've studied aren't always correct. But, they're very very always correct. But, they're very very always correct. But, they're very very very very good. And it's based upon the very very good. And it's based upon the very very good. And it's based upon the best information we have as of now. And best information we have as of now. And best information we have as of now. And things change over time. Like I said, things change over time. Like I said, things change over time. Like I said, viruses evolve, demographics change, viruses evolve, demographics change, viruses evolve, demographics change, protocols change. And so, the result is protocols change. And so, the result is protocols change. And so, the result is that what was a good diagnostic tool no that what was a good diagnostic tool no that what was a good diagnostic tool no longer is.

We need these models degrade instead of We need these models degrade instead of learn. They're not learning and growing learn. They're not learning and growing learn. They're not learning and growing like a doctor is. They're degrading over like a doctor is. They're degrading over like a doctor is. They're degrading over time. time. time. So, even the medically approved systems So, even the medically approved systems So, even the medically approved systems are degrading over time. are degrading over time. are degrading over time. And doctors are too busy to even use And doctors are too busy to even use And doctors are too busy to even use those systems, and so they're going to those systems, and so they're going to those systems, and so they're going to the private the the the personal systems the private the the the personal systems the private the the the personal systems that aren't even trained to be that aren't even trained to be that aren't even trained to be as good as the medical systems and using as good as the medical systems and using as good as the medical systems and using them instead, which is again degrading them instead, which is again degrading them instead, which is again degrading over time as well. So, So, now this person says, you know, we're now this person says, you know, we're now this person says, you know, we're entering a phase of implementation entering a phase of implementation entering a phase of implementation science. The next billion dollars science. The next billion dollars science. The next billion dollars shouldn't be spent on making algorithms shouldn't be spent on making algorithms shouldn't be spent on making algorithms smarter. It should be meant It should be smarter. It should be meant It should be smarter. It should be meant It should be spent on making them usable. Sounds spent on making them usable. Sounds spent on making them usable. Sounds great. great. great. But I highlighted this last phrase. It's But I highlighted this last phrase. It's But I highlighted this last phrase. It's not part of the quote. It's part of the not part of the quote. It's part of the not part of the quote. It's part of the the author's the author of this the author's the author of this the author's the author of this article's quote. Um maybe we should make article's quote. Um maybe we should make article's quote. Um maybe we should make sure they work first.

sure they work first. sure they work first. Yes, it sounds great to say we should Yes, it sounds great to say we should Yes, it sounds great to say we should you know, make these AI systems usable, you know, make these AI systems usable, you know, make these AI systems usable, but they already aren't great. They're but they already aren't great. They're but they already aren't great. They're not working yet. not working yet. not working yet. So, to make them more easy to use would So, to make them more easy to use would So, to make them more easy to use would just mean they get used more just mean they get used more just mean they get used more when they are drifting away from even when they are drifting away from even when they are drifting away from even where they were, which wasn't correct where they were, which wasn't correct where they were, which wasn't correct very much. very much. very much. Okay? So, the percentages of how these Okay? So, the percentages of how these Okay? So, the percentages of how these systems work get worse and worse over systems work get worse and worse over systems work get worse and worse over time. They were never great to begin time. They were never great to begin time. They were never great to begin with in a lot of areas. And so, with in a lot of areas. And so, with in a lot of areas. And so, we're in a place where we're trying to we're in a place where we're trying to we're in a place where we're trying to make AIs more more enabled for for make AIs more more enabled for for make AIs more more enabled for for medical use, and yet these AIs are medical use, and yet these AIs are medical use, and yet these AIs are they're not doing a good job to begin they're not doing a good job to begin they're not doing a good job to begin with. with. with. They're not doctors. They're not doctors. They're not doctors. There's a reason why we need to have a There's a reason why we need to have a There's a reason why we need to have a human in a loop on everything and it's human in a loop on everything and it's human in a loop on everything and it's because these AI systems hallucinate, because these AI systems hallucinate, because these AI systems hallucinate, they are based upon bad training data, they are based upon bad training data, they are based upon bad training data, the training data is stale by the time the training data is stale by the time the training data is stale by the time it's actually being used and it gets it's actually being used and it gets it's actually being used and it gets more stale over time, it doesn't take in more stale over time, it doesn't take in more stale over time, it doesn't take in enough variables about enough variables about enough variables about the hospital protocols and other things the hospital protocols and other things the hospital protocols and other things that can affect the outcomes and so many that can affect the outcomes and so many that can affect the outcomes and so many other circumstances. So, other circumstances. So, other circumstances. So, both on the personal side as well as the both on the personal side as well as the both on the personal side as well as the corporate side of healthcare, corporate side of healthcare, corporate side of healthcare, AIs are not good AIs are not good AIs are not good for addressing our healthcare. They're for addressing our healthcare. They're for addressing our healthcare. They're not good systems. Yes, they can be 90% not good systems. Yes, they can be 90% not good systems. Yes, they can be 90% effective. That's not good enough.

effective. That's not good enough. effective. That's not good enough. And the fact is that 90% is usually only And the fact is that 90% is usually only And the fact is that 90% is usually only in certain circumstances. They get a lot in certain circumstances. They get a lot in certain circumstances. They get a lot worse when they have more variables to worse when they have more variables to worse when they have more variables to work with. So, what can we do? work with. So, what can we do? work with. So, what can we do? Because this is a problem and our Because this is a problem and our Because this is a problem and our healthcare will get worse because of it. healthcare will get worse because of it. healthcare will get worse because of it. We've seen software get worse and that's We've seen software get worse and that's We've seen software get worse and that's a real problem because we've seen more a real problem because we've seen more a real problem because we've seen more bugs, we've seen more viruses, we've bugs, we've seen more viruses, we've bugs, we've seen more viruses, we've seen more um attacks where you've lost seen more um attacks where you've lost seen more um attacks where you've lost data, etc. But it's software, right? data, etc. But it's software, right? data, etc. But it's software, right? Like Like Like at the end of the day, yes, I don't want at the end of the day, yes, I don't want at the end of the day, yes, I don't want my data stolen and I I want to make sure my data stolen and I I want to make sure my data stolen and I I want to make sure my software is protected, my computer is my software is protected, my computer is my software is protected, my computer is protected, etc. But when it comes to my protected, etc. But when it comes to my protected, etc. But when it comes to my health, health, health, I don't want the same things to happen. I don't want the same things to happen. I don't want the same things to happen. I don't want people to go, "Oops, yeah, I don't want people to go, "Oops, yeah, I don't want people to go, "Oops, yeah, we we kind of made healthcare worse we we kind of made healthcare worse we we kind of made healthcare worse there for a few years." I don't want there for a few years." I don't want there for a few years." I don't want that. that. that. So, what can we do? So, what can we do? So, what can we do? Number one, Number one, Number one, we need to push for consequences for bad we need to push for consequences for bad we need to push for consequences for bad AI medical advice. AI medical advice. AI medical advice. If [snorts] If [snorts] If [snorts] a company like Google or Bing or any a company like Google or Bing or any a company like Google or Bing or any other company is going to give out other company is going to give out other company is going to give out medical advice, medical advice, medical advice, there needs to be consequences for it there needs to be consequences for it there needs to be consequences for it being wrong.

being wrong. being wrong. Now, Now, Now, if you have and that's this is hard to if you have and that's this is hard to if you have and that's this is hard to do because, you know, if if I have a do because, you know, if if I have a do because, you know, if if I have a blog where I say, "This is what I think blog where I say, "This is what I think blog where I say, "This is what I think about about, you know, medicine. I think about about, you know, medicine. I think about about, you know, medicine. I think that we should all go back to eating that we should all go back to eating that we should all go back to eating hay. I don't know. It's horrible hay. I don't know. It's horrible hay. I don't know. It's horrible example. But, you know, if I said that, example. But, you know, if I said that, example. But, you know, if I said that, there's no consequences for me other there's no consequences for me other there's no consequences for me other than people saying, than people saying, than people saying, "You're a little loopy." right? "You're a little loopy." right? "You're a little loopy." right? Um Um Um but when an AI says, "This is what you but when an AI says, "This is what you but when an AI says, "This is what you should do." it's giving out medical should do." it's giving out medical should do." it's giving out medical advice. advice. advice. And there should be some consequences And there should be some consequences And there should be some consequences for that. for that. for that. Because if it pointed you to medical Because if it pointed you to medical Because if it pointed you to medical organizations who say, "This is what you organizations who say, "This is what you organizations who say, "This is what you should do based upon the information we should do based upon the information we should do based upon the information we have." you could read more about it. You have." you could read more about it. You have." you could read more about it. You could You could verify the sources. You could You could verify the sources. You could You could verify the sources. You could, you know, look to see if you could, you know, look to see if you could, you know, look to see if you trust those those hospital or doctors or trust those those hospital or doctors or trust those those hospital or doctors or or whoever did the study. And you could or whoever did the study. And you could or whoever did the study. And you could look at the study and say, "Hey, it was look at the study and say, "Hey, it was look at the study and say, "Hey, it was a study of 20 people. That's not really a study of 20 people. That's not really a study of 20 people. That's not really a great study." versus that study a great study." versus that study a great study." versus that study studied 5,000 people and that seems more studied 5,000 people and that seems more studied 5,000 people and that seems more consequential. Um consequential. Um consequential. Um this was, you know, double-blind and all this was, you know, double-blind and all this was, you know, double-blind and all the rest. You can make those decisions the rest. You can make those decisions the rest. You can make those decisions if you have the information and if if you have the information and if if you have the information and if you're basing it off of people you you're basing it off of people you you're basing it off of people you trust.

trust. trust. But the AI hides all that. So, But the AI hides all that. So, But the AI hides all that. So, I think we should need to push for more I think we should need to push for more I think we should need to push for more consequences for bad medical advice from consequences for bad medical advice from consequences for bad medical advice from AIs. AIs. AIs. Number one. Number two, Number one. Number two, Number one. Number two, we need to use existing data protection we need to use existing data protection we need to use existing data protection laws to their full extent. There are laws to their full extent. There are laws to their full extent. There are laws in the book that can help with some laws in the book that can help with some laws in the book that can help with some of this and we need to use those to of this and we need to use those to of this and we need to use those to their full extent. We can't back off and their full extent. We can't back off and their full extent. We can't back off and say, "Well, yeah, but it's LLM say, "Well, yeah, but it's LLM say, "Well, yeah, but it's LLM companies. They're too big." We We got companies. They're too big." We We got companies. They're too big." We We got to be able to hold their feet to the to be able to hold their feet to the to be able to hold their feet to the fire and say, "No, if you're going to be fire and say, "No, if you're going to be fire and say, "No, if you're going to be in this field, you need to have some in this field, you need to have some in this field, you need to have some level of accountability." level of accountability." level of accountability." In too many ways, we allow these LLMs to In too many ways, we allow these LLMs to In too many ways, we allow these LLMs to be unaccountable to anyone. And that's be unaccountable to anyone. And that's be unaccountable to anyone. And that's not okay. Because they're causing not okay. Because they're causing not okay. Because they're causing problems both personally as well as problems both personally as well as problems both personally as well as corporately in healthcare. corporately in healthcare. corporately in healthcare. Number three, Number three, Number three, don't sign your rights away. So, don't sign your rights away. So, don't sign your rights away. So, doctors' offices more and more are are doctors' offices more and more are are doctors' offices more and more are are using AI systems for note-taking. using AI systems for note-taking. using AI systems for note-taking. But legally at least in the US, and I But legally at least in the US, and I But legally at least in the US, and I believe in Europe and other places, um believe in Europe and other places, um believe in Europe and other places, um they have to give you the option of they have to give you the option of they have to give you the option of opting out.

opting out. opting out. Do so. Do so. Do so. Because first of all, we've seen those Because first of all, we've seen those Because first of all, we've seen those AI note takers aren't always going to be AI note takers aren't always going to be AI note takers aren't always going to be perfect. perfect. perfect. They're going to be less than ideal, and They're going to be less than ideal, and They're going to be less than ideal, and if that happens, if that happens, if that happens, then you may be then you may be then you may be you know, you may be given wrong it you know, you may be given wrong it you know, you may be given wrong it wrong medicine, you may be not given wrong medicine, you may be not given wrong medicine, you may be not given medicine you shouldn't given, you may medicine you shouldn't given, you may medicine you shouldn't given, you may not be diagnosed something you should be not be diagnosed something you should be not be diagnosed something you should be diagnosed for, you may have information diagnosed for, you may have information diagnosed for, you may have information your file that could endanger you in your file that could endanger you in your file that could endanger you in other ways other ways other ways that could, you know, put your children that could, you know, put your children that could, you know, put your children at list risk or other things where if it at list risk or other things where if it at list risk or other things where if it comes out, well, you know, this person comes out, well, you know, this person comes out, well, you know, this person is, you know, mentally unstable. Why? is, you know, mentally unstable. Why? is, you know, mentally unstable. Why? Because the AI hallucinated that. Because the AI hallucinated that. Because the AI hallucinated that. That's not okay. That could put you in That's not okay. That could put you in That's not okay. That could put you in real danger. So, real danger. So, real danger. So, don't sign your rights away. don't sign your rights away. don't sign your rights away. Also with that, a lot of these systems Also with that, a lot of these systems Also with that, a lot of these systems don't work on site. don't work on site. don't work on site. So, if you have a doctor doing AI note So, if you have a doctor doing AI note So, if you have a doctor doing AI note taking, taking, taking, probably what they're doing is recording probably what they're doing is recording probably what they're doing is recording that and sending it off to a third-party that and sending it off to a third-party that and sending it off to a third-party facility. facility. facility. Do you want that?

Do you want that? Do you want that? Do you want your health care information Do you want your health care information Do you want your health care information sent off to a place where sent off to a place where sent off to a place where it may be AI is scanning that, it may be it may be AI is scanning that, it may be it may be AI is scanning that, it may be people listening to it. people listening to it. people listening to it. I don't want that. Okay? So, don't sign I don't want that. Okay? So, don't sign I don't want that. Okay? So, don't sign your rights away. Number four, your rights away. Number four, your rights away. Number four, let's make AI usage painful let's make AI usage painful let's make AI usage painful in the health care industry. in the health care industry. in the health care industry. I really think we need to make it I really think we need to make it I really think we need to make it painful. Because if we make it too easy painful. Because if we make it too easy painful. Because if we make it too easy to just use AI wherever you want in to just use AI wherever you want in to just use AI wherever you want in health care, we're going to have the health care, we're going to have the health care, we're going to have the same degradation in experience as the same degradation in experience as the same degradation in experience as the software industry has had, as a lot of software industry has had, as a lot of software industry has had, as a lot of other industries have had. And the other industries have had. And the other industries have had. And the problem is, it's not something we go, problem is, it's not something we go, problem is, it's not something we go, "Whoopsie, we've got a problem." It's "Whoopsie, we've got a problem." It's "Whoopsie, we've got a problem." It's "Whoopsie, you now have cancer." Or, "Whoopsie, you now have cancer." Or, "Whoopsie, you now have cancer." Or, "Whoopsie, we didn't catch that cancer "Whoopsie, we didn't catch that cancer "Whoopsie, we didn't catch that cancer in time." Or, "Whoopsie, we gave you the in time." Or, "Whoopsie, we gave you the in time." Or, "Whoopsie, we gave you the wrong medication, now you have an wrong medication, now you have an wrong medication, now you have an allergic reaction. Like, these are all allergic reaction. Like, these are all allergic reaction. Like, these are all major problems that can happen. major problems that can happen. major problems that can happen. We need to make sure that we make it We need to make sure that we make it We need to make sure that we make it painful for AI to painful for AI to painful for AI to be in the medical space. Not that you be in the medical space. Not that you be in the medical space. Not that you don't want anything in the medical don't want anything in the medical don't want anything in the medical space, space, space, but because we want to make sure that but because we want to make sure that but because we want to make sure that it's used properly and used in a way it's used properly and used in a way it's used properly and used in a way that is going to be that is going to be that is going to be beneficial to us long term, not just beneficial to us long term, not just beneficial to us long term, not just rushed in the door because it makes rushed in the door because it makes rushed in the door because it makes those companies extra money. Cuz right those companies extra money. Cuz right those companies extra money. Cuz right now it feels like everything is being now it feels like everything is being now it feels like everything is being rushed.

rushed. rushed. We should not be rushing our health We should not be rushing our health We should not be rushing our health care. care. care. Our health care should be Our health care should be Our health care should be something where we we really make sure something where we we really make sure something where we we really make sure we do it right the first time. we do it right the first time. we do it right the first time. We shouldn't be moving fast and breaking We shouldn't be moving fast and breaking We shouldn't be moving fast and breaking things when it comes to health care. things when it comes to health care. things when it comes to health care. All right? Thanks for listening. As All right? Thanks for listening. As All right? Thanks for listening. As always, I am Tim Corey.