* The following transcript is computer generated and may contain errors.

Jared Johnson (eHealthcare Strategy & Trends): Hello, everyone, and welcome to the second session of Day Three of AI & Healthcare Marketing Week. This session is AI Hallucinations in Healthcare: The Hidden Risk to Your Brand, Compliance, and Patient Trust. I'm Jared Johnson. I'm host of the Healthcare Rap Podcast, and I'm also a member of the eHealthcare Strategy & Trends Editorial Advisory Board. I'll be your moderator for today's presentation.

I want to give a couple of housekeeping details before we begin, and then we'll dive right in.

First, today's discussion will be approximately 45 minutes, followed by a 10- to 15-minute Q&A. To submit questions, you can do that at any time during the presentation. Just type them into the control panel and hit send. If it's for a particular panelist, feel free to indicate that when you submit your question.

We'll tend to hold questions until the end. There might be times during the presentation where it makes sense to interject them, but for the most part, we'll hold them until the end. So feel free to submit them at any time, and then we'll get to them during that last 15 minutes or so.

Today's session is being recorded, and you'll receive a link to access the recording as soon as it's been processed and is available. With that, let's dive into our topic today.

I do want to set the stage, and then I'll introduce our panelists. This session today is all about how AI is rapidly becoming the front door to healthcare. We all know how it's shaping how patients are discovering providers, evaluating treatment options, and making care decisions.

But as LLMs are increasingly sitting between your organization and your audience, there's this new, largely invisible risk that's emerging: AI hallucinations.

These are inaccuracies that can misrepresent your services, distort clinical information, and even fabricate details about your organization, which puts your brand reputation, your regulatory compliance, and patient trust at stake.

So in this session, we're going to unpack how and why AI hallucinations occur in healthcare, why the risk is disproportionately high in this industry, and what it means for marketing, compliance, and digital strategy leaders.

We'll explore how organizations can proactively identify where AI is getting their brand wrong. We'll draw from real-world experience and emerging tools, including LLM monitoring systems. That's one of the things I'm looking forward to hearing more about. These systems are designed to detect hallucinations and assess sentiment at the service-line level.

Personally, what I'm hoping to get out of this is not only learning about those monitoring systems that I mentioned, but also understanding how this impacts a patient—somebody who's looking for care.

We'll get into all of that, but let me introduce our panelists.

First, we have Stewart Gandolf, CEO of Healthcare Success; Brandon Schakola, Senior Director of Digital Services for Healthcare Success; Pansy Lee, Head of Product at inlets.ai; and finally, John Davey, Vice President of Marketing Technology for Mount Sinai Health System.

I want to thank all our panelists for joining today. To give our attendees as much of your time as possible, I won't read your full bios, but they are available on the website.

With that, I'll turn it over to Stewart and Brandon, and we'll go from there.

Stewart Gandolf (Healthcare Success): Excellent. Thank you, Jared. I'm super excited to be here today.

Brandon, are you driving the slide deck?

Today we're going to talk about AI hallucinations, and I know this is a topic many of you are interested in. It's certainly one that has our team's rapt attention. AI right now is ruling the land, of course, as you know, and we're seeing a lot of fear and a lot of concern—rightfully so—about whether AI is telling patients the right things or the wrong things about our organizations.

I think, unfortunately, I'll vindicate your concerns because, yes, it probably is telling your patients the wrong things about your organization. What are competitors gaining because of misrepresentation? Probably. Do you even have an idea of what AI is saying about your organization? Maybe not. These are valid concerns.

Sometimes AI gets it right. We know search has changed forever, starting around March of last year. People saw their website traffic crash as AI began taking over and becoming a bigger and bigger part of everything.

One of the things we talk a lot about with clients, and when we're speaking at events like this, is that Google used to serve up ideas about who to call. Now AI is giving recommendations about who to call.

We'll also talk about the hallucination side of this, which is really about patient safety. There are some scary things happening, and I'm pretty sure Brandon, John, and Pansy will be talking about some of the safety issues that we're seeing today.

Next slide, Brandon.

As we talk today, we're going to cover the problem. I've already set it up a little bit, but we'll discuss what's at stake for your brand, your compliance, your patients, and your communities. Many of you are serving the broader community.

We'll also talk about what makes AI so problematic—and so different—in a healthcare setting, because the stakes in healthcare are much higher than buying shoes. The fact that hallucinations occur here can be quite troubling to us, and I'm sure to you as well.

The good news is that, while I've set up the problem, there are tools out there. There are solutions, and we'll be giving you some things you can walk away with today to improve how you grapple with this thorny—and growing—problem.

Next slide.

One of the key concepts we talk about a lot at Healthcare Success is that when AI first burst onto the scene in search, the common wisdom was, "Well, you just have to be everywhere."

Not only is that impossible, but where do you even start? I found that explaining AI visibility in this context really resonates with audiences, so I thought I'd take a moment to share this because it's important as we think about AI visibility.

When I talk about AI visibility, I'm referring to AI Overviews, ChatGPT, Perplexity, Claude, and the other LLMs. Think of it this way: everything begins with the foundation—technical SEO.

Technical SEO has always mattered, but now it matters even more and in very different ways. Things like schema and the Knowledge Graph are much more important than they used to be. The way you structure your website was important before, but now it's even more important because it needs to make your information readily available and easy for LLMs to find.

The same thing applies to content. Content was always important, but now it's important in different ways. For example, you've all heard about FAQs by now, but think about how you organize around a topic with blogs.

Instead of having a bunch of small, disconnected blog posts, the machines are looking for clear information, delivered quickly, with depth and authority. So again, it's about building your content differently. Local SEO matters in its own way, especially for providers. Reputation and trust matter beyond just ratings.

One key point here is digital PR and off-site authority. That matters a lot more than it used to because you need a broader network beyond your own website.

Remember, LLMs are all about choosing and recommending. They're looking for trust at the end of the day. That's probably the common theme you'll hear throughout today's discussion. How do we take all of this and turn it into trust?

Finally, there's the issue of the brand. All of these things ladder up to your brand—but not your brand guidelines, colors, fonts, or logo. Those matter too, but what we're really talking about is the brand that the LLMs perceive. That's the challenging part because AI services may be interpreting your brand in ways that are both surprising and, sometimes, disturbing.

So, quickly, this is a fun fact. I did this prior to today's meeting and asked, "How much does Healthcare Success cost?" Just for fun. I actually asked about this about a month ago, and I remember thinking, "Ooh, I'm really nervous about this," because the answers were all over the board. Most of the LLMs were completely wrong.

We've been updating content and creating content around this topic. As recently as several weeks ago, a couple of LLMs were saying our fees begin at $500 a month, which isn't accurate. We don't work with individual practices, and our fees certainly don't begin at $500 a month. It's really important to monitor this.

This answer from ChatGPT is closer than what a lot of the other models returned. It's pulling information from our website and other sources, trying to assemble an answer. These are exactly the kinds of things you should be checking. Whatever information is most important to your organization, you want to make sure AI is getting it right.

Next slide, Brandon.

One last point I'll make is that LLMs will simply make things up if they don't have enough information. Here's a fun example. Burger King apparently had a new burger called the "Baby Burger." The LLM went to Burger King's website, couldn't properly read the information, and then started pulling content from all kinds of different places—steakhouses and other restaurants—trying to figure out what a Baby Burger was.

As you can see from the example, it created a confusing mess. That's a cute example when you're talking about burgers. It's not nearly as funny when you're talking about healthcare.

With that, I'll turn things over to Brandon.

Brandon Schakola (Healthcare Success): Yeah, thank you.

What we have to understand is that this is fundamentally an information retrieval problem.

If Burger King isn't present as the authority on its own brand at the time of retrieval, these models are so aggressive that they'll simply run out and try to assemble an answer from anywhere because they have this tendency to want to please the user. This one didn't do such a great job.

We also have to remember that, unlike rankings in traditional search results, AI Overviews aren't stable. They're just that—an overview. They're a snapshot at a particular moment in time, not a permanent ranking or certainty.

One example we ran across involved an emergency room organization. When we started evaluating what ChatGPT knew about them, ChatGPT essentially warned users, "You may want to put some protection on before you visit this website." That obviously doesn't bode very well if you're an emergency room and someone is in that moment of truth where they urgently need care. In this case, ChatGPT couldn't confidently summarize the organization.

Perplexity would surface it as a source if someone searched for an emergency room in that particular state, but it didn't have any preloaded facts about the brand. Google AI Overviews posed the risk that the organization might not appear at all because of missing schema. Claude, which tends to be a little more cautious, described the facility based on crawled content but didn't have much confidence in what it found.

What we eventually discovered was happening behind the scenes was that, because there wasn't enough cohesion around the brand itself, it essentially got stored in vector memory alongside anything else with a similar name.

There was another interesting issue involving "ER" versus "wear." The organization became associated with a website that redirected visitors to a boutique through an Irish subdirectory—a classic signal of a Chinese drop shipping operation—which was generating low-trust signals on sites like ScamAdviser. So this organization had accumulated a significant amount of technical debt without even realizing it. This kind of thing can happen much more broadly if you're not paying attention to your brand as a dataset.

All of this is happening at the same time we're hearing about GPT Health and the idea that we're simply going to upload healthcare information into these models and let them process it. One study from Nature found that GPT Health under-triaged emergencies about 51 percent of the time. If you already have a coin flip's chance of surviving an emergency, are you really willing to add even more uncertainty to that? Probably not.

Why is this especially dangerous? There are huge portions of the United States that depend on emergency departments and urgent care because they're located in healthcare deserts where those facilities may be the only care available.

When someone is buying shoes or searching for a burger, trust is usually the last step before making a purchase. Healthcare works differently. People begin with trust. One of the biggest problems we're facing right now is that trust is eroding.

Most of you are probably familiar with Google's E-E-A-T framework—Experience, Expertise, Authoritativeness, and Trustworthiness. That makes this an even steeper uphill battle for healthcare organizations.

With that, I think I'll turn things over to John Davey so he can frame some of the problem from his perspective.

John Davey (Mount Sinai Health System): Yeah, I'll start by saying these tools are also helping. I want to establish that fact because we're going to spend today talking about hallucinations and some of the terrifying outcomes they can create in healthcare. I'll speak for myself—I lose sleep over this.

At the same time, we know healthcare has historically been very bad at making our services easy to find, easy to access, and easy to purchase for prospective patients. These tools make that easier in many ways. As Stewart mentioned earlier, sometimes they work very well. Where they don't work well is around hallucinations involving diagnoses, treatments, and clinical information.

That's very troubling from the perspective of a health system. It's troubling for the world at large, and it's troubling for patients themselves. When you focus on trust, these hallucinations contribute to a decline in trust that has already been happening over time.

That lack of trust really accelerated during COVID. We saw it in our own survey data. We subscribe to NRC data, and because Mount Sinai Health System operates in New York City—a highly competitive market with incredible clinical care options—we've been able to watch those trends very closely.

We like to think all of the major health systems helped save New York City during COVID, and during that period there was tremendous trust in healthcare. But shortly afterward, we saw that trust begin to decline. Public awareness of our brands didn't change. People still knew who we were. What changed was that people began questioning us.

Much of that questioning was driven by questionable information they were finding online, even before today's AI tools and their hallucinations entered the picture.

You can see here on this chart there have also been some recent political injections into the public discourse around healthcare. Without getting into the politics of this, they've accelerated the erosion of trust.

That's the reality. We wish it weren't the case, but it is. These tools are adding to that lack of trust. We've built, as an academic medical center, service lines populated by world-class experts in virtually every discipline under the sun.

Those experts have dedicated their entire lives and careers to helping patients, so it's especially challenging for them when they see patients coming in asking questions based on AI hallucinations—or, even worse, patients choosing not to come in at all because they've pursued other options based on questionable information. It's a giant issue for us. As we're seeing here, the trust signal is eroding.

Put yourself in the shoes of a patient. We're all patients. Often we don't know where to go, so we turn to these tools. These tools, as we're learning, are designed to tell you what they think you want to know, even when that information isn't accurate. You can also see here data from a Mount Sinai and Lancet Digital Health survey. Even clinicians themselves are using AI tools to support clinical decisions, patient journeys, and treatment planning.

Among those clinicians doing research, a significant percentage have encountered AI hallucinations. The difference is that clinicians are healthcare experts. They can recognize when something is a hallucination. Patients generally can't. You can see the next data point there—51.3 percent—which represents the hallucination-free rate for medically specialized AI. That's about half. The 76.6 percent figure is for general-purpose AI.

You can also see the percentage of large language models that accepted fabricated claims. Nearly a third of the cases involved these tools being misled by the data they were accessing. That's incredibly disturbing. Again, these are the things that keep us up at night. That last statistic—that 50 to 90 percent of large language model responses aren't fully supported by cited sources—is especially concerning. This is really the slide that's meant to scare all of us. It truly is a giant problem.

For health systems like ours, and for the physicians I represent and have the privilege to work with, it's a frightening time. At the same time, many of our physicians aren't even aware this is happening. Part of surfacing this information is helping people understand how our brand—and, in our case, our healthcare services and every service line—is being perceived through these AI tools.

I'll speak for ourselves. Among our 9,000 physicians, there are many who aren't yet fully aware of how much the world has changed and how prospective patients are now thinking differently.

If you think about the change curve, there are even some providers at the far end of that curve who are still reluctant to accept this because it's understandably so disruptive to the worldview they've spent their entire careers building.

This slide really sets up the problem. The first step in overcoming any problem is acknowledging that you have one. Then you move on to asking what can be done about it, how to measure success correctly, and what potential solutions exist.

Brandon Schakola (Healthcare Success): One of the interesting takeaways here is that the medically fine-tuned models were actually performing worse. There's something about clinical language that allows them to be manipulated more easily. It's not just a social media phenomenon that's driving this.

John Davey (Mount Sinai Health System): Absolutely. This brings us to what we call the clinical prose paradox. The guardrails these AI systems have are sometimes bypassed by authoritative-sounding language. They can be tricked. We're already seeing the results of this in some very alarming ways.

The intuitive assumption is that AI tools are designed to catch misinformation. The most obvious source of misinformation is social media. You can see some recent examples here, including hantavirus, Ebola, and of course everything that's still lingering from COVID. Our physicians have spent years trying to educate the public and correct misinformation surrounding COVID. Many of our physician providers regularly serve as media sources specifically to correct inaccurate information that's circulating publicly.

Unfortunately, these AI tools are also accessing misinformation and finding sources of dangerous content that provide answers to questions in ways that can ultimately harm—or worse—the people searching for them.

These are examples taken across different platforms. I won't walk through every one of them, but the first example on the left involved a prediction based on a 2022 post claiming that a 2026 hantavirus outbreak had somehow been foretold.

Again, these stories get picked up by major media outlets, and suddenly people begin asking whether it's prophecy or coincidence. These AI systems are simply trying to predict the next word or the next concept to answer a user's question, and that's creating real problems. The response to that claim, of course, came through dedicated fact-checking organizations like Snopes, as well as direct responses from the World Health Organization.

The anti-vaccine narrative is another giant example. We've watched this narrative continue to build. Being respectful of everyone's political viewpoints and belief systems, and sticking strictly to the facts—which is our responsibility in healthcare—we've seen influencers publish content that isn't based on evidence or data while building very large audiences.

Some of these AI systems end up oversampling that information. One example claimed that an outbreak was simply a pretext to push a Moderna vaccine and that antiviral treatments had failed. Within days, that narrative completely collapsed because its central premise simply wasn't true.

TikTok has become another significant source of misinformation feeding these systems. One CBS news clip received more than seven million views and over half a million likes within 24 hours. The reporting itself wasn't false, but the way influencers framed it around "plague ship" narratives dramatically amplified public alarm. There were elements of truth that became exaggerated into conclusions that simply weren't supported by the facts.

Another example involved claims that ivermectin should work against an RNA virus. Again, that was medically baseless. Experts from NewsNation, the CDC, and infectious disease specialists all confirmed there was no effective antiviral treatment in that case and that supportive care remained the appropriate treatment.

The point of all these examples isn't to debate the individual issues. It's to demonstrate that when medically unsupported—or partially supported—information exists online, these AI tools can pick it up, blend it together, and then we have to find ways to combat the misinformation they generate.

Brandon Schakola (Healthcare Success): Exactly.

We're living in a different world now. We used to think primarily in terms of search. Even social media functioned largely through search—you looked for influencers or people who knew something. Today, people ask AI complete questions.

They'll search for something like, "Mohs surgery for skin cancer near me." That's fundamentally different from traditional search. AI search depends on a number of underlying systems. Every time someone enters a prompt or asks a complex healthcare question, AI breaks that request apart into dozens—or sometimes hundreds—of subqueries.

This example shows someone searching for Mohs surgery or skin cancer treatment in a specific geographic area. What you're seeing here is activity by landing page, showing hundreds of AI-generated retrieval requests over a period of time. Search hasn't disappeared. If anything, the complexity of search has multiplied exponentially.

One important thing to notice is that rankings don't work the same way anymore. It's no longer just about being in Position One. AI may pull information from anywhere in the first hundred search results. That means hallucinations can be multiplied across search engines, social media, and AI systems alike.

We have to be vigilant about identifying those hallucinations, understanding what's causing them, and figuring out what we can actually do about them. Most organizations simply aren't looking. Most people aren't looking.

Over the past nine months or so, I've been working closely with Pansy Lee and Emily Clark over at inlets.ai—formerly known as Pedal AI. We've been developing a solution to help address these challenges.

People don't search for healthcare the same way they search for shoes. Trust is a huge factor, but healthcare search is also location-based in many cases. Sometimes people are searching for providers, sometimes for service lines, sometimes for facilities. That's a completely different challenge than trying to sell the latest Nike or Jordan shoe.

As we worked with Pansy, we realized we needed to take a principles-first approach. So I'd like to introduce Pansy so she can talk about how we've been thinking through this.

Pansy Lee (inlets.ai): This is our methodology.

We always start with the persona because we want to replicate the patient journey as closely as possible so organizations can see what patients are actually seeing. From there, we build out query libraries. What are the questions a patient is likely to ask around a particular service line?

In this example, it was dermatology. We want organizations to see how they're showing up throughout that journey—from the patient's perspective, through all of those questions. We run those queries across the major LLMs, including ChatGPT, Google Gemini, and Google AI Overviews, and then we begin scoring that visibility over time.

We also compare your organization—your hospital or clinic—to other hospitals and clinics that a patient might be considering. We even compare individual physicians and providers at your organization to other providers that patients might also be evaluating. Ultimately—

Brandon Schakola (Healthcare Success): Sorry to interrupt. One thing I wanted to point out is why it's important to start with personas. A lot of other tools—and there are some really good ones out there, like Profound, Waikay, and AirOps. They're built by really smart people.

But many of them approach this like traditional SEO rank tracking. Organizations upload thousands of prompts, and what comes back is mostly noise. Or the recommendation ends up being something like, "Go spend more time on Reddit." We didn't find that particularly helpful.

What we've discovered by using personas is that we tend to achieve roughly 85 percent greater accuracy for the questions that actually matter. You don't have to boil the ocean. That became an important design principle in how we built this.

Pansy Lee (inlets.ai): Exactly. On the next slide, that's what we did. We took just a small set of queries across three different personas and monitored them over time.

It's important to track the same persona asking the same question over time because you're watching how those conversations—and your category ownership—evolve. Another point I'll make is that we believe it's important to monitor every day instead of weekly or monthly. As you can see in the visibility chart, there are peaks and valleys.

If you only check once a week or once a month, you might catch the LLM on a bad day and send your entire team down a rabbit hole trying to solve a problem that was simply temporary. We're constantly measuring an organization's share of voice and how highly it ranks across AI responses. We combine those measurements into what we call category ownership.

Brandon Schakola (Healthcare Success): More importantly, we weren't just looking at a brand, a topic, and a persona. Different people across an organization use different language. Different audiences ask questions differently. We wanted to replicate—or at least approximate—that variety. We also wanted to focus on very specific target markets.

In this example, we were looking at a particular city—Clifton. That was another important finding. You can become incredibly focused in this process.

Pansy Lee (inlets.ai): Next, we have what we call the competitive comparison.

For each query, we're looking at where your organization appears within the AI rankings. For example, someone newly diagnosed with skin cancer might be searching for a highly skilled Mohs specialist. Unfortunately, for this particular organization, they weren't mentioned in the AI response at all. Later, you'll see in our Action Center that we actually surface specific recommendations you can implement to close those gaps for particular personas and queries.

Brandon Schakola (Healthcare Success): As we started looking across this multi-location dermatology practice in New Jersey, we noticed patterns emerging. Just as we do in SEO, we were able to group these issues into themes. If multiple locations shared the same underlying issue, we knew we could solve it structurally across the organization. We could address visibility gaps. If third-party coverage was missing, we could build a program around local citations or provider citations.

Other issues pointed us toward on-site content improvements or enhancements to Google Business Profiles. One important concept here is connecting the real world to your schema markup and other authoritative sources. You're essentially building guardrails so that when an AI system starts to get lost, it has enough authoritative signals to guide it back toward the correct answer. A couple of hallucination examples stood out.

The first involved what the AI believed was a broken Mohs surgeons page. As Pansy and I dug into the responses, we discovered there actually was a Mohs surgeons page. The hallucination wasn't that the page was broken. The hallucination was the URL itself. The LLM had completely invented a URL structure that had never existed. We checked the Wayback Machine and older versions of the website. It had simply made it up.

Another issue involved physician authority. Many providers didn't have enough information available on third-party platforms like Zocdoc, Healthgrades, or Ratings MD for the AI to confidently recognize them as real-world entities. Meanwhile, competitors did have that validation. As a result, those competitors were being recommended instead.

Another interesting example involved a physician named Robert Cantor. The AI claimed his provider biography was missing. The reality? He had never been part of that dermatology group. The model had simply confused him with a physician from an entirely different dermatology practice because the branding across practices was similar enough to create confusion.

Maybe that's just an unfortunate reality of dermatology branding. We'll joke about that later.

Beyond individual physicians, we can also evaluate locations themselves. We can understand how AI is recommending different parts of your organization. Is your care considered comprehensive? Does AI recognize your technology? How does it perceive patient satisfaction? Does it believe your surgeons are recognized experts?

In one example, expertise around Mohs surgeons was being assembled from one page on the organization's website and another page from a completely different organization whose brand happened to be similar. Those organizations were effectively occupying the same conceptual space inside the model's memory.

We can also drill down to individual providers. In one case, we found that a physician's Mohs specialization wasn't even being pulled from the organization's own website. It was coming entirely from another source somewhere else on the web. If you're responsible for a healthcare brand, that should terrify you.

Pansy Lee (inlets.ai): Actually, if we go back to the previous slide for just a second... This is our competitive sentiment matrix. As Brandon mentioned, we're measuring several different attributes that both patients and LLMs appear to consider when comparing healthcare organizations. At the bottom of the chart you'll see links labeled "LLM verified from." Those are the sources the models are using to validate claims about your organization and your competitors.

One interesting thing we've found is that many AI optimization tools overemphasize Reddit as the primary source of information. Healthcare is different. LLMs aren't relying on Reddit nearly as much. If you look at these citations, they're pulling information from healthcare-specific platforms like Healthgrades and Zocdoc.

That means it's important to strengthen your presence and your relationships with the healthcare-specific sources that LLMs actually trust when they're making recommendations.

Brandon Schakola (Healthcare Success): That's a really excellent point. So, it's great that you have all of this information, but the next question becomes: what do you actually do with it?

I'm very much an in-the-trenches SEO and AI optimization person. It's easy to find things to worry about, but we need a practical way to solve these problems. Being able to take all of the issues we identify across different locations and providers, and understanding how those signals work together as a graph—very much like internal linking in SEO—is incredibly important.

There are signals associated with entities, names, locations, and physical places. What we found during this pilot was that 11 of the issues we identified affected three or more locations. That's great news because we can solve those structurally. We know we can address visibility gaps. We know we can improve third-party coverage by working with partners who specialize in local citations or provider citations. The remaining issues may involve on-site content, Google Business Profiles, and other optimization work.

Just as importantly, we need to connect real-world entities through schema markup and authoritative sources. You're essentially building a harness so that when AI begins drifting off course, you can guide it back toward the correct entity. It's not this Bill Frawley—it's this Bill Frawley.

As we continued working through additional pilots, we wanted to understand what happens when an organization has both local and national visibility. Pansy, maybe you can walk through the methodology we used with Mount Sinai.

Pansy Lee (inlets.ai): For Mount Sinai, we started the audit the same way we start every audit. We looked at searches like "cancer specialists in New York." Naturally, organizations like Mount Sinai, NewYork-Presbyterian, NYU Langone, and Columbia appeared. But then we started thinking about how patients actually search.

Cancer is different. Patients may be willing to travel or even stay overnight if they believe they're getting access to the best specialist. So we expanded the audit beyond New York.

That's when we discovered that Memorial Sloan Kettering consistently appeared as the number one recommendation for many different cancer types, advanced therapies, and experimental treatments. John, I know you previously worked at Memorial Sloan Kettering, and I believe you also worked at NYU Langone, so you obviously did a great job while you were there.

Now you have to compete against that. When we looked at the competitive visibility audit, we immediately saw that Mount Sinai had opportunities to improve. In several cases, it ranked third or fourth, and in others it wasn't mentioned at all.

On the next slide, we begin breaking down the attributes that determine those AI rankings. These are all outstanding hospitals. As John mentioned earlier, New York has incredible healthcare. Many of these organizations score within only a few points of each other because they're all high performers.

But AI has to rank them. It has to prioritize one over another. Sometimes even a difference of one-tenth of a point determines whether you're ranked first or second. This radar chart is especially helpful because it quickly highlights the attributes that deserve attention.

For Mount Sinai's cancer program, patients—and the LLMs—were looking closely at advanced technology, clinical outcomes, physician expertise, and research innovation. Those insights can directly influence your content strategy. For example, Mount Sinai might choose to invest more heavily in communicating its research efforts to help close that gap.

Brandon Schakola (Healthcare Success): In this case, there's also the way these websites are structured. Many large academic medical centers have multiple research subdomains. Sometimes those become fragmented. I know there have been ongoing efforts to consolidate some of those. John?

John Davey (Mount Sinai Health System): I'll jump in because this is a really good example. This is actually a double challenge for Mount Sinai.

Even before AI, we had to compete with Memorial Sloan Kettering Cancer Center, one of the greatest cancer hospitals—and cancer brands—in the world, particularly here in New York. That has always been a challenge, not only for us but for every health system offering world-class cancer care. Now we've layered AI on top of that.

These systems are accessing different sources of information and returning answers that make that competitive challenge even greater. As I think about everyone attending today's session, I'm in exactly the same position. I understand the problem. Now we have to figure out the concrete, actionable steps to address it.

You can't boil the ocean. Part of our responsibility is prioritization. We have to align these opportunities with the organization's business priorities. Cancer happens to be one of our highest priorities, as it is for many health systems because of both its clinical importance and the return on investment. For those of you wondering how to approach similar problems, yes, there are tools that help identify the issues. Yes, there are tools that suggest what should be done. But none of us have unlimited people, unlimited budgets, or unlimited time.

What we're doing—starting with cancer—is prioritizing. We're determining where we should begin. Even six months ago, many of our cancer leadership teams didn't fully understand this problem. Part of my team's responsibility is educating upward. We need leadership buy-in. They need to understand that this affects the business. If the business is being affected—and it is—we need resources dedicated to solving it. That's a critical part of the process.

Brandon Schakola (Healthcare Success): We didn't stop at evaluating locations. These are major academic medical centers with extremely competitive physicians. We also wanted to understand how those physicians themselves were performing. The blue entries represent Mount Sinai. As we started digging deeper, we became curious about a couple of physicians who were performing particularly well. AI was validating roughly six different authority signals from third-party sources. If it couldn't find information on the primary Mount Sinai domain, it would retrieve it elsewhere.

One physician who caught our attention was Dr. Liang. We discovered there was almost no third-party directory presence for that physician. Even more interesting, when we tried accessing the provider page that should have existed on the main website, we couldn't reach it. That immediately raised questions.

At its core, this is still a retrievability problem. Stewart introduced us to John, and John generously allowed me to begin working with his technical teams to understand what was happening. I tried crawling the site. I tried retrieving the information. I couldn't. The crawler was being blocked.

One of the first things we discovered is that organizations sometimes continue using older SEO rules for blocking bots. Those rules can unintentionally become too aggressive. Sometimes the issue isn't even happening there. It may be happening at the CDN—or content delivery network—where caching systems are interfering with retrieval. John has been incredibly generous in allowing us to work with his technical teams to investigate those issues.

There's another interesting factor here. I believe this particular physician profile had been caught in the middle of an A/B test involving different versions of provider pages. As a result, the URL we expected to crawl no longer existed. Once that testing is complete, redirects can resolve the issue.

It's another reminder that sophisticated marketing activities like A/B testing and multivariate testing can unintentionally create problems for AI retrieval if bots aren't handled correctly. When we reviewed the Action Center, the highest-priority recommendation was simple: Fix the access-denied issues.

Another thing to keep in mind is how aggressively these AI bots crawl websites. Googlebot isn't especially aggressive. Bingbot isn't either. But some AI crawlers are.

For one client, we saw Googlebot crawl the site around 5,000 times per month. OpenAI's crawler was closer to 50,000 requests during that same period. That's an order of magnitude difference. In some cases, ByteDance has been even more aggressive—and they don't even have a major large language model yet. Of course, they're behind TikTok.

Another area we've been looking at is solving the clinical language problem. Can we run automated checks to determine whether the language on your website actually connects with the intended personas and ideal customer profiles? That's important because if your content isn't speaking to the audiences you're trying to reach, AI isn't going to connect those dots for you. Pansy, do you want to talk a little bit about that?

Pansy Lee (inlets.ai): Because we're tracking specific personas, we're able to detect whether the content you've written is actually speaking to those audiences. When it isn't, we'll flag that you're missing opportunities with a particular persona because you don't have content that's written specifically for that audience. Those are exactly the kinds of insights you'll find inside the Action Center.

Brandon Schakola (Healthcare Success): Once we're able to identify those issues, we can group them into themes and attack them systematically. That allows organizations to prioritize instead of trying to fix everything at once. To John's point, every organization has limited resources, but when you can identify structural issues that affect multiple service lines, providers, or locations, you can make tremendous progress with relatively focused effort.

That brings us to what you can actually do about all of this. You may not have a sophisticated AI monitoring platform today, but you can still adopt this methodology and begin using it immediately. Start by detecting where AI is misrepresenting your organization. My background is in cognitive science, so I'm used to looking at black boxes and trying to understand how systems break down—not just accepting whatever comes out of them. The more you work through this process, the more intuitive it becomes to recognize where the problems are and how to prioritize solving them.

From there, identify the gaps. Look for outdated pages. Look for missing information. Develop a plan to optimize those areas and then verify whether your changes actually improve AI's understanding. One interesting thing about this new environment is that you can sometimes see changes very quickly. It might take two days. It might take two weeks. Sometimes it takes longer, but you can begin watching how your brand changes over time.

Focus on creating content that's easy for AI to cite. Continue improving your schema markup—especially medical schema, because, frankly, almost every healthcare organization I've worked with has room for improvement there. Make sure your physician profiles and location pages are accessible, and continue investing in authoritative content. If everyone has essentially the same page about Mohs surgery or breast cancer, then your content becomes a commodity. There's very little reason for AI to recommend your organization over someone else's.

Instead, think about what makes your organization unique. Case studies. Research. Success rates. Unique programs. Facts that differentiate you. Those are the kinds of signals that help AI understand why your organization deserves to be recommended.

Finally, determine who owns this work inside your organization. Marketing certainly plays a role, but so do compliance, legal, your web development team, your clinical leadership, and your physicians. This really is an enterprise-wide initiative.

A few practical things you can do this week: manually monitor your AI visibility by checking your top three service lines and your top three providers to see what AI is saying today. Look at how much variation exists across different LLMs. Check your robots.txt file to make sure you aren't accidentally blocking important AI crawlers. It may even be worth testing what happens if you temporarily remove some restrictions. Then audit your provider schema and service-line schema to ensure those signals are as complete as possible.

If you'd like additional help, you can always request a demonstration of the inlets.ai platform. It's designed to complement the tools you're already using. You can also work with us at Healthcare Success on an AI visibility audit. We believe these kinds of pilots can genuinely improve patient outcomes by helping organizations provide more accurate information through AI.

As Stewart mentioned earlier, much of this material is also covered in our AI Visibility Stack eBook. You'll receive that as a PDF after today's session.

With that, Jared, we'll open it up for questions.

Jared Johnson (eHealthcare Strategy & Trends): Thank you, panelists. Great presentation. There was a lot of information packed into that discussion. We've had several questions come in, so let's jump right into those. The first one isn't directed at anyone specifically.

"Is this just the tip of the iceberg? Do you think this will remain controllable as AI becomes more ubiquitous?"

Brandon Schakola (Healthcare Success): I have some thoughts, but Pansy, why don't you go first?

Pansy Lee (inlets.ai): I think this diagram really says it all. The foundation is still technical SEO and schema—making sure everything is connected correctly. Then you layer on content, local SEO, reputation, trust, and relationships with authoritative third-party sources. Brandon, I don't know if you'd add anything else.

Brandon Schakola (Healthcare Success): I think if organizations don't pay attention to that entire stack, then yes, this really is just the tip of the iceberg. You're always going to uncover additional issues as you continue digging.

These models are constantly being retrained, and they can actually lose information over time because of the way parametric memory works. Facts that were once present can disappear. If you aren't reinforcing those facts through authoritative sources before the next training cycle, you'll lose ground. Sometimes that means strengthening Wikidata. Sometimes it means Healthgrades, Ratings MD, or other trusted third-party resources. You have to take advantage of the recency bias built into these systems, and that's where digital PR becomes increasingly important.

Pansy Lee (inlets.ai): Beyond robots.txt files, schema, and content, our systems are actually able to identify roughly a dozen different visibility issues that contribute to LLM hallucinations and inaccurate representations of your organization.

Jared Johnson (eHealthcare Strategy & Trends): Great. The next question is: "Is one LLM worse than the others when it comes to hallucinations?"

Brandon Schakola (Healthcare Success): They all hallucinate differently. That's really the truth of it. Even local models and open-source models have different biases based on how they were trained and distilled. Perplexity tends to lean more heavily toward factual information, but it can also be influenced by search because, if it doesn't know something, it's going out to search the web.

The different models also rely on different search ecosystems. ChatGPT tends to lean more toward Bing, while Claude relies more heavily on Brave Search. Depending on which search stack they're using, they'll develop different biases and produce different results. From our perspective, you really need to monitor at least the major four models to understand how your organization is being represented.

One thing we've discovered—and there have been studies published on this very recently—is that even when you compare responses across the four major systems, you'll only find about eleven percent consistency between them. That's why you almost have to monitor multiple models simultaneously to triangulate what's happening and understand where the discrepancies are coming from.

Jared Johnson (eHealthcare Strategy & Trends): Great. We have one more audience question, and then I think Stewart has one as well.

This question just came in: "Healthcare websites aren't dying, as I keep hearing. They're just changing. Is that accurate? It seems like success is becoming much more dependent on the technical side of SEO."

Pansy Lee (inlets.ai): I'd definitely say healthcare websites aren't dying.

AI is actually trying to go to your website first. That's the first place it wants to retrieve information from. Because of that, it's really important not to block AI crawlers. Your website is still the surface you control. Once you block those crawlers, AI has to go elsewhere to find information about your organization, and those are places where you don't control the messaging.

John Davey (Mount Sinai Health System): From my perspective, that's a major question we're discussing internally as well.

Different leaders attend different conferences and sometimes come back asking, "If AI is answering questions, what's the point of having a website anymore? Why are we continuing to invest in it?"

The reality is that your website remains the primary source these AI systems use. We absolutely need to evolve how we structure websites and how we think about content, but they remain foundational. That's another area where we, as marketing leaders, have to continue educating upward throughout our organizations.

Stewart Gandolf (Healthcare Success): Let me just build on something Pansy mentioned earlier because I think it connects back to the Burger King example.

Do you really want to give up your vote on your own reputation? Do you want to simply abdicate that responsibility to everybody else?

John, I'd love to get your perspective because, from the health system side, your organization has always been such a leader in community health. This feels like it's much bigger than a marketing issue.

What we're hearing from many health systems is that this isn't always owned by marketing. Sometimes it's communications. Sometimes it's the medical office. Sometimes it's the C-suite because the issue extends well beyond branding. Marketing often gets pushed to the kids' table, but this is still a critical issue whether your health system has a large marketing budget or not.

Do you see that as well?

John Davey (Mount Sinai Health System): I know we're right at the end of our time, but that's a tremendous question.

One advantage we have at Mount Sinai is that we've embraced AI across multiple parts of the organization. We're using it in clinical care, education, research, and many other areas. Because of that, we have leaders across the health system who understand both the opportunities and the challenges AI presents.

At the same time, that understanding doesn't always trickle down to frontline providers. One of the biggest risks is the impact this has on the health of our community in New York City. Leadership recognizes that risk. What we're still working through is how to appropriately resource solutions so we can respond to it effectively.

Like everything else with AI, we're continuing to iterate. We're learning as we go, and we're getting there.

Brandon Schakola (Healthcare Success): I'll just close with one final thought.

Healthcare websites aren't dying. Websites aren't dying. The web itself isn't dying.

What's happening is that the web is evolving. We're adding another layer that includes AI agents, agentic browsing, new protocols, and new ways information needs to be structured and delivered. That's simply the next stage of the web. Thank you, everyone

Jared Johnson (eHealthcare Strategy & Trends): Thanks, Brandon.

That's all the time we have for questions. I want to thank all of our presenters again. This has been a terrific discussion.

AI & Healthcare Marketing Week continues tomorrow with two more sessions. Beyond the Dashboard begins at 1:00 p.m. Eastern, followed by Building a Durable AI Strategy for Health Systems at 3:30 p.m. Eastern.

You can learn more or register for those sessions at eHealthcareStrategy.com.

Thanks again to everyone for attending. We hope you'll join us again tomorrow.