By the end of the Healthcare Innovation San Diego Summit, I had heard a lot about AI. That probably isn’t surprising, given the program we put together for Sept. 28 at the Hyatt Regency La Jolla.

AI was woven through the day, from Shane Thielman, Scripps Health’s chief information and digital officer, in the opening keynote, to discussions about automation, ambient clinical AI, cybersecurity and the use of computer vision and sensors inside operating rooms.

But as I listened to the conversations unfold, I found myself thinking less about what AI can do and more about what health systems are actually going to do with it.

That felt like an important shift. We have spent the past couple of years talking about the possibilities of generative AI, experimenting with pilots and trying to figure out where these technologies might fit. The conversations I heard in San Diego were much more grounded in the realities of implementation.

What problem are we actually trying to solve? How will we know whether the technology is working? What does value look like? How do we bring clinicians and other employees along? And, perhaps just as important, when do we decide that a particular technology isn’t delivering what we hoped it would?

Those questions came up in different ways throughout the day, even when the sessions themselves were focused on very different areas of healthcare technology.

Moving past the excitement

Shane Thielman set the tone in his opening keynote by talking about the widening gap between the pace of AI adoption and the ability to demonstrate meaningful outcomes.

Health systems are being inundated with new tools and new possibilities, but that doesn’t mean every one of those possibilities deserves to become an enterprise investment.

At Scripps, that has meant putting structure around how new technologies are evaluated and eventually operationalized. The conversation wasn’t simply about whether a technology is innovative. It was about clinician experience, patient experience, operational efficiency, clinical and financial outcomes, safety and the workforce.

Scripps has developed a governance approach that looks at both the advantages and risks of new technologies while focusing on how they will actually be put to work and what outcomes they are expected to produce.

That last part kept coming back throughout the Summit. Healthcare organizations aren’t short on ideas for using AI. In fact, they may be facing the opposite problem. There are so many potential use cases that deciding where to focus has become a technology strategy issue in itself.

AI is showing up in revenue cycle, access, referrals, prior authorization, documentation, patient communication, diagnostics, cybersecurity and business operations. Some of the examples we heard were highly clinical, while others involved the kind of administrative work that has frustrated healthcare organizations for years.

What made those examples interesting wasn’t necessarily the technology behind them. It was the very practical question of whether taking a particular task off someone’s plate could create measurable value somewhere else in the organization.

That is a very different conversation from the one we were having a couple of years ago.

Sometimes the best answer is not another technology

One of the things I appreciated about the Summit was how willing the speakers were to talk about the limits of technology as well as its potential.

During our Innovation in Practice discussion, Jeff Reeves, M.D., who leads perioperative informatics and innovation at UC San Diego Health, joined CIO Josh Glandorf and Chief Administrative Officer Trevor Bennett to discuss work involving sensors, computer vision and AI in the operating room.

The technology can capture what is happening in the OR, recognize processes and potentially automate some of the documentation and tracking that staff currently handle manually. There are potential applications around room turnover, equipment and supplies, environmental services and other processes that can consume significant amounts of staff time. But the conversation quickly moved beyond the technology itself.

Before adding another system to an already complicated environment, health systems have to ask what they are actually trying to accomplish and whether the capability they need might already exist somewhere within their enterprise systems. They also have to determine whether the technology is accurate enough to act on, whether it can be integrated into existing workflows and whether the people who will use it understand what it is doing.

Glandorf made a point that applies far beyond this particular project. If an organization tries to solve 15 different problems with 15 different technologies, it may eventually create a 16th problem simply by adding more complexity to the environment.

That is an easy trap to fall into right now.

There is a tremendous amount of pressure to keep up with the pace of AI, and vendors are understandably eager to show healthcare organizations what their technology can do. But every new application carries an operational burden.

Someone has to implement it, train people on it, integrate it, support it, govern it and eventually decide whether it is worth continuing. The price tag on the software is only one part of that equation.

The question of value is getting harder, and more interesting

ROI came up repeatedly during the day, reflecting how much the healthcare AI conversation has shifted toward implementation and measurable value.

In the AI Innovations and Disruptors session, panelists Chad Wilson, CEO of Wild Duck Labs AI Research Lab; Tyson Blauer, senior director of insights at KLAS Research; and Adam Cherrington, founder, chief advisor and patient champion at Aligned Health Advisory, talked about the need to define success before launching a project rather than trying to figure out afterward whether the technology delivered anything meaningful.

That sounds obvious, but it is surprisingly easy to get caught up in the excitement of a new capability and start measuring whatever is easiest to measure.

The same issue came up during a discussion about ambient clinical AI. Ambient AI has moved remarkably fast from something organizations were experimenting with to something that is now being deployed in everyday clinical workflows. The conversation at this point is no longer simply whether it works. The more difficult question is how a health system demonstrates that it is worth the investment.

Vivek Reddy, M.D., chief health informatics officer at University of Utah Health, talked about how his organization initially focused on clinician satisfaction and burnout. That was the problem they were trying to address. But when organizations start looking at financial return, the measures can become much more complicated. Are physicians seeing more patients? Are documentation times actually decreasing? Can the organization reduce its reliance on scribes? Is the technology allowing clinicians to spend more time interacting with patients and families? Sometimes the answer isn’t as neat as we might like.

Mario Bialostozky, M.D., chief medical information officer and associate chief quality officer of Rady Children’s Hospital San Diego, explained that his organization hasn’t necessarily seen a dramatic reduction in documentation time because clinicians tend to use the ambient technology for more complex patients.

Those are precisely the encounters where the technology may be most valuable, but they are also much harder to compare with a simple visit that requires very little documentation in the first place.

That raises a broader question about what we choose to call value. If a physician is able to spend more of a complicated visit looking at a child and talking with that child’s family rather than focusing on the computer, that may be a meaningful outcome even if it doesn’t show up neatly as a reduction in minutes spent in the EHR.

When Cherrington asked what innovation looks like from a physician’s perspective, Bialostozky came back to a surprisingly simple measure: time. He talked about the value of reducing the time physicians spend wrestling with burdensome administrative processes and giving some of that time back to the things that matter, whether that means more time with patients, more time with family or simply more time for themselves.

In a healthcare environment where so much of the work around patient care has become increasingly administrative, that was a useful reminder that the return on an innovation does not always have to be measured in dollars or productivity. Sometimes, giving a physician back some of their time is valuable in its own right.

It was refreshing to hear people talk about what they are still figuring out

What I appreciated about the Summit was how willing the speakers were to talk about what they are still trying to understand.

Reddy shared that University of Utah Health had been working with ambient AI for several years and had tested multiple vendors before moving to an enterprise license this year.

At Northeast Valley Health Center, CIO Stephen Gutierrez described a more cautious approach, with the organization holding back on adoption until some of its questions around the technology and data were addressed.

And at Rady Children’s, Bialostozky described a particularly thoughtful approach to evaluating the technology. The organization tested two ambient AI products, with groups of providers using each solution before switching them so both groups could experience both products. Those experiences ultimately helped Rady determine which technology it wanted to use.

Across the discussion, the question of value kept coming back up, with the panelists talking about everything from clinician satisfaction and documentation time to patient volume, the use of scribes and the less easily measured benefit of giving clinicians more attention to their patients.

That kind of honesty is important because there is a tendency in healthcare technology conversations to present implementation as a straight line. You identify a technology, you launch it, the numbers improve and everybody moves on to the next project. But that isn’t always how it works.

Sometimes adoption is uneven. Sometimes clinicians use a technology differently than expected. Sometimes the metric you thought would matter doesn’t tell you very much. And sometimes the right decision is to slow down, change course or walk away from a project.

That last point came up in several conversations during the day. Organizations need predetermined checkpoints and criteria for evaluating whether a technology is actually delivering what it was supposed to deliver. Otherwise, it becomes very easy to keep investing simply because you have already invested so much. In a field moving as quickly as AI, knowing when to stop may be just as important as knowing when to start.

Then there is the human side of all of this

Trust was another theme that kept surfacing, sometimes explicitly and sometimes underneath the conversation.

We talked about clinicians who are being asked to adopt AI into their workflows. We talked about employees who understandably want to know how these technologies will affect their jobs. And we talked about cameras, microphones and sensors being introduced into clinical environments. Those questions aren’t going away simply because the technology works.

During the UC San Diego discussion, there was a thoughtful conversation about the importance of being very intentional about what the technology is there to do. If cameras and AI are being used to improve workflow, document processes or identify opportunities to make an OR more efficient, the people working in that environment need to understand that purpose. They need to know what is being collected, how it is being used and, just as importantly, what it is not being used for.

That is where governance becomes much more than a committee reviewing technology. It becomes part of implementation.

The same thing came through in the ambient AI discussion. The panelists talked about privacy, the way patients are responding to AI-generated information and the need to keep a human in the loop.

The technology doesn’t have to be perfect to be useful, but the organization does have to understand where the technology’s limitations are and where human judgment still needs to take over. That seems particularly important as AI begins moving from simply generating information to taking actions and making decisions.

Cybersecurity brought the conversation full circle

By the afternoon, it was clear that this wasn’t really just an AI lesson. It was an implementation lesson, and cybersecurity brought that point into focus. How much change can an organization realistically absorb? How do you evaluate new technologies without chasing every new tool that comes along? How do you build resilience when the underlying technology environment is changing so quickly?

The cybersecurity panelists, including Chad Wilson, Elliott Jones, CISO at Kaiser Permanente; David Loor, senior director of cybersecurity at Keck Medicine of USC; and Charles Flack, senior director of information technology at Cascadia Health, talked about the growing volume of patches and vulnerabilities, the increasing sophistication of attacks and the ways AI is being used on both sides of the security equation. They also talked about the practical benefits they are seeing from AI in areas such as investigations and incident response.

But again, the message wasn’t simply that AI will solve cybersecurity. It was that organizations need to understand their vulnerabilities, evaluate the available solutions and make deliberate decisions about where technology can actually help.

One panelist described the danger of moving from one “cool tool” to another without stepping back to ask what problem the organization is really trying to solve. That sounded remarkably familiar after a full day of healthcare AI conversations.

Whether we were talking about an ambient documentation tool, an operating-room sensor, an automation project or a cybersecurity platform, the underlying discipline was essentially the same. Understand the problem. Define what success looks like. Bring the people affected by the change into the conversation. Measure what happens. And be willing to change course when the technology isn’t delivering.

The conversation has changed

I came away from San Diego thinking that the healthcare AI conversation has reached an interesting point. We are past the stage where simply saying “AI is coming” is particularly useful. It is already here, and health systems are using it in increasingly practical ways.

The harder work now is figuring out how to make those technologies fit into organizations that are complicated, heavily regulated, financially pressured and, ultimately, still built around people taking care of other people.

That means the questions we ask about AI have to become more sophisticated, too. What problem are we solving? What does success look like? Whose time are we giving back? What happens to the workflow? What happens to the workforce? How do we build trust? How do we know when a technology is ready to scale, and how do we know when it is time to let it go?

Those aren’t necessarily the questions that generate the most excitement when a new technology is unveiled. But after spending a full day in a room with healthcare leaders who are actually trying to make these technologies work, they are the conversations worth having.

And frankly, they are the conversations I want Healthcare Innovation to keep having. Because the interesting part of this next chapter isn’t going to be watching AI arrive. It’s watching healthcare figure out what to do with it.

Melinda Taschetta-Millane is Market Content Director of Healthcare Editorial, and Head of Content for Healthcare Innovation.