AI Agenda Live Recap: Why AI Pilots Fail—and How to Find ROI

For every enterprise artificial intelligence success story, there are several cautionary tales—pilots that falter due to poor adoption, privacy headaches or bad-fit use cases. Recent studies point to a majority of AI pilots failing. While some dispute the figure, what’s clear is that many projects never make it past the pilot stage, making experimentation a costly gamble for chief information officers and chief financial officers under pressure to show return on investment.

At a recent panel, The Information reporter Aaron Holmes spoke with three leaders about why so many AI pilots stall—and what separates the winners from the rest:

• Jared Coyle, chief AI officer, SAP Americas
• Keith Strier, senior vice president, global AI markets, Advanced Micro Devices
• Kabir Nagrecha, co-founder and CEO, Tessera Labs

Why Pilots Fail…

When asked what separates successful pilots from wasted investments, Coyle pointed to a familiar culprit: people.

“Of the 34,000 organizations running our AI capabilities, the most successful are those that make it easy for human behavior to adapt,” said Coyle. The more disruptive and less intuitive the change, the more likely it is to flop.

Nagrecha agreed, noting that adoption hinges on conviction from the top: “For the folks on the ground to feel empowered to change, they need to hear support and conviction from leadership.”

If executives champion AI and empower teams to adapt workflows, adoption follows. Without that cultural shift, even the most promising tools get abandoned.

…And When They Work

Ironically, the most effective AI pilots tend to be the least flashy. Coyle noted that “the best AI is really boring”—automating document processing, scanning receipts or generating meeting agendas. While these use cases may not wow onstage, they save time and embed seamlessly into workflows.

Nagrecha added that ambitious targets also matter: “A company that aims for a 50% gain is more likely to succeed than one shooting for 10%, because 50% forces you to rethink processes instead of tinkering at the margins.”

Measuring a Pilot’s Success

When it comes to tracking a pilot’s ROI, Strier warned against equating failed pilots with wasted time.

“The whole nature of pilots is to learn, to try things out and fail quickly,” he said.

That said, CIOs still want numbers. Panelists said metrics should be tied to outcomes—cost savings, revenue gains or productivity boosts—rather than proxy measures like token counts.

As Nagrecha put it: “You might need five tokens to solve a huge problem and a hundred thousand to solve a tiny one.”

The Future of AI Pricing

A year ago, many experts predicted the cost of AI would go down, with even Google CEO Sundar Pichai anticipating the technology would one day be as cheap as air. Instead, inference costs have ballooned. As models process more tokens and force apps to abandon unlimited-use pricing, business leaders might understandably be more judicious about how they allocate their AI budgets.

That said, Strier believes affordability is inevitable.

“The only way AI is going to proliferate and become even semi-accessible is if the cost comes way down—the cost of inference, the cost of architecture, the cost of energy,” he said. “It’s not just for [venture capitalists] who want to make money; it’s for the benefit of the world.”

For startups like Tessera, Nagrecha said pricing often comes down to distinguishing “painkiller” problems—mission-critical outcomes companies will pay for—from “vitamin” problems like email summarization.

“When you’re solving a painkiller problem, you’re not paying for tokens—you’re paying for outcomes,” he said.

Finding AI Efficiencies

Compliance and privacy remain hurdles, particularly in regulated industries. Some enterprises ban AI outright, but employees often use it anyway. Strier compared it to LinkedIn’s early days, when bans quickly proved futile.

A smarter approach, panelists suggested, is to provide safe playgrounds for experimentation, then scale what works. Coyle shared one such case at SAP: An employee built a simple agenda generator to cut down prep time for marketing events. SAP rolled it out companywide, saving significant hours with what was essentially a “boring” use case.

For CIOs and CFOs, the lesson is less about chasing the flashiest demos and more about rethinking workflows and culture.

“Stop dart-boarding use cases,” advised Coyle. “Instead, reimagine what your business should actually be doing.”