US businesses poured billions into GenAI last year, but MIT’s Project NANDA report found that 95 per cent of pilots produced no measurable impact on a company’s profit and loss statement.
When AI stalls, the model is never the real problem. What sits underneath it, such as the quality of the data and governance, is. Failure rate isn’t the most interesting part of this report. The reason behind it – that organizations have spent years fixing the wrong problem, is. An ROI gap is the clear result of this.
Why boardrooms are fixated on models
Incorrect decisions lead to the wrong problems being fixed. CFOs will answer the question ‘How is your AI Strategy going’ with their choice of large language model. The ease of conversation around model choice has become the default procurement question. Benchmark scores give a clean comparison and contracts can be signed in weeks. A CFO walks out of the room with something to show the board. The harder question ‘is the company’s data in good shape?’ remains unanswered.
This question also remains unanswered if you simply fix the data that specifically runs the tools. There’s still no simple score to identify errors such as customer records being duplicated across different systems, or the loss of information between platforms. For the CFO, it’s easier to sign off a model than it is to fix the data that is broken behind it. Therefore, the wrong problem is addressed first.
Perceived cost also fuels reluctance to fix the right data. Re-architecting data pipelines and integrating fragmented systems is a massive, open-ended capital expenditure from the point of view of a CFO. To simply purchase a model is a fixed, predictable line item that is easy to place in a budget. Similarly, a rebuilt enterprise architecture feels like a financial black hole.
61 per cent of organizations have already invested in AI without fixing their initial data problem. This is set to continue – with 72 per cent prioritizing future AI investment over their data infrastructure.
This rapidly emerging trend must change.
The real issue is plumbing
If you just fix the data that works with the model, the model will continue to get all of the credit – or all of the blame.
If we consider AI a tap, it doesn’t matter how good the tap is if the pipes behind the walls are blocked or leak. Unfortunately, most data organization’s pipes behind their walls are broken. Their AI projects look impressive in the testing stage, but are never correctly used once they move into production.
The same is true for plumbing – it only becomes an issue when it fails. To simply apply a patch to broken data is not sufficient – the issue must be addressed at the source. This in turn means that someone has to be responsible for when data goes out of date. Responsibility also needs to be assigned to ensure tools are reaching for the correct data, not an older version.
The organizations getting real value from AI didn’t succeed because they picked a better model, but because their data was already in a good state.
Why agentic AI raises the stakes
The current wave is agentic AI – tools that take actions on their own, usually across several systems at once. A chatbot that gives a wrong answer is a one-off mistake, often easy to spot and easy to fix. An agent working through a broken system, on the other hand, can make one bad decision, based on one bad piece of information. This bad then is reinforced and spread, and can lead to major error.
Agentic AI doesn’t just increase the number of things that can go wrong, it diminishes the human in the loop that catches the mistake. This governance failure is the reason more than 40 per cent of agentic AI projects are expected to be cancelled by 2027.
If data that underlies your system isn’t reliable, one small error can ripple through every process that agent touches, and every other tool it talks to. The room for error gets smaller the more these systems become independent, which is why the plumbing work isn’t optional anymore. It is an essential part in what decides if agentic AI can be used safely.
Where CFOs, COOs, CIOs and their CEOs need to actually invest
The next round of AI spending shouldn’t go on more powerful models or heavier usage. It should fix and connect the data underneath and mapped journeys that run end-to-end. Crucially, this does not have to be a net-new capital expenditure. Smart enterprises approach this as a self-funding mechanism for transformation. By identification and excision of structural operational inefficiencies – such as retiring unused legacy applications, consolidating overlapping vendor licenses, and eliminating low-margin tech waste – businesses can liberate the trapped capital needed for critical technology modernization whilst also unlocking bottom-line improvements.
To get the most out of AI tools, it’s essential to bring together all the fragmented data into one accurate picture. We are seeing and supporting organizations, particularly those in the $1B to $5B mid-market sweet spot where operational complexity is high but enterprise agility remains particularly possible, use these operational redesign programs to directly fund their AI data infrastructure and cloud migrations. Link legacy infrastructure savings directly to the AI compute budget, and the C-suite can build the proper foundations, processes and governance before AI is given more independence.
Once that focus is in place, the conversation changes from which AI to buy to how operational redesign can drive revenue increases and cost savings to fund the organization’s capability to truly capitalize on it. The organizations that get AI working won’t necessarily have the best model, they’ll have the best foundations (data, processes, talent, infrastructure, and governance), and that’s what lets AI run properly on a day-to-day basis. The 95 per cent stalled because they built on foundations that couldn’t support what they were asking AI to do. The businesses that shift their focus to self-funded modernization of what sits underneath for their specific organizations will be the ones who succeed, both in the short-term and the long-run.
