Legacy technology is the hidden barrier to AI ROI

The infrastructure question nobody asks first

AI tools get purchased, partially deployed and underperform – not because the tools are wrong, but because the infrastructure beneath them can’t support them. This pattern is common enough that it has become a cliché, but it remains underappreciated in the planning phase. Businesses budget for licences, sometimes for training, rarely for the integration and data work that determines whether the tool delivers anything at all.

The businesses realising meaningful AI ROI are largely the ones that approached it as an infrastructure problem first and a tooling problem second. The ones still waiting for results are often still negotiating with the systems underneath.

What “legacy” means in the AI context

Legacy technology is often understood as hardware – old servers, end-of-life operating systems, equipment past its refresh date. That matters, but it’s not the primary obstacle for AI adoption. The more significant issue is architectural: systems that don’t expose data through APIs, databases that can’t be queried in real time, ERP platforms so heavily customised that only a handful of specialists can modify them, CRM systems carrying years of inconsistent data entry, and reporting processes that live in spreadsheets exported manually once a week.

AI systems – whether copilots, predictive tools or automation agents – need data to operate on. They need that data to be accessible, structured and reasonably clean. They need to read from live systems and, increasingly, to write results back into them. Legacy technology typically fails on all three counts: data exists but is trapped in siloed systems; access is manual rather than live; and the ability to write results back is absent because the system was never designed to receive external inputs.

Where the gap shows up in practice

Data silos. An AI tool built to support demand forecasting needs stock data, sales history, supplier lead times and market signals. In a business where the warehouse management system, ERP and e-commerce platform don’t communicate with each other, that data exists in three places, in three formats, with no reliable way to combine it automatically. The implementation stalls while a manual export process is designed – which then becomes the ongoing bottleneck. The tool is only ever as current as the last export.

Systems with no API layer. Older ERP and accounting platforms – particularly heavily customised installations from the mid-2000s – frequently have no API. Data extraction requires scheduled scripts, custom connectors or vendor-supplied integration modules that carry additional licensing costs. When every AI integration requires its own extraction layer, per-integration cost rises sharply and the maintenance burden grows with every system update the vendor ships.

Inconsistent and unstructured data. AI models are sensitive to data quality in a way that human processes often aren’t. A business analyst can look at a “product category” column and understand that someone used five different naming conventions over five years. A language model working on product recommendations cannot. Before AI can be applied, data needs to be cleaned, normalised and structured – work that typically reveals years of undocumented inconsistency and sits entirely outside the AI tool budget. A customer-facing AI assistant that requires a prior data remediation project is a common scenario.

On-premises infrastructure. Cloud-hosted AI services need connectivity to business data. When core systems run on-premises behind firewalls, on private networks, that connectivity becomes both a security and an architecture problem. VPN tunnels, secure API gateways and data replication pipelines are all achievable – but the cost sits outside the AI tool budget and the implementation time sits outside most AI deployment timelines.

Entrenched workarounds. Many businesses have normalised working around their own systems. Data that should be in the CRM lives in a spreadsheet because the CRM was never configured properly. Approval workflows that should be automated are email chains because the integration was never built. When AI is applied to these processes, it encounters the workaround rather than the underlying system – and optimising a workaround rarely produces meaningful efficiency gains.

The ROI maths

AI tool pricing has come down significantly. Microsoft Copilot, specialist vertical tools and API-based AI services are all accessible at a cost most businesses can absorb. The ROI calculation changes substantially once you factor in integration work.

A business that purchases an AI tool expecting productivity gains from day one typically discovers a project: data mapping, API development or procurement, data cleaning, testing and user training. That project can cost two to five times the annual tool licence in implementation effort. Productivity gains, once realised, need to be weighed against total investment rather than subscription cost alone.

Integration overhead scales with the number of legacy systems involved. A business with a modern data stack, a clean CRM and a cloud ERP will pay a fraction of the integration cost that a business with five siloed systems and no API layer will pay – for the same AI tool. The ROI gap between the two scenarios, using identical tooling, can be significant.

This is not an argument against AI investment. It’s an argument for factoring infrastructure into the business case before committing to a tool. A realistic assessment of integration cost changes both the budget and the timeline – and often changes the tool selection too, since some tools require less integration than others for the same use case.

Prioritising modernisation

Wholesale modernisation – replacing an ERP, migrating to a new CRM, rebuilding a data warehouse from scratch – is expensive, disruptive and rarely the right starting point. The practical approach is targeted: identify the infrastructure constraint that limits the specific AI use case with the clearest business value, address that constraint, and measure the impact before broadening scope.

If the highest-value AI opportunity is customer service automation, the priority is getting clean, accessible customer data into a form the AI can use – connecting the CRM via API, building a data pipeline that aggregates contact history, or cleaning a specific dataset. Not replacing the CRM.

If the opportunity is operational forecasting, the priority is creating reliable, timely data feeds from operational systems – an API integration between the WMS and a data warehouse, rather than replacing either system.

This approach has a compound effect: each integration and data quality improvement makes the next AI use case easier and cheaper to deploy. A business that builds a proper data layer for one AI project is substantially further along for the second.

Where to start

A practical starting point is an audit of data assets and integration points: what data exists, where it lives, how it’s accessed, what quality it’s in and what would be needed to make it available to external systems. This surfaces both the opportunity – what AI could use – and the gap – what needs to change before AI can use it effectively.

From that audit, the highest-ROI starting points for modernisation are typically API enablement for systems holding the most valuable data (CRM, ERP, operational platforms), data warehouse implementation to centralise disparate sources, and targeted data quality work on the specific datasets needed for target AI use cases.

These aren’t AI projects. They’re infrastructure projects that AI depends on. Treating them as such – with clear business cases and measurable outcomes – produces a more accurate picture of what AI investment actually requires and what it can realistically return.

The gap between AI promise and AI delivery is rarely a tooling problem. It’s almost always a data and integration problem dressed up as one.