Much of the public conversation about AI focuses on the shortcomings that are easiest to see and understand: the confident wrong answer, the invented reference, the summary that misses the point. Those shortcomings are real, but they distract from a more important problem: human and organisational failures that are rarely noticed, because they are simply how work is carried out. AI capability has grown remarkably quickly, yet organisations that adopt it often struggle to show any lasting benefit. If the technology increasingly works, yet the projects built on it still fail, the cause must lie elsewhere. One cause is that organisations count how widely AI is used, and mistake that usage for the improved outcomes it was meant to deliver. For governments now investing heavily in AI, failing to recognise where the problem lies leads to a misallocation of resources, and poorer outcomes for the public.
We also judge machines and people very differently. Every workforce produces confident wrong answers, misremembered facts, and summaries that miss the point, yet no one concludes that people are unfit for work. Psychologists call this double standard algorithm aversion: people lose confidence in an algorithm far more quickly after seeing it err than in a person who makes the same mistake. Some caution is justified, because an AI error can be repeated across thousands of cases at once, often with misleading confidence, and is harder to hold to account. Yet human fallibility has never led organisations to stop employing people. Instead, they manage the risk, with review, audit, and accountability, and with systems that support and enable people’s work. The same approach should apply to AI.
The capability of AI is no longer the main obstacle. The Industrial Revolution harnessed energy to replicate physical work at scale, and AI is beginning to do the same for elements of thinking. Give a doctoral thesis to an open-source language model running on an ordinary computer, and within a minute it can summarise the central problem, set out how the argument is structured, and point to the claims most vulnerable to critique. That analysis once took experienced academics many hours. Whatever is holding organisations back, it is increasingly unlikely to be what the technology can do.
Yet most AI initiatives fail to deliver meaningful gains in productivity. A 2024 RAND study noted estimates that more than 80% of AI projects fail, around twice the rate of other information technology projects. A 2025 study from MIT’s NANDA project reported that around 95% of organisations were seeing no measurable return from generative AI. Its methods have been questioned, but the direction of the evidence is consistent. Notably, the causes these studies identify are mostly not technical: misunderstanding the problem to be solved, adopting new technology without a clear outcome in mind, and systems that never fit the way work is actually done. These are not failures of capability, but failures to redesign work around a new tool.
The most common mistake is to apply AI to a single part of the work, without considering the outcome the work as a whole is meant to achieve. An AI tool introduced into an unchanged process inherits all of that process’s constraints: the same reviews and approvals, the same handoffs, and the same understanding of how performance is measured. If an officer can draft a brief in minutes, but the brief then waits a week in a queue for clearance, no real benefit has been realised. Just as a chain is only as strong as its weakest link, a process is only as effective as its most constrained step, so a gain at one point may have a negligible effect on the overall outcome. Improving outcomes is therefore a question of service design, not of software. Organisations that apply AI to isolated tasks should expect gains that are equally isolated.
History has seen this pattern before. When factories first adopted electricity, many simply replaced their central steam engine with a large electric motor, and kept the shafts and belts that steam had required. For decades, productivity barely moved. As the economist Paul David showed, the large gains came only in the 1920s, once factories were redesigned around individual motors on each machine, with new layouts, new workflows, and new kinds of jobs. The technology had been available for a generation. The gains came not from using electricity to improve parts of the old system, but from redesigning the system itself, so that the technology could enable better outcomes.
Human judgement, and how it is exercised, is central to all of this. People need to rely on an AI system enough to use it, but not so much that they stop questioning it, or notice too late when it is wrong. That balance depends on how a system presents its uncertainty, what information it surfaces or summarises, how often people see it make mistakes, and whether they feel empowered to override it. It also depends on incentives. Staff who are held responsible for errors, but rarely recognised for the benefits AI brings, may reasonably avoid the tool, duplicate its work manually, or defer to it, because pointing to what the AI recommended appears to shift responsibility elsewhere. The technology, in other words, is often the easier part. The harder work lies in understanding which decisions depend on human judgement, and what allows people to exercise it well. Training that teaches the mechanics of an AI tool, but not how to assess its outputs, or where judgement is of greatest importance, leaves this problem untouched.
In practice, AI initiatives should begin with a view of the work as a whole, not with the tool itself. Work does not happen in isolation: every task draws on inputs from elsewhere, transforms them, and passes its outputs further along the chain. Before adopting a system, an organisation should understand what feeds into the work and what depends on it, be clear about which outcomes AI is meant to improve, and decide who will be accountable for them. The redesign then extends to roles and responsibilities, reviews and approvals, and recognition, ensuring that staff are both capable and authorised to apply their judgement where it matters, and rewarded, not only blamed, for doing so. Without such a holistic view, the benefits of AI will be nearly impossible to realise.
What is measured has to change as well, from the effort expended to the quality of the outcomes achieved. Measures such as documents produced, cases closed per officer, or hours spent were designed for a time when thinking was slow and scarce. When AI makes drafting and analysis cheap, volume is no longer the constraint, and those measures lose much of their meaning. The aim is not more outputs, but better outcomes: decisions that are accurate and well reasoned, reliable outcomes delivered sooner, and errors caught before they cause harm. Counting how often AI tools are used measures none of these. What matters is what AI enables, not how much of it is consumed. Unless the measures change, organisations will reward the appearance of AI adoption, while missing whether it has made the work any better.
Measuring outcomes is harder than counting usage, but far from impossible, and much of what is needed already exists in government. Decision quality can be assessed by independently reviewing a random sample of decisions, before and after AI is introduced. Downstream indicators show whether that quality holds: how often decisions are reworked, overturned on review or appeal, or give rise to complaints. Timeliness should be measured from the moment a person seeks a service to the moment they receive a reliable outcome, not by how long a single task takes. Recording how often staff override an AI system, and whether those overrides prove correct, shows whether judgement is being exercised or quietly abandoned. Introducing a system in stages allows teams using it to be compared with teams not yet using it. None of this is new. What is usually missing is the decision to apply it, and an owner accountable for outcomes across the whole process, not just the step where AI was introduced.
This matters because as long as organisations confuse AI usage with the outcomes AI can enable, and view elements of work in isolation from the system around them, they will keep investing in AI while the services it was meant to improve stay much the same. Many organisations are standing on the factory floor, admiring a newly installed lightbulb, and wondering why productivity hasn’t transformed. Tools are bought, pilots are run, and metrics track how much AI is being used, while the work itself shows no meaningful improvement in quality or efficiency. For government, that means public money spent on capability that changes little, or that quietly erodes the judgement it was meant to support. The value of AI to the public will be decided not by how much of it government uses, but by whether government is prepared to treat each element of its work as part of a larger system, and has the courage to change that system when better outcomes require it.
Sources
- Ryseff, J., De Bruhl, B. F. and Newberry, S. J. (2024), The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed: Avoiding the Anti-Patterns of AI, RAND Corporation
- Dietvorst, B. J., Simmons, J. P. and Massey, C. (2015), “Algorithm Aversion: People Erroneously Avoid Algorithms After Seeing Them Err”, Journal of Experimental Psychology: General, 144(1), 114–126
- David, P. A. (1990), “The Dynamo and the Computer: An Historical Perspective on the Modern Productivity Paradox”, American Economic Review, 80(2), 355–361
- MIT report finds most AI business investments fail, reveals “GenAI divide” (Virtualization Review, 19 August 2025)