The consequences of the current path are severe. But the case for the pro-human path doesn't rest just on avoiding them. This section compares the Race to Replace with the Pro-Human Path as AI development strategies, arguing that the Pro-Human Path does not mean forgoing the positive fruits of AI; indeed it is more likely to bear them – at least for the fruits most people actually want.

In fact the AI that is currently already working and providing positive benefit is generally Tool AI: coding assistants, research aids, diagnostic support, synthesis of search results and data, editorial assistance, etc. What's working far less well is partial replacement by AI that doesn't do as good a job and that people don't trust.

The line between a productivity tool and a replacement can be thin: higher productivity can mean less people needed to do a fixed amount of work. But the difference is real, and the intention matters.

Humanity does not have to blindly be drawn into an AI development direction ultimately driven by a few giant companies' quest to optimize their own revenue and power. We can choose one that works well for the rest of us.

The Economic Case

The friction tax on autonomous agents

Rosy projections for autonomous agents assume frictionless deployment into the human economy. But raw capability does not translate directly into deployable value.

Deployment tax. Enterprise infrastructure requires deterministic systems and auditable trails. Autonomous agents that can't explain their reasoning are enterprise hazards. Tool AI augments existing workflows immediately; agents require years of infrastructure adaptation.

Liability tax. AI systems lack legal personhood; liability will be found to flow to deployers. Insurance for autonomous agents in open-ended environments will be enormous – if available at all. Bounded tools fit existing liability and insurance models.

Societal tax. Labor-replacing AI agents will trigger political backlash and regulatory response. Tool AI creates value without systematically destroying human agency.

Alignment tax. Acceptable behavior in open-ended environments requires enormous ongoing investment. Every new capability creates new failure modes. Tool AI sidesteps the hardest problems by keeping scope bounded and humans in the loop.

When these costs are priced in, the economics of the autonomous path look far less attractive.

The undersupplied collaboration path

Meanwhile, the tool path is undersupplied. Research on "Pro-worker AI" shows that AI's potential as a force-multiplier for human expertise is at least as transformative as its automation potential – and currently underexploited. A market that would normally focus on collaboration tools is being crowded out by vast investments driven by pro-automation ideology.

The bottom line

Despite the mad rush to develop general-purpose AI agents and jam them everywhere, trustworthy Tool AI may ultimately be the faster path to economic adoption. Building for trust takes longer upfront, but systems that can be verified and relied upon can actually get responsibly deployed in consequential settings. The AGI-oriented path rushes impressive demos to market, but they fail to deliver when reliability matters.

The Competitive Logic

Beyond avoiding friction taxes, Tool AI offers positive competitive advantages.

Reliability is the bottleneck

The largest barrier to AI adoption is not capability but reliability. Organizations cannot use AI for consequential decisions – in engineering, banking, infrastructure, law, medicine – if they cannot rely on the results.

Purpose-driven tools with defined scope make reliability assessments tractable. Verifiable reliability is already a clear market demand. Meeting it is a decisive advantage.

Liability moat

The current paradigm is headed for liability chaos: systems making consequential decisions with no one accountable. Tool AI resolves it cleanly. Humans remain central and can bear responsibility because what the technology is doing is enhancing their capabilities. Building controllable systems now creates compliance advantage as the legal system catches up.

Those pushing general AI agents tell us we must remake our legal and economic system to accommodate vast numbers of autonomous actors that can't be held accountable. Let's not do this.

Defining a race worth winning

The frontier race requires tens of billions per training run. Most companies and countries cannot compete. But rather than competing in a Race to Replace they cannot win, other actors can define the terms of a race worth winning: efficient, specialized systems optimized for specific purposes—highly capable within their domain, more verifiable, and achievable without frontier-scale compute.

Rather than direct all energy into giant training runs and general AI development with "safety" tacked on, significant resources can instead be put into:

• The AI testing and assurance stack – which can be powered by AI itself – accelerating the development of reliable and trustworthy AI.
• Responsible, gated access to data that commercial AI can or should not be trained on: medical records, government databases, educational records, proprietary research.
• Domain partnerships with experts that define problems that actually need solving, improve products, and provide credibility general-purpose providers lack.

These directions leverage broad expertise outside of the big AI companies and can help many more institutions and countries participate in developing the AI we actually want.

Tool AI Delivers What We Actually Want

The case for AGI assumes we need AGI to get AI's benefits. This assumption is false for most of what people actually want.

What researchers and professionals actually need

AGI companies promise to "cure cancer" and "solve climate change." But ask researchers what they actually need, and they don't describe general-purpose systems "optimized for everything." They don’t need a system built to drive engagement, generate social media slop, or flirt with their kids. They need tools optimized for scientists: literature synthesis that tracks provenance, hypothesis support that quantifies uncertainty, data analysis that explains its reasoning. They need systems that tell them the truth rather than what they want to hear. At the same time, businesses that would like to adopt AI systems to accurately rewrite spreadsheets don’t need them to be expert virologists or solve Einstein’s equations.

Generality in AI training does give economies of scale in leaning many skills at some. But it also gives real and unavoidable tradeoffs. Mitigating these tradeoffs through specialization to particular expertises can win.

Ironically, the capabilities AI companies tout when describing how they want to help – scientific research, medical diagnosis, educational support – are all tool capabilities. It's just not what they're selling to investors, and it's not what's really driving the race. What's driving the race is money and power, and for that business model to work it requires human replacement, not human augmentation.

What Tool AI can do

Narrow but extremely capable systems like AlphaFold and AlphaDev demonstrate purpose-driven tools whose capabilities are unmatched by general-purpose models. But purpose-driven doesn't mean narrow. Tool AI systems can be developed specifically and intentionally to aid in:

Scientific research: Literature synthesis, hypothesis support, data analysis, pattern recognition. Human scientists direct; AI amplifies reach.

Medical care: Diagnostic support, drug discovery, treatment optimization. Validated tools under clinical oversight, physicians retaining judgment.

Education: Personalized tutoring, accessibility tools, content creation. Systems that assist rather than replace.

Productivity: Writing, code, analysis, summarization, research assistance. Already substantial value—without autonomous goal-pursuit.

Accessibility: Vision, hearing, mobility, cognitive support. The paradigm case of capability extension.

Tools can even be autonomous – like autonomous vehicles – so long as control properties are established, which is simplified by the technology having a defined purpose and bounded scope.

What AGI would add

The claim for AGI is speed and scale: do everything humans can do, but faster and cheaper. The pitch is for human replacement across all cognitive work.

However, faster progress comes packaged with enormous disruption – not as a side effect but as the explicit business model. And human replacement is not something most people actually want, for themselves or their society.

What superintelligence would add

For superintelligence, the claim is more dramatic: capabilities we cannot achieve otherwise. Immortality. Nanotechnology. "Colonizing the galaxy."

But it is self-evident that something capable of producing those outcomes is also an existential risk to humanity. A system vastly more capable than humans across all domains cannot be meaningfully controlled by humans. If you're building something that could give you everything, you're building something that could take everything. And even the potential creators of this technology admit we have no real idea how to realise the gains, whilst avoiding the risks.

The impact of Tool AI

The disruption from Tool AI would be large, and hard to manage. But at least it is possible.

Tool AI will displace workers – the same pattern we have seen and addressed with previous productivity-boosting technologies. Disruptive, but at least potentially addressable: retraining, education reform, safety nets, policies to share gains. Hard, but not unprecedented.

AGI is designed to automate all cognitive labor – not as side effect but as goal. In this scenario, what can humans retrain as? No serious plan exists beyond vague gestures at universal basic income.

The choice is not disruption versus no disruption. It's disruption we know how to address, versus disruption we don't.

Superhuman AI Is Not Inevitable

"Inevitability" is a story told by those who benefit from the race.

Frontier AI companies use it to justify speed over caution. Investors need it to sustain valuations premised on capturing the labor economy. Geopolitical hawks invoke it to block any constraint.

Competition is not the only reason they are in such a hurry. They realize that society and the legal system will catch up with AI. Fierce popular blowback is coming. The hope is to race to the "AGI prize" before everyone else can react, then disempower the rest of us. But AGI and superintelligence are not a predetermined outcome.

Technical chokepoints exist

Advanced AI requires specific resources, each a governance chokepoint:

• Compute: Massive resources, physically produced, legally owned, extraordinarily concentrated supply chain
• Hardware: The most sophisticated manufacturing on Earth, countable facilities worldwide
• Energy: Enormous requirements, physical infrastructure requiring permits
• Talent: A small number of people with relevant expertise, each with choices about what to work on

We've constrained dangerous technologies before – nuclear weapons exist in nine states, not ninety; reproductive cloning is feasible but not pursued. AGI and superintelligence are things a very small number of people are choosing to build so that they alone can benefit. We can choose differently.

Two Paths as Competitors

Tool AI is the Default Alternative

Imagine that tomorrow, some global law were passed – or some law of nature discovered or some divine intervention occurred – that stopped general-purpose expert-level autonomous agents or superintelligence.

Imagine tomorrow the Race to Replace was called off.

What would all these giant AI companies do?

They'd make Tool AI.