Understanding AI Capabilities
A clearer way to think about what makes AI systems powerful—and dangerous.
The term "AGI" obscures more than it reveals. Definitions are so varied that commentators can't agree on whether it's already been achieved, will be achieved imminently, or is decades away or even impossible. We need a better framework for understanding what we're building and what to be concerned about.
Google’s 900,000 square foot (83612.7 m^2) data center in St. Ghislain, Belgium. (Image by Google.)
The Problem with "AGI"
"Artificial General Intelligence" is typically defined as AI that matches or exceeds human capabilities across all cognitive domains. Sometimes it is seen as a somewhat arbitrary point when "capability" matches "human level." Other times it's viewed as an almost mystical "emergence" of something brand new that AI didn't have before.
Neither is accurate or particularly useful for understanding how we can develop AI well. We need a framework that more accurately captures the structure of AI capability and risk.
Three Dimensions: A, G, and I
A useful way to understand AI capability is to divide it into three key dimensions:
Autonomy (A):
The degree to which a system acts independently, without human oversight or intervention. A chess engine that waits for your move has low autonomy. An agent that browses the web, writes code, executes it, and iterates based on results has high autonomy.
Generality (G):
The breadth of areas in which the system can operate effectively. A narrow system plays chess or predicts protein structures. A general system writes code, drafts legal briefs, tutors students, and analyzes medical images.
Intelligence (I):
The system's competence at cognitive tasks within its domain of operation. A weak system makes frequent errors and misses obvious solutions. A strong system performs at or beyond expert human levels.
#TOOL-01
A-G-I Venn Diagram
A Pro-Human Path › About the Paths › Understanding AI Capabilities
How Autonomy, Generality, and Intelligence fit together to create the conditions necessary for ‘AGI’.
These dimensions are largely independent:
- A system can be highly intelligent but narrow (AlphaFold)
- A system can be general but have weak intelligence (early chatbots)
- A system can be autonomous but limited in scope (trading bots)
- A system can be intelligent and general but passive (current non-agent chatbots)
The independence of these three aspects is crucial. It means we can develop systems that are highly capable along some dimensions while deliberately limiting others.
The Dangerous Zone
Risk emerges primarily from the combination of all three dimensions at high levels. A system that is simultaneously:
- Highly autonomous (acts without human oversight)
- Highly general (operates across many domains)
- Highly intelligent (performs at or above expert level)
...can pursue goals across a wide range of domains, at expert level, without humans in the loop. This is where control becomes difficult and risks compound. This combination is also what enables humans to be wholesale replaced by digital systems.
#TOOL-02
A-G-I Danger Zone
A Pro-Human Path › About the Paths › Understanding AI Capabilities
The intersection of high Autonomy, Generality, and Intelligence creates outsized safety risks.
Consider the difference:
A narrow, intelligent, autonomous system (like an autonomous vehicle) poses bounded risks. It operates in a defined domain with clear success criteria. Failures are local.
A general, intelligent, passive system (like current LLMs used as assistants) can be highly capable but remains under human direction. The human decides what to do with its outputs.
A general, autonomous, weak system might flail around ineffectively. Annoying, but not catastrophic.
But an autonomous, general, intelligent system can formulate goals, develop strategies to achieve them across multiple domains, and execute without oversight. This is a whole different matter.
Reframing AGI
This framework gives us a more useful definition of what's actually concerning about "AGI":
The dangerous threshold is the combination of high autonomy, high generality, and high intelligence—not human-level intelligence as such.
A system could be superhuman in raw cognitive capability but remain controllable if it's narrow (bounded domain) or passive (human-directed). The risk comes from the intersection.
This reframing has practical implications:
- High intelligence alone has bounded risk and considerable upside. We can (and should!) build highly intelligent systems for medical diagnosis, scientific research, engineering design. Although we do have to manage misuse of dual-purpose tools, without generality and autonomy these tools will not wholesale replace people or end up in charge of things.
- Autonomy is a design choice. Systems don't have to be autonomous. We can architect for human direction, meaningful oversight, and genuine control.
- Generality has costs. A system that can do anything is hard to verify, hard to bound, and hard to control. Purpose-built systems with defined scope are easier to make safe.
- The path to safety runs through limiting the combination, not through hoping we can align a system that's maxed out on all three dimensions.
Tool AI in This Framework
The A-G-I framework clarifies what Tool AI means:
The core property of Tool AI is meaningful human control. This is dramatically easier for systems designed to remain low on at least one critical dimension—typically autonomy.
A Tool AI can be:
- Highly intelligent and general, but passive
- Highly intelligent and autonomous, but specialized
- General and somewhat autonomous, but with hard capability limits
When at least one dimension is intentionally constrained, controllable tools are much more achievable.
This still allows for enormous impact. It preserves human direction for consequential decisions. It ensures systems operate within defined boundaries. And it makes verification tractable: you can test a system against a specification when that specification exists.
Beyond AGI: Superintelligence
The danger zone doesn't end at human-level capability. If AGI – meaning expert level autonomy, generality, and intelligence – is created, the same forces that drove us toward it can continue past it.
No stable stopping point. Competitive pressure between AI developers, competitive pressure between AI users, and AGI's own capability to improve itself all push toward greater capability. "Human-level" is not a natural resting place.
Self-improvement accelerates. AGI systems can conceive and design improved versions of themselves. The rate of improvement, previously bottlenecked by human researchers, accelerates dramatically when AI does AI research and implements the improvements. Timescales could compress from months to weeks, days, or hours.
Superintelligence is uncontrollable. Control requires understanding what a system does, specifying what it should do, detecting deviation, and enforcing correction. Each becomes impossible when the system vastly outmatches its overseers. A system operating at hundreds of times human cognitive speed, with capabilities beyond human comprehension, cannot be meaningfully overseen. Even perfect alignment would leave humans as figureheads endorsing conclusions we cannot evaluate.
This is why the race to AGI is self-defeating. The "winner" doesn't get a powerful tool; they get the first system no one can control or understand. They don't gain superintelligence; they introduce humanity's replacement.
Implications for Development and Governance
The A-G-I decomposition suggests concrete approaches:
For developers:
- Measure and report autonomy levels, not just capability benchmarks
- Architect for human oversight by default
- Build domain-specific systems where possible
- Constrain autonomy when building general systems
For governance:
- Regulate based on the combination of dimensions, not capability alone
- Require higher assurance for systems high across multiple dimensions
- Create liability frameworks that track with autonomous decision-making
- Distinguish between high intelligence (a positive) and high autonomous general intelligence (potentially extremely negative)
For users and society:
- Demand transparency about autonomy levels in AI products
- Prefer tools that empower us, over agents for high-stakes applications
- Recognize that "more capable" doesn't mean "more autonomous"
The Choice
The current AI race treats any model that scores low on A, G, or I as a deficit to be remedied. A system that's narrow? Push to make it general. A system that requires human oversight? Push to make it autonomous. A system that's below expert level? Push harder.
This framing reflects a choice, not a necessity.
We don't criticize a human engineer for being bad at sports and force them to train in athletics; we accept that specialization is useful and reasonable. The same logic applies to AI. A system that's highly intelligent but narrow isn't deficient. A system that's capable but requires human direction isn't incomplete. These are design choices that preserve control and enable verification.
The race to AGI treats maximizing all three dimensions as the goal. The Pro-Human Tool AI framework recognizes that capability along one or two dimensions, with deliberate limits on others. This delivers what we actually want: powerful AI that remains under human control.
The question isn't "how do we maximize all dimensions?" It's "how do we get the benefits of AI capability while maintaining meaningful human control?"
The answer: keep at least one dimension in check.