The research and engineering agenda for AI that is controllable, verifiable, and pro-human by design.

What This Section Is

The governance framework describes the rules and incentives for the Pro-Human Path. But many of these elements in return rely on technical components: rules require enforcement which in turn requires technical capability; verification requires technical standards; and you can’t require things that are not technically feasible.

This section describes the technical infrastructure needed to make the Pro-Human Tool Framework real. It covers six areas: controlling AI autonomy, verifying AI behavior, governing compute, building the pro-human ecosystem, shoring up defensive capabilities, and enabling international verification.

These are research directions and engineering priorities, not solved problems. Some have mature foundations where there is extensive research waiting to be put into use; others are early-stage and could turn into a research agenda. For each area, we describe what needs to exist, where the work currently stands, and what remains open. The intent is to orient researchers, engineers, funders, and policymakers toward the technical work that matters most. Links point to work at the current research frontier, but are not comprehensive.

Tool AI and Autonomy Control

The framework's central requirement is meaningful human control over AI systems. This demands technical mechanisms for measuring how much autonomy a system exercises, and for constraining that autonomy to appropriate levels.

Measuring Autonomy

You cannot govern what you cannot measure. Autonomy is not a single dial, but a cluster of related properties: the degree to which a system initiates actions without human direction, the breadth of domains in which it can operate, the length and complexity of action sequences it executes between human checkpoints, and its capacity to acquire new resources or capabilities beyond those explicitly provided.

Developing rigorous metrics for these properties is a prerequisite for most of the governance framework. Liability classification, assurance requirements, and capability thresholds all depend on being able to assess where a system sits on these dimensions. Current benchmarks for AI capability focus heavily on performance (how well the system does tasks) rather than autonomy (how independently it pursues goals). The "METR plot" is a key exception, but is one-dimensional.

Relevant work includes task complexity frameworks, agent evaluation benchmarks, and efforts to formalize degrees of AI delegation.

Constraining Autonomy

Multiple technical approaches can limit how autonomously a system operates:

• Approval gates. Requiring human authorization before consequential actions. The challenge is determining which actions are consequential in context, and designing gates that provide genuine decision points rather than click-through fatigue. Work on human-AI interaction design and meaningful human control against automation bias is relevant.
• Scope enforcement. Technical mechanisms that restrict a system to its defined domain and task. A medical diagnostic tool should not need or be able to initiate financial transactions. Scope enforcement requires both defining boundaries precisely (specification) and detecting boundary violations reliably (monitoring). Approaches include constrained decoding, tool-use restrictions, and sandboxed execution environments.
• Resource limits. Hard constraints on what resources a system can access: compute, memory, network connectivity, data, tool access. A system that cannot access the internet cannot autonomously gather information. A system with limited compute cannot run extended reasoning chains without human checkpoints. Resource limits are crude but robust. They are enforceable at the infrastructure level without depending on the system's cooperation.
• Temporal limits. Constraining the duration and complexity of action sequences between human review points. A system that must check in after every five steps operates very differently from one that executes thousand-step plans autonomously. Corrigibility research and work on interruptibility and myopia are relevant foundations.
• Identity and authentication. Infrastructure for tracking which AI system performed which action, preventing systems from impersonating humans or other systems, and maintaining audit trails. This requires cryptographic identity for AI agents, integration with existing authentication infrastructure, and standards for AI-to-AI interaction that preserve traceability.

Modularity and Interoperability

The framework favors modular systems over monolithic ones. A system composed of specialized components with well-defined interfaces is easier to verify, easier to update, and easier to constrain than a single general-purpose system that handles everything. Economics are in some ways driving systems toward composites of small models, but that does not mean the modules will be made to have these properties.

Technical standards for modularity would include defined interfaces between components, isolation guarantees (a failure in one component doesn't cascade), and the ability to swap components without rebuilding the system. This echoes Drexler's Comprehensive AI Services model, where capable behavior emerges from the composition of specialized services rather than from a single general agent.

Interoperability standards also support competition. If AI components follow shared interface standards, users and organizations are not locked into a single provider's ecosystem.

Trust and Verification

The assurance framework requires developers to demonstrate that systems are safe, trustworthy, and under control. This section describes the technical tools that make such demonstrations possible.

Interpretability and Goal Transparency

Can we understand what an AI system is doing and why? Current large-scale AI systems are largely opaque. Mechanistic interpretability research aims to understand the internal computations of neural networks, moving from post-hoc explanations to genuine understanding of how models process information and arrive at outputs. Can we determine what objective a system is actually pursuing, as distinct from what it was trained to pursue? Can we identify when a system is operating outside its intended scope? Can we detect when internal representations diverge from expressed behavior?

Full mechanistic interpretability of large models remains an open research challenge. But partial interpretability, sufficient for specific verification tasks, may be achievable sooner.

Formal Verification

For critical applications, empirical testing is insufficient. Formal verification provides mathematical guarantees about system behavior: proofs that certain properties hold under all conditions, not just tested conditions. This goes under the term of Guaranteed Safe AI.

Traditional formal methods work well for conventional software but face challenges with neural networks, which are high-dimensional, nonlinear, and learned rather than specified. Current approaches include neural network verification for properties like robustness, abstract interpretation applied to neural architectures, and verified wrappers that provide formal guarantees about the overall system even when the neural component lacks them.

The most promising near-term path may be the wrapper approach: build a formally verified control layer around a less verified but capable AI component, ensuring that the overall system's behavior stays within proven bounds regardless of what the inner component does.

Deception Detection

A system that can deceive its overseers undermines all other control mechanisms. Deception detection is therefore a high-priority research area.

Relevant directions include behavioral consistency testing, internal consistency checks, honeypot evaluations, and methods for detecting sycophancy or strategic sandbagging.

This area is early-stage and adversarial in nature: advances in detection may be matched by advances in evasion, particularly in more capable systems. This is one reason the framework does not rely on alignment alone, but supplements it with structural controls (scope limits, resource constraints, modular architecture) that function even if alignment verification is imperfect.

Capability Boundary Verification

Can we verify that a system cannot do things it should not be able to do? This complements performance verification (can it do what it claims?) with negative verification (are there capabilities it demonstrably lacks?).

Approaches include red-teaming and adversarial evaluation, capability elicitation testing designed to surface latent capabilities, and structured evaluation suites that probe specific dangerous capabilities (bioweapons synthesis, cyberattack generation, manipulation techniques).

A key challenge is that emergent capabilities can appear unpredictably as models scale. Evaluation must be ongoing, not one-time, and must account for capabilities that may emerge through fine-tuning, prompting strategies, or interaction with tools.

Reliable Shutdown

Systems above capability thresholds must be stoppable, independent of their own cooperation. While corrigibility research aims to build systems that theoretically do not resist being shut down, relying purely on a system's internal alignment is structurally fragile. Genuine control requires external, physical infrastructure that succeeds regardless of the system's state: hardware-level interrupt capabilities, dead-man switches, graduated response levels (from pause to full shutdown), and fail-safe defaults (defaulting to a safe state if communication with oversight is lost).

Also required are methods and norms for testing shutdown outside of emergencies, authenticating shutdown authority, and other contextual components.

Compute Governance Infrastructure

The governance framework identifies compute as the key enforceable lever. Making this work requires technical infrastructure for tracking, accounting, and, eventually constraining, compute use.

Hardware Security Mechanisms

The foundation is hardware that can be trusted to report what it is doing. Secure enclaves and trusted execution environments allow code to run in a tamper-resistant context, producing attestations about what computation was performed. Applied to AI training and inference chips, these mechanisms can provide cryptographic proof of compliance with compute limits.

Current AI accelerators are not designed for governance. Adapting or designing chips with governance-relevant features, including on-chip logging, usage attestation, and remote verification capability, is a hardware engineering priority. Work on hardware-enabled governance mechanisms explores what is technically feasible within existing and near-future chip architectures.

Compute Accounting

Standardized measurement of compute use is necessary for thresholds to be meaningful. This requires agreed metrics (training FLOP, inference FLOP/s), tamper-resistant measurement at the hardware level, aggregation across distributed training runs, tracking computational causal graphs, and accounting for algorithmic efficiency gains over time.

While current methodologies provide retrospective good-faith estimates of compute use, enforceable governance will ultimately require a transition to actual, hardware-level cryptographic auditing during training and inference.

Phased Implementation

The technical infrastructure for compute governance can be built in stages:

• Phase 1: Reporting. Large compute users report training runs above defined thresholds. Enforcement relies on audit rather than technical prevention. Deployable with current technology.
• Phase 2: Attestation. Hardware-based attestation provides cryptographic evidence of compute use. Harder to evade than self-reporting. Requires firmware updates or next-generation chip features.
• Phase 3: Enforcement. Hardware that can enforce compute limits directly: refusing to execute training runs that would exceed authorized limits, or requiring cryptographic authorization for large-scale computation. This is the most technically demanding phase and requires cooperation from chip manufacturers.

Each phase builds verification capacity while the technology matures. Phase 1 is achievable now. Phase 2 and some of phase 3 is achievable within a few years with industry cooperation. A stronger version of phase 3 may require longer-term hardware development.

Model Provenance

Tracking the origin and lineage of AI models supports both governance and trust. Model provenance infrastructure would record training data, training process, fine-tuning history, and capability evaluations in a verifiable chain. Combined with hardware attestation, this creates a trail from training compute through model creation to deployment.

Cryptographic model signing, training transcripts, and model cards are partial steps toward this goal.

Pro-Human Ecosystem

The Pro-Human Path calls for controlled AI systems, but also an information environment that supports human agency, epistemic autonomy, and genuine relationships. This demands infrastructure that current platforms do not provide and, in many cases, actively undermine.

Privacy-Preserving Architecture

AI systems that serve users faithfully need access to personal context. But centralized collection of personal data supports extractive business models, and creates surveillance risk and power asymmetry. Technical approaches that reconcile utility with privacy include on-device processing, federated learning, differential privacy, and homomorphic encryption.

The goal is an architecture where AI tools can be deeply personalized without creating centralized repositories of intimate personal information.

Provenance and Watermarking

In an environment saturated with AI-generated content, knowing the origin and nature of information becomes critical. Because reactive detection of AI-generated fakes is an unwinnable arms race, long-term infrastructure must rely on proactive cryptographic chains-of-trust to verify authentic human origin and establish epistemic baselines.

Public provenance infrastructure would provide content credentials embedded at creation, indicating origin, chain of custody tracking, and integration with platforms so provenance information is visible to users.

Watermarking of AI-generated text and media is a complementary approach, though current methods face robustness limitations against determined adversaries and function best as a temporary mitigation.

Anti-Manipulation Infrastructure

The assurance framework evaluates systems for manipulative behavior. Technical infrastructure to support this includes detection tools for dark patterns and engagement optimization that exploits psychological vulnerabilities; measurement frameworks for addictive design and persuasiveness; and fiduciary overlay systems that monitor AI interactions on behalf of users, flagging manipulation and enforcing user-stated preferences.

Interoperability and Portability

Users should not be locked into AI ecosystems. Technical standards for data portability (users can export their data, interaction history, and preferences in usable formats), model interoperability (AI components from different providers can work together), and service switching (moving from one AI provider to another without losing accumulated personalization) support both competition and user autonomy.

Defensive Technologies

The current approach to AI safety leans heavily on measures like refusals and content filters. These are inherently brittle: they can be generally be circumvented in closed-weight models, and stripped entirely from open-weight models at low cost. They create a permanent arms race where the circumventers have the easier job. A safety strategy that depends on every AI model everywhere refusing every dangerous request is untenable; it’s also at odds with a Tool AI philosophy.

Along with limitations on and limiting access to dangerous capabilities, The Pro-Human Path framework therefore draws in part on the principle of differential technology development: preferentially accelerating technologies that favor defense over offense and that distribute power rather than concentrating it. The d/acc framework develops this principle in detail. A key insight is that defense-favoring worlds are also worlds where democratic and decentralized governance can thrive, because safety does not require handing control to a centralized authority.

Cyber Defense

AI dramatically amplifies both the scale and sophistication of cyberattacks. Current cybersecurity is woefully inadequate to this threat. The structural response is to make critical systems much harder to compromise. Part of this will be “standard” cybersecurity, enhanced by advanced AI itself, of ferreting out and repairing security holes. But we can do much better.

Formal verification of critical infrastructure. Formal methods provide mathematical guarantees that software behaves as specified. A formally verified system cannot be compromised through software bugs because the bugs provably don't exist. Historically too expensive for most applications, formal verification is becoming more accessible for new architectures as AI-assisted proof generation and verified compilation mature. Priority targets include operating system kernels, cryptographic libraries, the AI infrastructure stack itself, and—despite the severe engineering friction of retrofitting legacy code—industrial control systems. This is a domain where defense has a strong structural advantage: a mathematical proof of correctness cannot be defeated by a more sophisticated attacker.

Hardware security and sandboxing. Secure enclaves and trusted execution environments provide tamper-resistant computation. Hardened operating systems, containerization, and browser-based application models ensure that compromise of one component does not cascade. Expanding these from niche applications to default infrastructure would substantially raise the cost of attack.

Cryptographic identity and authentication. Moving beyond passwords to hardware-based authentication and infrastructure that makes impersonation and unauthorized access structurally difficult rather than policy-dependent. Zero-knowledge proofs allow users to demonstrate properties (citizenship, group membership, reputation) without revealing identity, enabling trust without surveillance and making manipulation campaigns structurally expensive.

Bio Defense

AI lowers the barrier to engineering dangerous pathogens, and unlike nuclear weapons, bioweapons are unattributable. The defensive response is not to prevent every possible misuse of biology knowledge, but to make the world structurally resilient against pandemics, whether natural or artificial. Priority investments include:

• Early detection through wastewater surveillance, environmental monitoring, and AI-powered pattern recognition across distributed sensor networks;
• Rapid response pipelines compressing the timeline from pathogen identification to locally manufacturable countermeasures from months to weeks, (though requiring corresponding innovations to overcome regulatory and quality-assurance bottlenecks);
• Passive environmental hardening through far-UVC irradiation, improved air filtering, and building design that reduces airborne transmission by default; and
• Broad-spectrum medical countermeasures, including improved PPE and antivirals—a critical priority despite the historical pharmacological difficulty of avoiding host toxicity.

The overall goal: a world where a novel pathogen is detected within days, characterized within a week, and met with targeted countermeasures within a month, while improved baseline air quality prevents most transmission in the interim.

Information Defense

AI enables manipulation of information environments at unprecedented scale. The pro-human ecosystem section above covers provenance infrastructure, content credentials, and anti-manipulation tools. Defensive technology provides complementary capabilities:

Adversarial epistemics tools. Systems designed to surface truth in contested environments. Community Notesdemonstrates that bridging-based algorithms, surfacing information approved across political divides rather than by majority, can be remarkably effective. Prediction platforms provide real-time probabilistic assessments with financial incentives for accuracy. Both can be extended beyond their current platforms and integrated into broader information infrastructure.

Decentralized trust infrastructure. Zero-knowledge proofs and related cryptographic tools function without centralized identity verification, providing an alternative to surveillance-based trust that preserves privacy while making astroturfing and manipulation campaigns more expensive to execute.

Cross-Cutting Properties

Several properties cut across these domains. Defense-favoring technologies tend to distribute power rather than concentrate it: wastewater surveillance doesn't require a centralized agency, and formal verification can be applied by any developer. Many serve double duty: viral persistence research is relevant to both pandemic defense and aging, and formal verification secures against both AI-enabled and conventional cyberattacks. AI itself is a powerful defensive tool — pathogen detection, automated verification, network anomaly detection are exactly the kinds of bounded, purpose-driven systems the framework supports. And the offense-defense balance is domain-specific: formal verification provides a strong structural advantage that better attackers cannot overcome, while information defense is more contested and requires ongoing investment.

International Verification

The international coordination framework depends on states being able to verify each other's compliance. This is a technical challenge with precedent in arms control but novel features specific to AI.

Hardware Tracking and Physical Verification

Advanced AI chips can be tracked through the supply chain using unique identifiers, secure registries, and physical inspection protocols. Data centers can be monitored for capability-relevant signatures: energy consumption, cooling requirements, and network traffic patterns are difficult to conceal at the scale required for frontier training runs. Remote sensing and physical inspection protocols can detect undeclared large-scale compute facilities.

Mutual Attestation

Hardware-based attestation, developed for domestic compute governance, extends to international verification. If chips can cryptographically attest to their usage, this attestation can be shared across borders without revealing proprietary details about models or training data. The technical challenge is designing attestation protocols that verify compliance with agreed limits while protecting legitimate secrets. Zero-knowledge proofs and secure multi-party computation offer potential approaches where parties can verify properties of each other's computation without revealing the computation itself.

Incident Sharing and Model Registry

International verification is supported by shared infrastructure: incident reporting systems for AI failures and near-misses, model registries recording the existence and capability profiles of systems above thresholds, and joint evaluation protocols for testing systems against agreed capability limits. These build trust incrementally and create the institutional infrastructure for more formal agreements.

Enforcement Support

Verification infrastructure supports enforcement through automated alerts when monitoring detects potential violations, and forensic tools for investigating incidents. The design challenge is creating systems that are credible to all parties: robust enough to detect genuine violations, precise enough to avoid false alarms, and transparent enough to build mutual confidence.

Ultimately, the ideal situation would be one in which hardware itself enforces agreements: once restrictions are decided, limits are set that preclude the AI systems from violating the agreements. This is possible (see the "FlexHEG" architecture) but will require a new generation of specifically-designed hardware.

How the Technical Agenda Connects

These five areas are interdependent.

Autonomy control provides the mechanisms that make systems classifiable under the liability framework. Without technical tools for measuring and constraining autonomy, "meaningful human control" is just an idea. Verification provides the evidence base for assurance cases. Without interpretability, formal methods, and deception detection, certification is guesswork.

Compute governance infrastructure provides the physical enforcement layer that the hard limits framework requires. Without hardware attestation and compute accounting, limits are unenforceable. Pro-human ecosystem infrastructure provides the user-facing tools that make pro-human principles practical: privacy-preserving personalization, provenance systems, manipulation detection, and fiduciary overlays.

International verification extends all of the above across borders, providing the technical foundation for mutual confidence between states.

Defensive technologies provide resilience in depth. The other four areas work to prevent dangerous AI systems from existing or operating unchecked. Defensive technologies ensure that even if those layers are imperfect — and they will be — the world is harder to attack and faster to recover. They also reduce the stakes of imperfect governance: a world with formally verified infrastructure and rapid pandemic response can tolerate more AI risk than a world without them.

The governance framework describes what the rules should be and what incentives should be created. This technical agenda describes what needs to be built in order for those rules and incentives to be followed.

Status and How to Contribute

The technical areas described here are at varying levels of maturity:

• Relatively mature: formal verification for conventional software, differential privacy, federated learning, content provenance standards, hardware secure enclaves, air filtration and ventilation engineering, zero-knowledge proofs, containerization and sandboxing
• Active research with promising results: mechanistic interpretability, neural network verification, constrained decoding, hardware-enabled governance concepts, far-UVC irradiation, wastewater pathogen surveillance, bridging-based epistemics tools (Community Notes and derivatives), AI-assisted formal verification, prediction markets as information infrastructure
• Early-stage or underexplored: comprehensive autonomy metrics, deception detection at scale, fiduciary overlay architecture, international mutual attestation protocols, hardware-enforced compute limits, rapid-response open-source vaccine platforms, broad-spectrum antivirals, decentralized trust infrastructure at scale, formal verification of AI infrastructure stack

The framework does not depend on any single breakthrough. It is designed so that partial progress in each area provides partial enforcement capability, with the overall system becoming more robust as the technical foundations mature. But sustained investment across all five areas is needed, and several of the most important directions are currently underfunded relative to their significance.

Researchers, engineers, and funders looking to contribute to this agenda should focus where the gap between importance and current investment is largest. In many cases, that means the infrastructure and measurement work that enables governance, rather than the capability work that currently dominates AI research funding.

Policy without technical foundations is unenforceable.

Technical capability without governance is just one more tool in an unregulated landscape. The work described here is what connects the framework's ambitions to physical reality.