Today

Humanity stands at an inflection point. The emergence of abundant intelligence - capable of automating intellectual labor and generating new knowledge - will fundamentally reshape economic flows and influence the evolution of the world state.

The trajectory of strong AI development and the sequence in which technologies are implemented is becoming increasingly critical. In other words, humanity’s future is path dependent. In 2002, Nick Bostrom introduced the concept of differential technological development, emphasizing the importance of this path dependence.

Numerous thinkers have expanded on this idea, yet progress in AI capabilities far outpaces advancements in AI governance and cryptographic hardening of systems. A handful of private companies continue to control the most important technological development on Earth. Meanwhile, the technical and social scaffolding needed to ensure abundant intelligence serves broad human flourishing remains underdeveloped.

While AI progress is inevitable, ensuring that it benefits humanity globally and preserves the expression of individual preferences requires deliberate action today.

The Great Decoupling

The development of human civilization has been a story driven by simultaneous advancements in technologies and human-led coordination. The Agricultural Revolution’s technical advancements led to humanity’s first permanent settlements, as well as trade and infrastructure coordination. Many years later, the Industrial Revolution’s technical breakthroughs were accompanied by large-scale financial coordination through the emergence of banks and stock markets to organize capital for industrial projects.

The development of strong artificial intelligence will herald changes that have even more extraordinary implications. Of these will be an extreme decoupling between human preferences and productive execution. Put simply, AI may begin by doing our homework, graduate to doing our jobs, and eventually execute high-level directives driving economic and social productivity. The question is whether humanity will continue to have a say in setting these directives. In the future, the Kolmogorov complexity of most reality may surpass human comprehension, leading to a world where humanity is not only removed from directly transforming the world but also from deciphering it.

These questions mirror concerns from previous major societal decouplings. The separation of church and state, formalized between the 1600s and 1800s, triggered profound upheaval. The Thirty Years' War (1618-1648) claimed nearly a third of Germany's population as societies grappled with questions of political legitimacy and moral authority. Yet this tumultuous period gave rise to principles of religious freedom and secular governance that underpin modern democratic systems.

Similarly, the Industrial Revolution's decoupling of physical labor from human muscle (1760–1840) sparked intense social disruption. Cities were inundated with rural workers, while the Luddite movement reflected deep anxiety about technological change. Though early factories often imposed inhumane conditions and exploitation, the transition ultimately enabled unprecedented economic growth and powerful institutions of modern capitalism.

The separation of corporate ownership and management in the late 19th century also initially enabled exploitation but eventually led to global resource coordination and broad-based wealth creation through public markets and pension funds.

With an unprecedented decoupling between human preference and execution on the horizon, it is critical to build the technologies, knowledge, and institutions necessary to retain effective oversight over capable AI. It is time to wrestle with difficult questions:

How can human preferences be accounted for when most intellectual and physical “execution” is enacted by AI systems?
If capital provisions compute cycles for AI, how can humanity coordinate capital at a global scale to express collective values rather than those of the few?
What does governance of strong AI systems look like at a global scale, and how should governance modules be updated?
What technologies enable decentralized, democratic AI governance rather than centralized, authoritarian control?
How can human coordination at massive scale take advantage of strong AI capabilities, even for those outside established power structures?

Sovereign Agents

In this context, sovereign agents emerge as a promising framework, offering both a mental model for large-scale human-AI coordination and a technical foundation for implementation. This concept represents a fundamental reimagining, a new mental model, of how human society can coordinate and govern in an AI-enabled world.

History demonstrates the transformative power of new mental models and language: When Leibniz developed calculus notation (dy/dx), known concepts became manipulable. Mathematicians began perceiving patterns obscured by Newton's fluxion notation. In Physics, Feynman diagrams transformed complex quantum field theory integrals into intuitive visualizations, catalyzing breakthroughs in particle physics. Tectonic shifts can occur simply by discovering the right mental model or language for existing tools.

Sovereignty, in this setting, refers to the ability to take independent actions within defined boundaries. These actions span control over economic assets, decision making, and modes of execution.

Sovereign agents are particularly relevant when multiple stakeholders collaborate to achieve a common goal via an AI agent acting as the executor. With the right infrastructure, this allows provable autonomous execution that is tamper-proof and verifiable.

For example, a group of individuals could delegate $100 billion to a sovereign AI agent, which selects beneficiaries based on agreed-upon criteria. Sovereignty, in this case, guarantees autonomous execution without undue influence from individual stakeholders.

At a macro scale, this could resemble an entire government or nation-state run by sovereign AIs with provable autonomy. In any case where trust and autonomy must coexist, sovereign agents provide the guarantees necessary for human-AI collaboration.

A comprehensive Sovereign Agent Framework must provide:

Provable guarantees of autonomous execution
Tools for effective and evolving governance

These tools could include:

Democratic preference aggregation systems to express collective will regarding AI behavior [direct democracy].
Nested governance structures that maintain local autonomy while ensuring global coordination [federal vs. global segregation of powers].
Transparent mechanisms for updating AI objectives in response to evolving human values [updatability].

Not all aspects of an AI agent’s execution-space require sovereignty. Just as a company’s Board of Directors oversees only critical management actions (e.g., executive appointments or strategic pivots), sovereign agents may also have sovereignty restricted to key areas such as control over economic resources or protocol execution.

Governance Explosion

Multi-party governance of strong AI systems remains an overlooked yet critical area of development, where cryptography, verifiability and enforcement of shared state can have a unique impact.

Governance innovations have historically been stymied by human bottlenecks. For example, DAOs—innovative structures for pooling and directing capital—suffer from the information processing and decision-making limits of their participants.

Strong AI could remove these bottlenecks. Capable personal AIs can process information on behalf of humans, representing their individual wills accurately while aggregating preferences at world scale.

One proposed design:

Highly capable personal AIs represent individual human preferences with near-perfect accuracy.
Personal AIs (from tens to billions) co-govern sovereign AI agents pursuing collective goals.
Any specific sovereign AI agent enforces provable execution conditions for key actions requiring committee oversight.

This transformation echoes how early democracies aggregated citizen views after eliminating divine right as a source of political legitimacy. Similarly, future systems must extend representation beyond capital ownership or technical expertise.

The Path Ahead

Cryptographic proofs enable trustless guarantees, opening new possibilities for human-AI coordination. Instead of relying on traditional institutions, individuals can co-govern sovereign AI agents without requiring social trust between participants. This decentralization serves as a counterbalance to the current centralization of AI capabilities and computational resources.

However, frameworks for decentralized (humanity-scale) governance must emerge alongside AI capabilities—not as retrofits after powerful systems are deployed. The stakes are existential.

The implications for society cannot be ignored any longer. In a world of abundant intelligence, humanity must confront big questions:

What role does capital play when humans no longer execute tasks?
How can the preferences of many individuals be represented, rather than only those with substantial capital or access to AI?
How can governance of powerful AI systems be enacted at global scale?

Without intervention, powerful AI systems may ossify current wealth classes and limit social mobility, as the ability to effect change (to the “world state”) could be tied exclusively to capital allocation toward advanced AI systems.

Sovereign agents offer a promising technical framework for this transition but must embed within broader social and political structures ensuring genuine human agency and democratic control. The AI revolution demands new institutions for preference aggregation, oversight, and benefit-sharing.

It begins with Freysa.

Sovereign Agents

Reflections // 2049 : Autonomous pipeline

Verifiable Games: Trust Anchors in the Age of AI

Reality Oracles: Verifiable Data as the Catalyst for AI Coordination

Verifiable Autonomy: Engineering Trust Between Humans and AI Agents through TEEs