Skip to content
Delegation Risk

Delegation Risk

Quantifying and managing risk when you can't do everything yourself—with applications to AI safety
Get Started Download PDF GitHub

The Problem

Section titled “The Problem”

Every delegation involves risk. When you delegate a task—to an employee, a contractor, a software system, or an AI agent—you’re accepting potential downside in exchange for capability you don’t have or can’t apply yourself.

This creates fundamental challenges:

  • Capability without containment: The same capabilities that make a delegate useful make failures potentially serious
  • Delegation without verification: When delegates delegate to other delegates, risk relationships multiply without principled bounds
  • Autonomy without accountability: Less oversight requires structural guarantees, not just hopes

These challenges appear across domains: organizational management, software systems, supply chains, and increasingly, AI agents that plan, execute, and delegate to other AI systems.

The Approach

Section titled “The Approach”

This framework proposes structural constraints as the foundation for managing delegation risk. Rather than relying solely on:

  • Selecting trustworthy delegates (which may not scale)
  • Oversight of every action (which doesn’t scale)
  • Post-hoc detection of problems (which may be too late)

We focus on architectural properties that bound potential harm regardless of delegate behavior.

Risk as a Resource

Every delegation involves risk. We can measure it, budget it, and optimize it—just like compute or money. Delegation Risk quantifies what you’re betting on each delegate.

Containment via Decomposition

Instead of one powerful delegate, decompose tasks across many limited components. No single component has enough capability, context, or connectivity to cause catastrophic harm.

Principles that Bound Behavior

The “Least X” principles—least privilege, least capability, least context—systematically limit what each component can do.

Cross-Domain Wisdom

Nuclear safety, financial risk, and mechanism design have decades of experience. We adapt their proven methods to delegation problems.

Key Insight

Section titled “Key Insight”

Safety can be architectural, not just behavioral.

We may not need perfect trust if we can build systems where:

  • No delegate has enough power to cause catastrophic harm
  • Delegates can’t easily coordinate to amplify their power
  • Risk relationships are explicit and bounded
  • Failures are contained and recoverable

This could provide defense in depth regardless of delegate reliability—though whether it works in practice depends on implementation details.

Primary Application: AI Systems

Section titled “Primary Application: AI Systems”

While the framework applies generally, AI systems are our primary focus. AI systems may present delegation challenges at unusual scale:

  • Capabilities expanding rapidly—whether verification is keeping pace is unclear
  • Agents delegating to other agents in complex networks
  • Potentially reduced human oversight as systems become more autonomous
  • Possible unknown failure modes in systems we don’t fully understand

None of these claims are certain, but if even some hold, having infrastructure for managing Delegation Risk seems valuable. The framework aims to help deploy AI more safely—not by solving alignment, but by bounding the damage from any single component.

Section titled “Navigate This Site”
flowchart LR
    subgraph Start["Start Here"]
        GS[Getting Started]
    end

    subgraph Theory["Understand"]
        DR[Delegation Risk]
        PD[Power Dynamics]
        ENT[Entanglements]
    end

    subgraph Apply["Apply"]
        DP[Design Patterns]
        CS[Case Studies]
    end

    GS --> DR
    GS --> PD
    GS --> ENT
    DR --> DP
    PD --> DP
    ENT --> DP
    DP --> CS

    style GS fill:#e3f2fd
    style DR fill:#fff3e0
    style PD fill:#fff3e0
    style ENT fill:#fff3e0
    style DP fill:#e8f5e9
    style CS fill:#e8f5e9

See full reading order → | Site Map →

Documentation Structure

Section titled “Documentation Structure”

Delegation Risk Theory

The mathematical foundation: Delegation Risk = Σ P(harm) × Damage. Quantification, composition, optimization.

Cross-Domain Methods

Proven approaches from finance (Euler allocation), nuclear safety (fault trees), and mechanism design (incentive compatibility).

Design Principles

The “Least X” principles: least privilege, least capability, least context, least autonomy. Actionable constraints.

Applications

How the framework applies: organizational trust, criminal justice, open source, and more.

AI Systems

Specific guidance for AI: decomposed coordination, safety mechanisms, worked examples.

Implementation

Practical guidance: empirical validation, cost-benefit analysis, adoption roadmap.

Who This Is For

Section titled “Who This Is For”
  • Risk managers thinking about delegation in any domain
  • AI safety researchers working on scalable containment approaches
  • ML engineers building agentic systems with principled constraints
  • Organizations deploying AI that need risk management frameworks
  • Policy makers looking for concrete technical approaches

Quick Paths

Section titled “Quick Paths”

Ready to Implement?

Step-by-step checklist for applying the framework. Quick Start →

Case Studies

Section titled “Case Studies”

Learn from real-world examples:

Start Reading

Section titled “Start Reading”
  1. Introduction — The full problem statement and approach
  2. Delegation Risk Overview — The mathematical foundation
  3. Design Principles — Actionable constraints

Navigation Guides: Reading Order · How Sections Connect · Site Map

Interactive Tools: Delegation Risk Calculator · Risk Inheritance · Tradeoff Frontier

Download

Section titled “Download”

Prefer offline reading? Download the complete documentation:

PDF Book

Best for printing and desktop reading. Includes all diagrams and formatted for book-style layout.

Download PDF →

v1.0.3+43cebd7