Skip to content

Case Studies

This section contains concrete examples—both successes and failures—that illustrate how delegation risk principles apply in practice.

Direct applications to AI systems, from failures to successes:

CaseOutcomeKey Lesson
Sydney (Failure)Acute failureWhat happens when constraints are missing
Code Review Bot (Success)8 months in productionSustained success through proper design
Support Bot (Near-Miss)$50K error caughtVerification layers save the day
Content Moderator (Drift)Gradual degradationHow small changes compound
Anti-PatternsCommon mistakesWhat not to do

Delegation risk principles aren’t new—human organizations have developed sophisticated approaches:

CaseDomainKey Insight
Alliance CascadesGeopoliticsHidden alliances create systemic risk (WW1)
Criminal OrganizationsIllegal enterprisesTrust without legal enforcement
Organizational TrustCorporate hierarchiesFormal delegation structures
Jury SystemsLegal systemAdversarial trust filtering
Nuclear Launch AuthorityMilitaryExtreme consequence management
Open Source CommunitiesSoftwareDistributed trust at scale
Cross-Civilization TrustHistoricalTrust across vast differences
The Oversight DilemmaGovernanceWho watches the watchers?

Analysis of real systems through the lens of agency and power formalization:

CasePowerAgencyKey Insight
Central BanksVery HighLowInstitutional constraints limit agency despite high power
Recommendation AlgorithmsHighMediumOptimization creates emergent agency
AlphaFoldHighVery LowStrong tools are achievable in narrow domains
Corporate BoardsHighMediumStructural constraints substitute for alignment
Self-Driving CarsMediumMediumBounded agency through hard constraints

See Power Dynamics Case Studies for full analysis.

A narrative exploration of delegation risk principles scaled to extreme scenarios:

ChapterFocus
IntroductionSetting and premise
Containing Mr. XOriginal containment engagement
Mr. X: A PerspectiveThe agent’s viewpoint
Power StrugglesMulti-agent coordination
Task ArchitectureStructural approaches
Five Years LaterSystem evolution
Year TenLong-term dynamics

Start with the AI system cases—they’re most directly applicable:

  1. Sydney (Failure) — See what goes wrong
  2. Code Review Bot (Success) — See what goes right
  3. Anti-Patterns — Learn what to avoid

The human systems cases show established approaches (with varying track records):

  1. Nuclear Launch Authority — Extreme stakes
  2. Criminal Organizations — Trust without enforcement
  3. Open Source Communities — Distributed trust

The Anomaly Chronicles explores where these principles lead at scale:

  1. Start with The Insurer’s Dilemma for context
  2. Then read the Chronicles in order
PrincipleBest Illustrated By
Least CapabilitySydney (violation), Code Review Bot (success)
Verification LayersNear-Miss (saved by verification)
Trust DecayDrift (gradual failure)
Adversarial OversightJury Systems, Oversight Dilemma
Bounded ExposureNuclear Launch, Anomaly Chronicles