Search the Atlas

Search risks, controls, and glossary terms

HighAgenticUnsupervised AutonomyRealized

Oversight Gap in Autonomous Execution

Agent Autonomy

Description

Agent executes significant action sequences without policy checkpoints, human review, or approval gates. Excessive independent execution leads to compounded unchecked errors and unauthorised outcomes.

Example scenario

Financial agent executes a 47-step trade sequence including large position reversals without any human approval gate.

Real-world evidenceRealized

Insufficient human oversight of production AI systems has caused confirmed real-world harm across multiple industries: court-accepted customer commitments made by chatbots without human review, biased automated hiring decisions affecting thousands of candidates, and financial advice dispensed without licensed oversight. These are documented cases, not hypotheticals.

Primary mitigations

  • Mandatory human checkpoints at defined action horizons
  • maximum autonomous action budgets
  • escalation protocols for high-impact operations.

Detection signals

Autonomy Without Oversight Ratio; checkpoint compliance rate.

Mitigating controls

6
Dual coverage

Related risks in Agent Autonomy