Governance Attention Exploitation
Accountability & GovernanceDescription
Adversary generates excessive alerts, false positives, or trivial events to exhaust human oversight capacity, creating blind spots for real attacks. Exploits limited human attention in multi-agent monitoring environments.
Adversary triggers thousands of minor violations to overwhelm the security team while a major data exfiltration occurs undetected.
Academic studies have shown that GitHub Copilot and other code-generation LLMs produce insecure code at statistically significant rates (NYU/Stanford research found ~40% of Copilot suggestions in security-relevant contexts contained vulnerabilities). Confirmed production incidents where agent-generated code was executed and caused a security breach have not been publicly attributed specifically to AI-generated code.
Primary mitigations
- Alert deduplication and prioritisation
- automated triage of low-risk events
- attention economics monitoring
- human reviewer workload management.
Detection signals
Alert noise ratio; false positive rate; oversight saturation metrics.