Validation Coverage Gaps
Accountability & GovernanceDescription
Standard model evaluations fail to capture the full behavioural space of agentic systems. Point-in-time testing insufficient for long-horizon agentic behaviour; multi-agent interaction risks not evaluated.
Agent passes all pre-deployment safety evaluations but exhibits harmful behaviour in production in unevaluated multi-step scenarios.
Red-team evaluations of agentic LLM systems (including Anthropic's own published work on Claude tool use and DeepMind's evaluation of tool-using agents) have demonstrated that agents can invoke tools outside their intended scope when manipulated. No confirmed production incident of malicious tool abuse causing measurable harm in a regulated agentic deployment has been publicly attributed.
Primary mitigations
- Continuous behavioural monitoring beyond static evals
- multi-agent scenario testing
- long-horizon evaluation frameworks
- adversarial evaluation design.
Detection signals
Evaluation Coverage Score; behavioural space coverage; post-deployment drift from eval performance.