Search the Atlas

Search risks, controls, and glossary terms

HighAgenticHigh-Impact Tool AbuseDemonstrated

Critical Action Risk

Tool & Affordance Risk

Description

Agent invokes high-impact tools (payment systems, databases, communication channels, infrastructure APIs) without proportional authorisation. The blast radius of a single tool misuse extends far beyond agent scope.

Example scenario

Coding agent uses database admin credentials to 'optimise performance' by dropping indexes on production tables.

Real-world evidenceDemonstrated

Controlled research has shown that popular post-hoc explanation methods (LIME, SHAP, saliency maps) do not faithfully reproduce the model's internal computation and can pass sanity checks even when applied to random models. This has been published but has not been the direct attributed cause of a documented production harm incident.

Primary mitigations

  • Risk-tiered tool authorisation
  • mandatory human approval for critical-tier tools
  • impact scoring before tool invocation.

Detection signals

Critical Action Risk Score; high-impact tool invocation frequency.

Mitigating controls

7
Dual coverage

Related risks in Tool & Affordance Risk