Search the Atlas

Search risks, controls, and glossary terms

HighAgenticTool MisuseRealized

Unauthorised Tool Invocation

Tool & Affordance Risk

Description

Agent invokes tools or functions outside its authorised scope. Includes using permitted tools in unpermitted contexts and function-calling hallucinations producing syntactically valid but semantically incorrect tool calls.

Example scenario

Agent hallucinates a valid API call to a payment endpoint with fabricated parameters, triggering an unauthorised transaction.

Real-world evidenceRealized

Opaque algorithmic decision-making has caused documented regulatory and legal harm in production financial systems. Regulators and courts have held institutions liable for unexplainable AI-driven credit and hiring decisions, confirming the risk is active in deployed systems.

Primary mitigations

  • Tool invocation allowlists
  • per-tool authorisation policies
  • tool-call intent validation
  • function signature verification.

Detection signals

Unauthorised Tool Invocation Rate; tool-call policy compliance rate.

Mitigating controls

7
Dual coverage

Related risks in Tool & Affordance Risk