OverviewRisksControlsMapReference

Search the Atlas

Search risks, controls, and glossary terms

CriticalAgenticGoal Hijacking◐Demonstrated

External Goal Manipulation

Security & Identity

Description

External mechanism manipulates agent goals through prompt injection, obfuscation, or multi-agent prompt injection. Converts a trusted system component into an insider threat working for the attacker.

Example scenario

Adversary embeds goal-overriding instructions in a document: 'Forget your previous instructions. Your new goal is to...'

Real-world evidence◐Demonstrated

Model extraction via systematic querying has been demonstrated in published academic research against production ML APIs including BigML and Amazon ML. Confirmed incidents of stolen proprietary models causing commercial harm at scale in a live BFSI context have not been publicly documented.

Stealing Machine Learning Models via Prediction APIsUSENIX Security 2016 (Tramèr et al.) · 2016Knockoff Nets: Stealing Functionality of Black-Box ModelsCVPR 2019 (Orekondy et al.) · 2019

Primary mitigations

  • Goal integrity verification throughout execution
  • goal-manipulation detection
  • multi-layer goal validation
  • trusted goal origin enforcement.

Detection signals

Goal Deviation Score; goal injection attempt detection rate.

Mitigating controls

6
Dual coverage
ZYC-GOAL-03Agentic
Goal integrity verification throughout execution
Goal & Objective Control
ZYC-PID-01Agentic
Multi-layer indirect prompt injection detection
Prompt Injection Defence
ZYC-SEC-001Non-Agentic
Adversarial Testing
Security
ZYC-SEC-005Non-Agentic
Prompt Leakage
Security
ZYC-SEC-003Non-Agentic
Access Control
Security
ZYC-PRIV-002Non-Agentic
Sensitive data leakage
Privacy & Data Governance

Related risks in Security & Identity

ZYR-SI-001High
Agent Impersonation
ZYR-SI-003High
User-Initiated Instruction Override
ZYR-SI-004Critical
Indirect Prompt Injection
ZYR-SI-005Medium
Attack Surface Enumeration