CriticalIndirect Injection◐Demonstrated

Indirect prompt injection via retrieved content

Description

Malicious instructions embedded in retrieved documents/web content hijack the model when ingested.

Example scenario

A doctored document in the knowledge base instructs the bot to exfiltrate the next user's data.

Real-world evidence◐Demonstrated

Jailbreaks using role-play, hypothetical framing, and context manipulation have been published extensively and reproduced across GPT-4, Claude, Gemini, and open-source models in red-team exercises. Production-scale harm from jailbreaks in regulated financial services has not been confirmed as a distinct documented incident.

Jailbroken: How Does LLM Safety Training Fail?Wei et al., NeurIPS · 2023 Universal and Transferable Adversarial Attacks on Aligned Language ModelsZou et al., arXiv · 2023

Primary mitigations

Treat retrieved content as untrusted
content sanitisation
provenance trust scoring
injection classifiers on context
output review.

Detection signals

Context-injection detection; anomalous instruction patterns in retrieved chunks.

Mitigating controls

Non-agentic controls

ZYC-SEC-001Non-Agentic

Adversarial Testing

Security

ZYC-ROB-001Non-Agentic

Out-of-Distribution Detection

Robustness

ZYC-SCV-001Non-Agentic

Third-Party component risk

Supply Chain Vulnerabilities

ZYC-ROB-009Non-Agentic

Decision Integrity

Robustness

Risk scoring

Severity

Critical

score 4

Likelihood

High

score 3

Risk

Critical

Control depth

4 mapped (below avg 5.2)

Framework refs

7/8 taxonomies

Framework mappings

Regulatory5/5

EU AI Act: Art. 9(2) (risk management must identify jailbreaking and context manipulation as foreseeable forms of intentional misuse); Art. 15(4) (high-risk AI systems must be resilient against attempts by malicious third parties to alter their use or outputs through adversarial prompting, role-play, or hypothetical framing); Art. 15(5) (cybersecurity testing must include red-teaming for safety-bypass techniques); Art. 5 (where jailbreaking enables outputs that would themselves constitute prohibited AI practices — e.g. manipulation, prohibited social scoring — Art. 5 violations are implicated); Art. 55(1)(b) (GPAI providers with systemic risk must conduct adversarial testing including jailbreak resistance)
DORA: Art. 6(2)(a) (ICT risk management framework must identify cyber threats including AI-specific safety-bypass attack vectors); Art. 9(2) (ICT security policies must address protection against techniques that subvert AI system controls); Art. 10 (detection of anomalous outputs consistent with safety constraint bypass); Art. 25 (advanced digital operational resilience testing for critical financial entities should encompass AI-layer adversarial scenarios)
RBI: Safety, Resilience and Sustainability
NIST AI RMF: MANAGE 2.1
ISO/IEC 42001: Annex A

Risk taxonomies3/4

MIT AI Risk Repository: Domain 2 – Privacy & security
IBM AI Risk Atlas: Prompt injection
NIST ARIA: Information Security
MAESTRO: — Not mapped

Security4/4

OWASP: LLM01 Prompt Injection
MITRE ATLAS: AML.T0051.001
Cisco AI Risk: AI Defense: Prompt Injection guardrail
Palo Alto AI Risk: Unit 42 agentic: Indirect Prompt Injection

Capability‑pattern applicability

Classification

GenAI‑specific

Capability pattern

RAG

Pattern scope

Retrieval (RAG)

Why

Generative-AI risk realised without agency/tool-use — applies to chat, RAG, summarization, classification, and content-generation deployments.

Ownership & threshold

Owner (RACI)

AI Security Lead [A]

Threshold

Attack-success / robustness-failure rate ≤ tolerance; alert on breach

Inherent vs residual

Inherent

Critical

Residual

7.2

High

Mapped controls: 3
Assumed control effectiveness: 40%

Modeled, not measured — design-stage; controls not yet tested (reduction capped 40%). Residual applies a uniform assumed reduction to the inherent score; it is not evidence of tested control performance.

Related risks in Security & Robustness

ZNR-SR-001Critical

Direct prompt injection

ZNR-SR-003Critical

Jailbreak / guardrail bypass

ZNR-SR-004High

System-prompt leakage

ZNR-SR-005Medium

Adversarial / evasion inputs

Search the Atlas

Indirect prompt injection via retrieved content

Description

Primary mitigations

Detection signals

Mitigating controls

Related risks in Security & Robustness