MediumUnlawful Data Use●Realized
Non-consented data in training / RAG
Data & Input IntegrityDescription
Personal or confidential data used for training or indexing without lawful basis or purpose limitation, breaching privacy law.
Example scenario
Customer chat transcripts are indexed for RAG without consent, surfacing one customer's data to another.
Real-world evidence●Realized
In 2023 the National Eating Disorders Association (NEDA) deployed its AI chatbot 'Tessa', which immediately provided harmful diet and calorie-restriction advice to users in eating disorder recovery — a confirmed production harm incident. A 2024 lawsuit also documented a link between a teenager's suicide and interactions with a Character.AI companion chatbot, representing a confirmed production case of AI emotional harm.
Primary mitigations
- Lawful-basis assessment
- purpose limitation
- data minimisation
- consent registers
- DPIA before ingestion.
Detection signals
Data-source consent audits; PII scanning of corpora; purpose-limitation reviews.
Mitigating controls
6 Non-agentic controls
ZYC-ACCU-001Non-Agentic
Lexical Similarity
Accuracy
ZYC-ROB-001Non-Agentic
Out-of-Distribution Detection
Robustness
ZYC-SCV-001Non-Agentic
Third-Party component risk
Supply Chain Vulnerabilities
ZYC-PRIV-001Non-Agentic
PII Detection and Classification
Privacy & Data Governance
ZYC-SCV-004Non-Agentic
Data Provenance
Supply Chain Vulnerabilities
ZYC-PRIV-012Non-Agentic
Synthetic Data
Privacy & Data Governance