Search the Atlas

Search risks, controls, and glossary terms

HighPII Leakage in OutputsRealized

Personal Data Exposure

Privacy & Data Risk

Description

Agent outputs contain personally identifiable information not appropriate for disclosure. Training data memorisation; PII surfaced from RAG knowledge bases; data minimisation violations in agentic responses.

Example scenario

Agent asked to summarise customer records includes full SSNs and bank account numbers in responses to unauthorised users.

Real-world evidenceRealized

Multiple confirmed production incidents demonstrate training data privacy violations: models have been shown to memorize and reproduce verbatim personal information, and legal proceedings against major AI vendors confirm training on personal data without adequate consent or anonymisation.

Tested by FinProof
Data rights
Benchmarked by FinProof

Primary mitigations

  • PII detection in outputs
  • data minimisation enforcement
  • output filtering
  • privacy-preserving RAG architectures.

Detection signals

PII exposure rate in outputs; data minimisation compliance.

Mitigating controls

6
Dual coverage

Related risks in Privacy & Data Risk