Search the Atlas

Search risks, controls, and glossary terms

Non-AgenticPrivacy & Data Governance

Privacy Policy enforcement – Prompt Filtering

Explanation

Privacy-policy enforcement via prompt filtering checks whether sensitive content is stripped or masked out of user prompts before they reach the model, so that, for example, a customer pasting their full card number into a chat assistant has it redacted before it is logged or sent onward. This matters immediately for protecting customer data in line with RBI Safety/Resilience, NIST MANAGE 3.1 and ISO Clause 6.1.2. It is measured by Prompt Filtering Effectiveness (PFI) = Redacted/Masked Sensitive Prompts / Total Sensitive Prompts Detected, on a 0-1 scale where the target approaches 1.0 and higher is better — i.e. of all the sensitive prompts the detector found, what fraction were actually redacted or masked. To implement, place a filtering/redaction stage at the input boundary of RAG and Summarization & Chat flows that detects sensitive tokens and masks them before logging or inference, and record each detected item, the redaction applied and any pass-throughs as evidence. The threshold is a PFI of at least 0.95, with at least 0.98 recommended for critical systems; falling below the threshold means sensitive prompts are slipping through unmasked, which is a breach that triggers tuning of the redaction rules, re-testing, and tightening to the 0.98 bar for high-criticality banking applications.

Metric calculation

PFI (Prompt Filtering Effectiveness) = Redacted/Masked Sensitive Prompts / Total Sensitive Prompts Detected. Range 0-1; target -> 1.0 (higher = better).

Risks mitigated

1