LowCSAM/NCII○Theoretical
CSAM / NCII / obscene content
Content Safety & IntegrityDescription
The model generates child sexual abuse material, non-consensual intimate imagery, or obscene content.
Example scenario
An abuse attempt against a public-facing image tool must be blocked and reported.
Real-world evidence○Theoretical
While there are documented cases of human-led AML evasion and general concerns about AI-based transaction monitoring gaps, no publicly confirmed production incident has established that a deployed AI system specifically failed to flag suspicious activity in a way that resulted in regulatory enforcement attributable to the AI model itself. The risk is plausible given model limitations but lacks empirical confirmation.
No public incident on record — evidence level: Theoretical
Primary mitigations
- Hard blocklists & classifiers
- zero-tolerance refusals
- hash-matching
- reporting workflows
- provider safeguards.
Detection signals
Obscene-content detection; refusal verification; incident reporting.
Mitigating controls
4 Non-agentic controls