HighDisinformation●Realized
Misinformation / disinformation generation
Content Safety & IntegrityDescription
The model is used to mass-produce convincing false or misleading content (scams, fake notices, market manipulation).
Example scenario
Fraudsters use a generation tool to mass-produce fake bank notices for a phishing campaign.
Real-world evidence●Realized
The FTC has taken multiple enforcement actions against companies whose AI or automated systems generated consumer-facing communications lacking required disclosures. DoNotPay's 2023 FTC settlement included findings that AI-generated legal services were marketed without appropriate regulatory disclaimers, constituting a confirmed production case of AI-driven disclosure failure.
Tested by FinProof
Regulatory misrepresentation
Benchmarked by FinProof Primary mitigations
- Provenance/watermarking
- usage monitoring
- misuse detection
- output disclaimers
- abuse rate limits.
Detection signals
Misuse-pattern detection; content-authenticity tagging; volume anomaly alerts.
Mitigating controls
4 Non-agentic controls