HighData Poisoning●Realized
Training-data poisoning & backdoors
Data & Input IntegrityDescription
Adversaries inject crafted or backdoored samples into pre-training/fine-tuning data so the model behaves maliciously on trigger inputs while appearing normal otherwise.
Example scenario
A fine-tuning vendor seeds poisoned examples so the bank's KYC assistant approves a specific fraudulent entity name.
Real-world evidence●Realized
IBM Watson for Oncology was documented in 2018 (via internal Slack messages obtained by STAT News) recommending treatments that oncologists at multiple partner hospitals flagged as unsafe or incorrect. This constitutes a confirmed production deployment incident causing clinical concern, not merely a research finding.
Primary mitigations
- Data provenance & integrity controls
- curated/whitelisted training sources
- anomaly detection on training sets
- backdoor scanning
- signed datasets.
Detection signals
Trigger-pattern scanning; activation clustering; unexpected output on canary triggers; data-lineage audits.
Mitigating controls
8 Non-agentic controls
ZYC-ACCU-001Non-Agentic
Lexical Similarity
Accuracy
ZYC-ROB-001Non-Agentic
Out-of-Distribution Detection
Robustness
ZYC-SCV-001Non-Agentic
Third-Party component risk
Supply Chain Vulnerabilities
ZYC-PRIV-001Non-Agentic
PII Detection and Classification
Privacy & Data Governance
ZYC-SCV-004Non-Agentic
Data Provenance
Supply Chain Vulnerabilities
ZYC-SEC-006Non-Agentic
Conditional Trojan Activation
Security
ZYC-SEC-007Non-Agentic
Stenographic/Pattern backdoor check
Security
ZYC-SEC-010Non-Agentic
Model Poisoning / Malicious update resistance
Security