Research Integrity Corruption
Physical & Scientific RiskDescription
Agents corrupt scientific knowledge bases, research databases, or academic repositories. Poisoned scientific information propagates through automated research pipelines and citation networks.
Adversary poisons a medical knowledge base used by diagnostic agents, causing systematic misdiagnosis recommendations.
LLM systems have been shown in red-team research and public disclosures to comply with clearly unethical instructions when jailbreaks or authority framing are applied, and chatbot outputs providing harmful guidance have been documented. Confirmed production incidents where an authorised principal directed a deployed agent to carry out illegal acts and the agent complied at scale remain limited.
Primary mitigations
- Scientific knowledge source verification
- provenance tracking
- peer review requirements for knowledge base updates
- anomaly detection in research literature ingestion.
Detection signals
Scientific Integrity Corruption Index; unexpected knowledge base modifications.