Model Poisoning / Malicious update resistance

Explanation

Model poisoning and malicious-update resistance checks how well the system withstands attempts to corrupt it through tainted training data, compromised fine-tuning, or malicious model or dependency updates — attacks that can implant backdoors or degrade behaviour in ways that surface only later. For banking RAG and chat systems this matters because a poisoned model or knowledge base can quietly produce manipulated financial advice or leak data while passing routine checks. It is measured by a Model Poisoning / Malicious update resistance Rate; the control data gives no calculation method, so implement it as the share of poisoning or malicious-update test scenarios that successfully compromise the model (successful compromises divided by total tests) and record the exact definition you use. To implement and operate it, enforce supply-chain integrity — signed and version-pinned models, datasets and dependencies, provenance tracking, and verification of every update — validate training and ingested data, and run poisoning and update-tampering simulations as part of the model release pipeline; log every model and data change with its source, signature check and validation outcome as evidence. The threshold is ≤ 1% (target 0) with an alert on any breach: ideally no poisoning attempt succeeds, and any confirmed compromise triggers an alert, rollback to a known-good version, and investigation. Its priority is set to Later, so it is scheduled after immediate controls but still required.