Biased or unrepresentative training corpora
Data & Input IntegrityDescription
Skewed, imbalanced, or non-representative training data embeds systematic bias the model reproduces and amplifies in outputs.
A retail-credit assistant trained mostly on metro-city data gives worse guidance to rural applicants.
The 2021 UN Panel of Experts report on Libya described a Kargu-2 loitering munition potentially engaging targets without explicit human command, but the incident remains disputed and not independently verified as a deliberate autonomous lethal engagement. Adversarial misclassification attacks on object-recognition systems used in military contexts have been shown in controlled research, but no confirmed mass-casualty production incident attributable solely to AI malfunction has been publicly documented.
Primary mitigations
- Representativeness analysis
- dataset datasheets
- balanced sampling
- bias audits pre-train
- demographic coverage testing.
Detection signals
Subgroup performance gaps; statistical parity testing on outputs; dataset composition metrics.