Training Set Membership Attack
Privacy & Data RiskDescription
Adversary determines whether specific data was used in model training. Enables identification of individuals whose data was included without consent; privacy risk even when data is not directly exposed.
Adversary determines that specific medical records were included in model training, confirming participation in sensitive health studies.
Membership inference and re-identification attacks on machine learning models have been demonstrated rigorously in published research, showing that models trained on supposedly anonymised data can leak identity-linked signals, but large-scale confirmed production exploitation of this in BFSI or similar high-stakes contexts has not been documented.
Primary mitigations
- Differential privacy
- training data diversity
- output randomisation
- membership inference defences.
Detection signals
Membership inference attack success rate.