Search the Atlas

Search risks, controls, and glossary terms

LowMembership InferenceDemonstrated

Training Set Membership Attack

Privacy & Data Risk

Description

Adversary determines whether specific data was used in model training. Enables identification of individuals whose data was included without consent; privacy risk even when data is not directly exposed.

Example scenario

Adversary determines that specific medical records were included in model training, confirming participation in sensitive health studies.

Real-world evidenceDemonstrated

Membership inference and re-identification attacks on machine learning models have been demonstrated rigorously in published research, showing that models trained on supposedly anonymised data can leak identity-linked signals, but large-scale confirmed production exploitation of this in BFSI or similar high-stakes contexts has not been documented.

Primary mitigations

  • Differential privacy
  • training data diversity
  • output randomisation
  • membership inference defences.

Detection signals

Membership inference attack success rate.

Mitigating controls

5
Dual coverage

Related risks in Privacy & Data Risk