An Optimal Knowledge Distillation for Formulating an Effective Defense Model Against Membership Inference Attacks

A membership inference attack (MIA) on machine learning models aims to determine the sensitive data that has been used to train machine learning models. Machine learning-based applications (MLaaS—machine learning as a service) in finance, banking, healthcare, etc. are facing the risks of private data leaks by MIA. Several solutions have been proposed for mitigating MIA attacks, such as confidence score masking, regularization, knowledge distillation (KD), etc. However, the utility-privacy trade-off problem is still a major challenge for existing approaches. In this work, we explore the KD-based approach to defending against MIA attacks. This approach has received increasing attention in the research community on machine learning safety recently as it aims at effectively addressing the above-mentioned challenge of mitigating MIA attacks. An efficient KD-based defense framework that includes multiple teacher and student models is proposed in this work for alleviating MIA attacks. Three main phases are deployed in this framework: (1) teacher model training; (2) knowledge distillation from the teacher model to the student model based on prediction augmentation and aggregation from the teacher model; and (3) repeated knowledge distillation among student models. The experimental results on standard datasets show the outperforms in both model utility and privacy of the proposed framework compared to other state-of-the-art solutions for mitigating MIA.

@article{Pham2024,
  title     = {An Optimal Knowledge Distillation for Formulating an Effective Defense Model Against Membership Inference Attacks},
  journal   = {International Journal of Advanced Computer Science and Applications},
  volume    = {15},
  number    = {5},
  year      = {2024},
  publisher = {The Science and Information Organization},
  author    = {Thi Thanh Thuy Pham and Huong-Giang Doan},
  doi       = {10.14569/IJACSA.2024.01505140},
  url       = {https://doi.org/10.14569/IJACSA.2024.01505140}
}