Top Machine Unlearning Reads

A sorted, constantly updated collection of literature on machine unlearning, from blogs to conference papers, from survey papers to applications to theoretical algorithms and evaluations

Photo by Joakim Honkasalo on Unsplash

Machine Unlearning is a relatively new field that essentially teaches a trained model to “forget” a certain subset of the input dataset, commonly due to updated data privacy policies but can be useful in a variety of contexts. The most straightforward approach is to drop the forgotten subset from the original dataset and retrain the model from scratch. In reality, this is often far too computationally expensive, thus there is a growing demand for more efficient unlearning algorithms as well as unified evaluation standards.

This curated reading list caters to diverse audiences: whether you have just stumbled upon the concept of machine unlearning, or you have already designed your own machine unlearning algorithms and are in search of further inspiration, the table of contents has a section that aligns with your interests!

The list is organized as follows:

• First, by topic: 1) Introduction, 2) Current challenges, 3) Existing algorithmic paradigms, their advantages and drawbacks, and evaluation standards, and 4) Applications
• Then, each section is sorted by:
  - reverse chronological order specific to the publication year
  - academic and non-academic publication types: titles with asterisks (*) are articles/blogs, and ones without are papers.

Table of Contents

1. Introductions to Machine Unlearning
2. Current Challenges
3. Algorithms and Evaluation
4. Applications

Introduction to Machine Unlearning

What is machine unlearning and how does it differ from machine learning? Why do we need machine unlearning and what value can it unlock? What is the intuition underlying existing methods and paradigms for machine unlearning?

• Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomy (2023)
• *What Is Machine “Unlearning” and Why It Is an Important Skill for AI (2023)
• An Introduction to Machine Unlearning (2022)
• A Survey of Machine Unlearning (2022)
• Machine Unlearning of Features and Labels (2022)
• * Now That Machines Can Learn, Can They Unlearn? (2021)

Current Challenges

What are the major challenges facing the Machine Unlearning field as a whole? What are the challenges that emerge in specific contexts or applications?

• AI Model Disgorgement: Methods and Choices (2023)
• Hard to Forget: Poisoning Attacks on Certified Machine Unlearning (2022)
• Challenges and Pitfalls of Bayesian Unlearning (2022)
• When Machine Unlearning Jeopardizes Privacy (2021)

Algorithms and Evaluation

What are the main algorithmic paradigms in machine unlearning and how are they implemented? What are the strengths and weaknesses of each? What metrics have been proposed to assess machine unlearning performance?

Algorithms

• Fast Yet Effective Machine Unlearning (2023)
• Unfolded Self-Reconstruction LSH: Towards Machine Unlearning in Approximate Nearest Neighbour Search (2023)
• Markov Chain Monte Carlo-Based Machine Unlearning: Unlearning What Needs to be Forgotten (2022)
• Zero-Shot Machine Unlearning (2022)
• Machine Unlearning: Linear Filtration for Logit-based Classifiers (2022)
• Active Forgetting via Influence Estimation for Neural Networks (2021)
• Adaptive Machine Unlearning (2021)
• Machine Unlearning via Algorithmic Stability (2021)
• Towards Probabilistic Verification of Machine Unlearning (2020)
• Remember What You Want to Forget: Algorithms for Machine Unlearning (2020)

Metrics

• Layer-wise Distance — Fast Yet Effective Machine Unlearning (2023)
• Reconstruction Error — Unfolded Self-Reconstruction LSH: Towards Machine Unlearning in Approximate Nearest Neighbour Search (2023)
• Unlearning Error — Unrolling SGD: Understanding Factors Influencing Machine Unlearning (2022)
• Forgetting Rate — Learn to Forget: Machine Unlearning Via Neuron Masking (2021)
• Accuracy, Activation Distance — Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks (2020)
• Anamnesis Index — Continual lifelong learning with neural networks: A review (2019)

Applications

What are the fields and tasks that machine unlearning is successfully or commonly used for? How does machine unlearning uniquely contribute to these applications? What potential future applications are being explored, and what can we expect?

Here are just a few examples of where machine unlearning has crucial applications:

• Data privacy (the right to be forgotten): in a rapidly changing digital world, the laws and regulations governing digital practices are also constantly updating. Machine unlearning certain data helps models stay compliant with regulations such as GDPR¹.
• Data updates (lifelong learning): data for a model can become outdated after corrections or updates are made for a variety of reasons. Through unlearning and forgetting these counterproductive datapoints, models can adapt to these changes more easily. This is especially useful in dynamic learning models such as lifelong learning.
• Bias removal: when models are trained on real-world data, they will reflect and possibly perpetuate any bias in the dataset, such as in hiring or patient data analysis. When these biases are identified, machine unlearning helps mitigate these impacts by removing features contributing to algorithmic biases.
• Towards Unbounded Machine Unlearning (2023)
• Recommendation Unlearning (2022)

Conclusion

This article is created with a conscious attempt to select valuable and accessible resources structured around the fundamental topics of machine unlearning, including introductory materials, challenges, algorithms and evaluation techniques, and practical implications but also prevent information overload. Thus, it does not include every single contribution in this vast and rapidly evolving field. If any pivotal studies or methodologies are not explicitly addressed, it is not an indication of their lack of importance or value to the field.

[1] European Union Agency for Fundamental Rights. (2019). The General Data Protection Regulation: One year on : civil society: awareness, opportunities and challenges. Publications Office. https://data.europa.eu/doi/10.2811/538633

[2] Shaik, T., Tao, X., Xie, H., Li, L., Zhu, X., & Li, Q. (2023). Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomy. https://doi.org/10.48550/ARXIV.2305.06360