Interpretable ML

Machine learning is increasingly used to inform consequential decision-making (e.g., pre-trial bail and loan approval). In these settings, explainability play an important role in the adoption and impact of these technologies. In particular, when algorithmic decisions affect individuals, it becomes important to explain how the system arrived at its decision, and also suggest actions to achieve a favorable decision. Counterfactual explanations –“how the world would have (had) to be different for a desirable outcome to occur”– aim to satisfy these criteria. Our research contributions to this field are twofold.

First, we focus on providing flexible approaches to generate (nearest) counterfactual explanations, i.e.,to identify the set of features resulting in the desired prediction while remaining at minimum distance from the original set of features describing the individual. In “Model-Agnostic Counterfactual Explanations for Consequential Decisions” we abandon the standard optimization approach to find nearest counterfactual explanations, as it these methods are often restricted to a particular subset of models (e.g., decision trees or linear models) and differentiable distance functions. Our Model-Agnostic Counterfactual Explanations (MACE) approach instead builds on standard theory and tools from formal verification to generate optimal, plausible and diverse counterfactual explanations for a wide variety of machine learning classifiers. Unfortunately, formal verification tools struggle to generate counterfactual explanations when the underlying classifiers is a complex neural network. To solve such a limitation, we then provided a framework based on Mixed-Integer Programming (MIP) to compute nearest counterfactual explanations for the outcomes of neural networks, with both provable guarantees and runtimes comparable to gradient-based approaches, see “Scaling Guarantees for Nearest Counterfactual Explanations”.

Second, we focus on supporting individuals affected by an unfavourable decision to revert it, or in other words, on algorithmic recourse. Algorithmic recourse and counterfactual explanations are closely related terms, that the literature has often used in an exchangeable way. Unfortunately, as shown in our work “Algorithmic Recourse: from Counterfactual Explanations to Interventions”, counterfactual explanations inform an individual where they need to get to (i.e., they help individuals to understand), but not how to get there (i.e., how to act to revert an unfavourable decision). To ensure algorithmic recourse one needs instead find the set of causal interventions with minimal effort for the individual that would trigger the desired outcome of the machine learning system. Such a guarantees, however, only hold under perfect causal knowledge (i.e., perfect knowledge of both the causal graph and structural equations). Unfortunately, in practice, the true underlying structural causal model is generally unknown. For this reason, our recent work focuses on developing probabilistic causal methods that allow for causal reasoning under limited causal assumptions~\cite{AAAI’22}, and ultimately identify the optimal actions for recourse, see “Algorithmic recourse under imperfect causal knowledge: a probabilistic approach”.

For more details on this topic, please check our Survey paper on algorithmic recourse! Find it referenced below.

Our ongoing research in this area focuses on providing practical approaches for both counterfactual explanations and algorithmic recourse for a wide range of machine learning algorithms. Moreover, we are also interested in research on making black-blox machine learning models, like deep neural networks, inherently interpretable. More updates coming soon!

References

2021

Mohammadi, Kiarash; Karimi, Amir-Hossein; Barthe, Gilles; Valera, Isabel

Scaling Guarantees for Nearest Counterfactual Explanations Proceedings Article

In: Fourcade, Marion; Kuipers, Benjamin; Lazar, Seth; Mulligan, Deirdre K. (Ed.): AIES '21: AAAI/ACM Conference on AI, Ethics, and Society, Virtual Event, USA, May 19-21, 2021, pp. 177–187, ACM, 2021.

Links | BibTeX

Karimi, Amir-Hossein; Schölkopf, Bernhard; Valera, Isabel

Algorithmic Recourse: from Counterfactual Explanations to Interventions Proceedings Article

In: Elish, Madeleine Clare; Isaac, William; Zemel, Richard S. (Ed.): FAccT '21: 2021 ACM Conference on Fairness, Accountability, and Transparency, Virtual Event / Toronto, Canada, March 3-10, 2021, pp. 353–362, ACM, 2021.

Links | BibTeX

2020

Karimi, Amir-Hossein; Barthe, Gilles; Schölkopf, Bernhard; Valera, Isabel

A survey of algorithmic recourse: definitions, formulations, solutions, and prospects Journal Article

In: CoRR, vol. abs/2010.04050, 2020.

Links | BibTeX

Karimi, Amir-Hossein; Barthe, Gilles; Balle, Borja; Valera, Isabel

Model-Agnostic Counterfactual Explanations for Consequential Decisions Proceedings Article

In: Chiappa, Silvia; Calandra, Roberto (Ed.): The 23rd International Conference on Artificial Intelligence and Statistics, AISTATS 2020, 26-28 August 2020, Online [Palermo, Sicily, Italy], pp. 895–905, PMLR, 2020.

Links | BibTeX

Karimi, Amir-Hossein; Kügelgen, Bodo Julius; Schölkopf, Bernhard; Valera, Isabel

Algorithmic recourse under imperfect causal knowledge: a probabilistic approach Proceedings Article

In: Larochelle, Hugo; Ranzato, Marc'Aurelio; Hadsell, Raia; Balcan, Maria-Florina; Lin, Hsuan-Tien (Ed.): Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.

Links | BibTeX