People
Isabel Valera
Saarland Informatics Campus
Building E1 1, R. 225
For administrative services, contact ml-office@lists.saarland-informatics-campus.de
To apply for PhD/PostDoc/HiWi/Thesis, see the information on the “Positions” page for the correct e-mail to use.
Otherwise, contact ivalera@cs.uni-saarland.de.
About me
I am a full Professor on Machine Learning at the Department of Computer Science of Saarland University (Saarbrücken, Germany), and Adjunct Faculty at MPI for Software Systems in Saarbrücken (Saarbrücken, Germany).
I am a fellow of the European Laboratory for Learning and Intelligent Systems (ELLIS), where I am part of the Robust Machine Learning Program and of the Saarbrücken Artificial Intelligence & Machine learning (Sam) Unit.
Prior to this, I was an independent group leader at the MPI for Intelligent Systems in Tübingen (Germany) until the end of the year. I have held a German Humboldt Post-Doctoral Fellowship, and a “Minerva fast track” fellowship from the Max Planck Society. I obtained my PhD in 2014 and MSc degree in 2012 from the University Carlos III in Madrid (Spain), and worked as postdoctoral researcher at the MPI for Software Systems (Germany) and at the University of Cambridge (UK).
Publications
2020
Karimi, Amir-Hossein; Barthe, Gilles; Balle, Borja; Valera, Isabel
Model-Agnostic Counterfactual Explanations for Consequential Decisions Proceedings Article
In: Chiappa, Silvia; Calandra, Roberto (Ed.): The 23rd International Conference on Artificial Intelligence and Statistics, AISTATS 2020, 26-28 August 2020, Online [Palermo, Sicily, Italy], pp. 895–905, PMLR, 2020.
Abstract | Links | BibTeX | Tags: amir, isabel, project-interpretableML
@inproceedings{DBLP:conf/aistats/KarimiBBV20,
title = {Model-Agnostic Counterfactual Explanations for Consequential Decisions},
author = {Amir-Hossein Karimi and Gilles Barthe and Borja Balle and Isabel Valera},
editor = {Silvia Chiappa and Roberto Calandra},
url = {http://proceedings.mlr.press/v108/karimi20a.html},
year = {2020},
date = {2020-01-01},
urldate = {2020-01-01},
booktitle = {The 23rd International Conference on Artificial Intelligence and Statistics,
AISTATS 2020, 26-28 August 2020, Online [Palermo, Sicily, Italy]},
volume = {108},
pages = {895--905},
publisher = {PMLR},
series = {Proceedings of Machine Learning Research},
abstract = {Predictive models are being increasingly used to support consequential decision making at the individual level in contexts such as pretrial bail and loan approval. As a result, there is increasing social and legal pressure to provide explanations that help the affected individuals not only to understand why a prediction was output, but also how to act to obtain a desired outcome. To this end, several works have proposed optimization-based methods to generate nearest counterfactual explanations. However, these methods are often restricted to a particular subset of models (e.g., decision trees or linear models) and differentiable distance functions. In contrast, we build on standard theory and tools from formal verification and propose a novel algorithm that solves a sequence of satisfiability problems, where both the distance function (objective) and predictive model (constraints) are represented as logic formulae. As shown by our experiments on real-world data, our algorithm is: i) model-agnostic ({non-}linear, {non-}differentiable, {non-}convex); ii) data-type-agnostic (heterogeneous features); iii) distance-agnostic (l0, l1, l8, and combinations thereof); iv) able to generate plausible and diverse counterfactuals for any sample (i.e., 100% coverage); and v) at provably optimal distances.},
keywords = {amir, isabel, project-interpretableML},
pubstate = {published},
tppubtype = {inproceedings}
}
Predictive models are being increasingly used to support consequential decision making at the individual level in contexts such as pretrial bail and loan approval. As a result, there is increasing social and legal pressure to provide explanations that help the affected individuals not only to understand why a prediction was output, but also how to act to obtain a desired outcome. To this end, several works have proposed optimization-based methods to generate nearest counterfactual explanations. However, these methods are often restricted to a particular subset of models (e.g., decision trees or linear models) and differentiable distance functions. In contrast, we build on standard theory and tools from formal verification and propose a novel algorithm that solves a sequence of satisfiability problems, where both the distance function (objective) and predictive model (constraints) are represented as logic formulae. As shown by our experiments on real-world data, our algorithm is: i) model-agnostic ({non-}linear, {non-}differentiable, {non-}convex); ii) data-type-agnostic (heterogeneous features); iii) distance-agnostic (l0, l1, l8, and combinations thereof); iv) able to generate plausible and diverse counterfactuals for any sample (i.e., 100% coverage); and v) at provably optimal distances.
Karimi, Amir-Hossein; Kügelgen, Bodo Julius; Schölkopf, Bernhard; Valera, Isabel
Algorithmic recourse under imperfect causal knowledge: a probabilistic approach Proceedings Article
In: Larochelle, Hugo; Ranzato, Marc'Aurelio; Hadsell, Raia; Balcan, Maria-Florina; Lin, Hsuan-Tien (Ed.): Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.
Abstract | Links | BibTeX | Tags: amir, isabel, project-interpretableML
@inproceedings{DBLP:conf/nips/KarimiKSV20,
title = {Algorithmic recourse under imperfect causal knowledge: a probabilistic approach},
author = {Amir-Hossein Karimi and Bodo Julius Kügelgen and Bernhard Schölkopf and Isabel Valera},
editor = {Hugo Larochelle and Marc'Aurelio Ranzato and Raia Hadsell and Maria-Florina Balcan and Hsuan-Tien Lin},
url = {https://proceedings.neurips.cc/paper/2020/hash/02a3c7fb3f489288ae6942498498db20-Abstract.html},
year = {2020},
date = {2020-01-01},
urldate = {2020-01-01},
booktitle = {Advances in Neural Information Processing Systems 33: Annual Conference
on Neural Information Processing Systems 2020, NeurIPS 2020, December
6-12, 2020, virtual},
abstract = {Recent work has discussed the limitations of counterfactual explanations to recommend actions for algorithmic recourse, and argued for the need of taking causal relationships between features into consideration. Unfortunately, in practice, the true underlying structural causal model is generally unknown. In this work, we first show that it is impossible to guarantee recourse without access to the true structural equations. To address this limitation, we propose two probabilistic approaches to select optimal actions that achieve recourse with high probability given limited causal knowledge (e.g., only the causal graph). The first captures uncertainty over structural equations under additive Gaussian noise, and uses Bayesian model averaging to estimate the counterfactual distribution. The second removes any assumptions on the structural equations by instead computing the average effect of recourse actions on individuals similar to the person who seeks recourse, leading to a novel subpopulation-based interventional notion of recourse. We then derive a gradient-based procedure for selecting optimal recourse actions, and empirically show that the proposed approaches lead to more reliable recommendations under imperfect causal knowledge than non-probabilistic baselines.},
keywords = {amir, isabel, project-interpretableML},
pubstate = {published},
tppubtype = {inproceedings}
}
Recent work has discussed the limitations of counterfactual explanations to recommend actions for algorithmic recourse, and argued for the need of taking causal relationships between features into consideration. Unfortunately, in practice, the true underlying structural causal model is generally unknown. In this work, we first show that it is impossible to guarantee recourse without access to the true structural equations. To address this limitation, we propose two probabilistic approaches to select optimal actions that achieve recourse with high probability given limited causal knowledge (e.g., only the causal graph). The first captures uncertainty over structural equations under additive Gaussian noise, and uses Bayesian model averaging to estimate the counterfactual distribution. The second removes any assumptions on the structural equations by instead computing the average effect of recourse actions on individuals similar to the person who seeks recourse, leading to a novel subpopulation-based interventional notion of recourse. We then derive a gradient-based procedure for selecting optimal recourse actions, and empirically show that the proposed approaches lead to more reliable recommendations under imperfect causal knowledge than non-probabilistic baselines.
Karimi, Amir-Hossein; Barthe, Gilles; Schölkopf, Bernhard; Valera, Isabel
A survey of algorithmic recourse: definitions, formulations, solutions, and prospects Journal Article
In: CoRR, vol. abs/2010.04050, 2020.
Abstract | Links | BibTeX | Tags: amir, isabel, project-interpretableML
@article{DBLP:journals/corr/abs-2010-04050,
title = {A survey of algorithmic recourse: definitions, formulations, solutions, and prospects},
author = {Amir-Hossein Karimi and Gilles Barthe and Bernhard Schölkopf and Isabel Valera},
url = {https://arxiv.org/abs/2010.04050},
year = {2020},
date = {2020-01-01},
urldate = {2020-01-01},
journal = {CoRR},
volume = {abs/2010.04050},
abstract = {Machine learning is increasingly used to inform decision-making in sensitive situations where decisions have consequential effects on individuals' lives. In these settings, in addition to requiring models to be accurate and robust, socially relevant values such as fairness, privacy, accountability, and explainability play an important role for the adoption and impact of said technologies. In this work, we focus on algorithmic recourse, which is concerned with providing explanations and recommendations to individuals who are unfavourably treated by automated decision-making systems. We first perform an extensive literature review, and align the efforts of many authors by presenting unified definitions, formulations, and solutions to recourse. Then, we provide an overview of the prospective research directions towards which the community may engage, challenging existing assumptions and making explicit connections to other ethical challenges such as security, privacy, and fairness.},
keywords = {amir, isabel, project-interpretableML},
pubstate = {published},
tppubtype = {article}
}
Machine learning is increasingly used to inform decision-making in sensitive situations where decisions have consequential effects on individuals' lives. In these settings, in addition to requiring models to be accurate and robust, socially relevant values such as fairness, privacy, accountability, and explainability play an important role for the adoption and impact of said technologies. In this work, we focus on algorithmic recourse, which is concerned with providing explanations and recommendations to individuals who are unfavourably treated by automated decision-making systems. We first perform an extensive literature review, and align the efforts of many authors by presenting unified definitions, formulations, and solutions to recourse. Then, we provide an overview of the prospective research directions towards which the community may engage, challenging existing assumptions and making explicit connections to other ethical challenges such as security, privacy, and fairness.
Valera, Isabel; Pradier, Melanie F.; Lomeli, Maria; Ghahramani, Zoubin
General Latent Feature Models for Heterogeneous Datasets Journal Article
In: J. Mach. Learn. Res., vol. 21, pp. 100:1–100:49, 2020.
Abstract | Links | BibTeX | Tags: isabel, project-robustgenerative
@article{DBLP:journals/jmlr/ValeraPLG20,
title = {General Latent Feature Models for Heterogeneous Datasets},
author = {Isabel Valera and Melanie F. Pradier and Maria Lomeli and Zoubin Ghahramani},
url = {http://jmlr.org/papers/v21/17-328.html},
year = {2020},
date = {2020-01-01},
urldate = {2020-01-01},
journal = {J. Mach. Learn. Res.},
volume = {21},
pages = {100:1--100:49},
abstract = {Latent variable models allow capturing the hidden structure underlying the data. In particular, feature allocation models represent each observation by a linear combination of latent variables. These models are often used to make predictions either for new observations or for missing information in the original data, as well as to perform exploratory data analysis. Although there is an extensive literature on latent feature allocation models for homogeneous datasets, where all the attributes that describe each object are of the same (continuous or discrete) type, there is no general framework for practical latent feature modeling for heterogeneous datasets. In this paper, we introduce a general Bayesian nonparametric latent feature allocation model suitable for heterogeneous datasets, where the attributes describing each object can be arbitrary combinations of real-valued, positive real-valued, categorical, ordinal and count variables. The proposed model presents several important properties. First, it is suitable for heterogeneous data while keeping the properties of conjugate models, which enables us to develop an inference algorithm that presents linear complexity with respect to the number of objects and attributes per MCMC iteration. Second, the Bayesian nonparametric component allows us to place a prior distribution on the number of features required to capture the latent structure in the data. Third, the latent features in the model are binary-valued, which facilitates the interpretability of the obtained latent features in exploratory data analysis. Finally, a software package, called GLFM toolbox, is made publicly available for other researchers to use and extend. It is available at https://ivaleram.github.io/GLFM/. We show the flexibility of the proposed model by solving both prediction and data analysis tasks on several real-world datasets.},
keywords = {isabel, project-robustgenerative},
pubstate = {published},
tppubtype = {article}
}
Latent variable models allow capturing the hidden structure underlying the data. In particular, feature allocation models represent each observation by a linear combination of latent variables. These models are often used to make predictions either for new observations or for missing information in the original data, as well as to perform exploratory data analysis. Although there is an extensive literature on latent feature allocation models for homogeneous datasets, where all the attributes that describe each object are of the same (continuous or discrete) type, there is no general framework for practical latent feature modeling for heterogeneous datasets. In this paper, we introduce a general Bayesian nonparametric latent feature allocation model suitable for heterogeneous datasets, where the attributes describing each object can be arbitrary combinations of real-valued, positive real-valued, categorical, ordinal and count variables. The proposed model presents several important properties. First, it is suitable for heterogeneous data while keeping the properties of conjugate models, which enables us to develop an inference algorithm that presents linear complexity with respect to the number of objects and attributes per MCMC iteration. Second, the Bayesian nonparametric component allows us to place a prior distribution on the number of features required to capture the latent structure in the data. Third, the latent features in the model are binary-valued, which facilitates the interpretability of the obtained latent features in exploratory data analysis. Finally, a software package, called GLFM toolbox, is made publicly available for other researchers to use and extend. It is available at https://ivaleram.github.io/GLFM/. We show the flexibility of the proposed model by solving both prediction and data analysis tasks on several real-world datasets.
Kilbertus, Niki; Rodriguez, Manuel Gomez; Schölkopf, Bernhard; Muandet, Krikamol; Valera, Isabel
Fair Decisions Despite Imperfect Predictions Proceedings Article
In: Chiappa, Silvia; Calandra, Roberto (Ed.): The 23rd International Conference on Artificial Intelligence and Statistics, AISTATS 2020, 26-28 August 2020, Online [Palermo, Sicily, Italy], pp. 277–287, PMLR, 2020.
Abstract | Links | BibTeX | Tags: isabel, project-fairml
@inproceedings{DBLP:conf/aistats/KilbertusRSMV20,
title = {Fair Decisions Despite Imperfect Predictions},
author = {Niki Kilbertus and Manuel Gomez Rodriguez and Bernhard Schölkopf and Krikamol Muandet and Isabel Valera},
editor = {Silvia Chiappa and Roberto Calandra},
url = {http://proceedings.mlr.press/v108/kilbertus20a.html},
year = {2020},
date = {2020-01-01},
urldate = {2020-01-01},
booktitle = {The 23rd International Conference on Artificial Intelligence and Statistics,
AISTATS 2020, 26-28 August 2020, Online [Palermo, Sicily, Italy]},
volume = {108},
pages = {277--287},
publisher = {PMLR},
series = {Proceedings of Machine Learning Research},
abstract = {Consequential decisions are increasingly informed by sophisticated data-driven predictive models. However, consistently learning accurate predictive models requires access to ground truth labels. Unfortunately, in practice, labels may only exist conditional on certain decisions—if a loan is denied, there is not even an option for the individual to pay back the loan. In this paper, we show that, in this selective labels setting, learning to predict is suboptimal in terms of both fairness and utility. To avoid this undesirable behavior, we propose to directly learn stochastic decision policies that maximize utility under fairness constraints. In the context of fair machine learning, our results suggest the need for a paradigm shift from "learning to predict" to "learning to decide". Experiments on synthetic and real-world data illustrate the favorable properties of learning to decide, in terms of both utility and fairness.},
keywords = {isabel, project-fairml},
pubstate = {published},
tppubtype = {inproceedings}
}
Consequential decisions are increasingly informed by sophisticated data-driven predictive models. However, consistently learning accurate predictive models requires access to ground truth labels. Unfortunately, in practice, labels may only exist conditional on certain decisions—if a loan is denied, there is not even an option for the individual to pay back the loan. In this paper, we show that, in this selective labels setting, learning to predict is suboptimal in terms of both fairness and utility. To avoid this undesirable behavior, we propose to directly learn stochastic decision policies that maximize utility under fairness constraints. In the context of fair machine learning, our results suggest the need for a paradigm shift from "learning to predict" to "learning to decide". Experiments on synthetic and real-world data illustrate the favorable properties of learning to decide, in terms of both utility and fairness.
Karimi, Amir-Hossein; Kügelgen, Julius; Schölkopf, Bernhard; Valera, Isabel
Towards Causal Algorithmic Recourse Proceedings Article
In: Holzinger, Andreas; Goebel, Randy; Fong, Ruth; Moon, Taesup; Müller, Klaus-Robert; Samek, Wojciech (Ed.): xxAI - Beyond Explainable AI - International Workshop, Held in Conjunction with ICML 2020, July 18, 2020, Vienna, Austria, Revised and Extended Papers, pp. 139–166, Springer, 2020.
Abstract | Links | BibTeX | Tags: amir, isabel
@inproceedings{DBLP:conf/icml/KarimiKSV20,
title = {Towards Causal Algorithmic Recourse},
author = {Amir-Hossein Karimi and Julius Kügelgen and Bernhard Schölkopf and Isabel Valera},
editor = {Andreas Holzinger and Randy Goebel and Ruth Fong and Taesup Moon and Klaus-Robert Müller and Wojciech Samek},
url = {https://doi.org/10.1007/978-3-031-04083-2_8},
doi = {10.1007/978-3-031-04083-2_8},
year = {2020},
date = {2020-01-01},
urldate = {2020-01-01},
booktitle = {xxAI - Beyond Explainable AI - International Workshop, Held in Conjunction
with ICML 2020, July 18, 2020, Vienna, Austria, Revised and Extended
Papers},
volume = {13200},
pages = {139–166},
publisher = {Springer},
series = {Lecture Notes in Computer Science},
abstract = {Algorithmic recourse is concerned with aiding individuals who are unfavorably treated by automated decision-making systems to overcome their hardship, by offering recommendations that would result in a more favorable prediction when acted upon. Such recourse actions are typically obtained through solving an optimization problem that minimizes changes to the individual’s feature vector, subject to various plausibility, diversity, and sparsity constraints. Whereas previous works offer solutions to the optimization problem in a variety of settings, they critically overlook real-world considerations pertaining to the environment in which recourse actions are performed.
The present work emphasizes that changes to a subset of the individual’s attributes may have consequential down-stream effects on other attributes, thus making recourse a fundamcausal problem. Here, we model such considerations using the framework of structural causal models, and highlight pitfalls of not considering causal relations through examples and theory. Such insights allow us to reformulate the optimization problem to directly optimize for minimally-costly recourse over a space of feasible actions (in the form of causal interventions) rather than optimizing for minimally-distant “counterfactual explanations”. We offer both the optimization formulations and solutions to deterministic and probabilistic recourse, on an individualized and sub-population level, overcoming the steep assumptive requirements of offering recourse in general settings. Finally, using synthetic and semi-synthetic experiments based on the German Credit dataset, we demonstrate how such methods can be applied in practice under minimal causal assumptions.},
keywords = {amir, isabel},
pubstate = {published},
tppubtype = {inproceedings}
}
Algorithmic recourse is concerned with aiding individuals who are unfavorably treated by automated decision-making systems to overcome their hardship, by offering recommendations that would result in a more favorable prediction when acted upon. Such recourse actions are typically obtained through solving an optimization problem that minimizes changes to the individual’s feature vector, subject to various plausibility, diversity, and sparsity constraints. Whereas previous works offer solutions to the optimization problem in a variety of settings, they critically overlook real-world considerations pertaining to the environment in which recourse actions are performed.
The present work emphasizes that changes to a subset of the individual’s attributes may have consequential down-stream effects on other attributes, thus making recourse a fundamcausal problem. Here, we model such considerations using the framework of structural causal models, and highlight pitfalls of not considering causal relations through examples and theory. Such insights allow us to reformulate the optimization problem to directly optimize for minimally-costly recourse over a space of feasible actions (in the form of causal interventions) rather than optimizing for minimally-distant “counterfactual explanations”. We offer both the optimization formulations and solutions to deterministic and probabilistic recourse, on an individualized and sub-population level, overcoming the steep assumptive requirements of offering recourse in general settings. Finally, using synthetic and semi-synthetic experiments based on the German Credit dataset, we demonstrate how such methods can be applied in practice under minimal causal assumptions.
The present work emphasizes that changes to a subset of the individual’s attributes may have consequential down-stream effects on other attributes, thus making recourse a fundamcausal problem. Here, we model such considerations using the framework of structural causal models, and highlight pitfalls of not considering causal relations through examples and theory. Such insights allow us to reformulate the optimization problem to directly optimize for minimally-costly recourse over a space of feasible actions (in the form of causal interventions) rather than optimizing for minimally-distant “counterfactual explanations”. We offer both the optimization formulations and solutions to deterministic and probabilistic recourse, on an individualized and sub-population level, overcoming the steep assumptive requirements of offering recourse in general settings. Finally, using synthetic and semi-synthetic experiments based on the German Credit dataset, we demonstrate how such methods can be applied in practice under minimal causal assumptions.
2019
Zafar, Muhammad Bilal; Valera, Isabel; Gomez-Rodriguez, Manuel; Gummadi, Krishna P.
Fairness Constraints: A Flexible Approach for Fair Classification Journal Article
In: J. Mach. Learn. Res., vol. 20, pp. 75:1–75:42, 2019.
Abstract | Links | BibTeX | Tags: isabel, project-fairml
@article{DBLP:journals/jmlr/ZafarVGG19,
title = {Fairness Constraints: A Flexible Approach for Fair Classification},
author = {Muhammad Bilal Zafar and Isabel Valera and Manuel Gomez-Rodriguez and Krishna P. Gummadi},
url = {http://jmlr.org/papers/v20/18-262.html},
year = {2019},
date = {2019-01-01},
urldate = {2019-01-01},
journal = {J. Mach. Learn. Res.},
volume = {20},
pages = {75:1--75:42},
abstract = {Algorithmic decision making is employed in an increasing number of real-world applicationstions to aid human decision making. While it has shown considerable promise in terms of improved decision accuracy, in some scenarios, its outcomes have been also shown to impose an unfair (dis)advantage on people from certain social groups (e.g., women, blacks). In this context, there is a need for computational techniques to limit unfairness in algorithmic decision making. In this work, we take a step forward to fulfill that need and introduce a flexible constraint-based framework to enable the design of fair margin-based classifiers. The main technical innovation of our framework is a general and intuitive measure of decision boundary unfairness, which serves as a tractable proxy to several of the most popular computational definitions of unfairness from the literature. Leveraging our measure, we can reduce the design of fair margin-based classifiers to adding tractable constraints on their decision boundaries. Experiments on multiple synthetic and real-world datasets show that our framework is able to successfully limit unfairness, often at a small cost in terms of accuracy.},
keywords = {isabel, project-fairml},
pubstate = {published},
tppubtype = {article}
}
Algorithmic decision making is employed in an increasing number of real-world applicationstions to aid human decision making. While it has shown considerable promise in terms of improved decision accuracy, in some scenarios, its outcomes have been also shown to impose an unfair (dis)advantage on people from certain social groups (e.g., women, blacks). In this context, there is a need for computational techniques to limit unfairness in algorithmic decision making. In this work, we take a step forward to fulfill that need and introduce a flexible constraint-based framework to enable the design of fair margin-based classifiers. The main technical innovation of our framework is a general and intuitive measure of decision boundary unfairness, which serves as a tractable proxy to several of the most popular computational definitions of unfairness from the literature. Leveraging our measure, we can reduce the design of fair margin-based classifiers to adding tractable constraints on their decision boundaries. Experiments on multiple synthetic and real-world datasets show that our framework is able to successfully limit unfairness, often at a small cost in terms of accuracy.
Adel, Tameem; Valera, Isabel; Ghahramani, Zoubin; Weller, Adrian
One-Network Adversarial Fairness Proceedings Article
In: The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019, pp. 2412–2420, AAAI Press, 2019.
Abstract | Links | BibTeX | Tags: isabel, project-fairml
@inproceedings{DBLP:conf/aaai/AdelVGW19,
title = {One-Network Adversarial Fairness},
author = {Tameem Adel and Isabel Valera and Zoubin Ghahramani and Adrian Weller},
url = {https://doi.org/10.1609/aaai.v33i01.33012412},
doi = {10.1609/aaai.v33i01.33012412},
year = {2019},
date = {2019-01-01},
urldate = {2019-01-01},
booktitle = {The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI
2019, The Thirty-First Innovative Applications of Artificial Intelligence
Conference, IAAI 2019, The Ninth AAAI Symposium on Educational
Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii,
USA, January 27 - February 1, 2019},
pages = {2412--2420},
publisher = {AAAI Press},
abstract = {There is currently a great expansion of the impact of machine learning algorithms on our lives, prompting the need for objectives other than pure performance, including fairness. Fairness here means that the outcome of an automated decisionmaking system should not discriminate between subgroups characterized by sensitive attributes such as gender or race. Given any existing differentiable classifier, we make only slight adjustments to the architecture including adding a new hidden layer, in order to enable the concurrent adversarial optimization for fairness and accuracy. Our framework provides one way to quantify the tradeoff between fairness and accuracy, while also leading to strong empirical performance.},
keywords = {isabel, project-fairml},
pubstate = {published},
tppubtype = {inproceedings}
}
There is currently a great expansion of the impact of machine learning algorithms on our lives, prompting the need for objectives other than pure performance, including fairness. Fairness here means that the outcome of an automated decisionmaking system should not discriminate between subgroups characterized by sensitive attributes such as gender or race. Given any existing differentiable classifier, we make only slight adjustments to the architecture including adding a new hidden layer, in order to enable the concurrent adversarial optimization for fairness and accuracy. Our framework provides one way to quantify the tradeoff between fairness and accuracy, while also leading to strong empirical performance.
Vergari, Antonio; Molina, Alejandro; Peharz, Robert; Ghahramani, Zoubin; Kersting, Kristian; Valera, Isabel
Automatic Bayesian Density Analysis Proceedings Article
In: The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019, pp. 5207–5215, AAAI Press, 2019.
Abstract | Links | BibTeX | Tags: isabel, project-robustgenerative
@inproceedings{DBLP:conf/aaai/Vergari0PGKV19,
title = {Automatic Bayesian Density Analysis},
author = {Antonio Vergari and Alejandro Molina and Robert Peharz and Zoubin Ghahramani and Kristian Kersting and Isabel Valera},
url = {https://doi.org/10.1609/aaai.v33i01.33015207},
doi = {10.1609/aaai.v33i01.33015207},
year = {2019},
date = {2019-01-01},
urldate = {2019-01-01},
booktitle = {The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI
2019, The Thirty-First Innovative Applications of Artificial Intelligence
Conference, IAAI 2019, The Ninth AAAI Symposium on Educational
Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii,
USA, January 27 - February 1, 2019},
pages = {5207--5215},
publisher = {AAAI Press},
abstract = {Making sense of a dataset in an automatic and unsupervised fashion is a challenging problem in statistics and AI. Classical approaches for exploratory data analysis are usually not flexible enough to deal with the uncertainty inherent to real-world data: they are often restricted to fixed latent interaction models and homogeneous likelihoods; they are sensitive to missing, corrupt and anomalous data; moreover, their expressiveness generally comes at the price of intractable inference. As a result, supervision from statisticians is usually needed to find the right model for the data. However, since domain experts are not necessarily also experts in statistics, we propose Automatic Bayesian Density Analysis (ABDA) to make exploratory data analysis accessible at large. Specifically, ABDA allows for automatic and efficient missing value estimation, statistical data type and likelihood discovery, anomaly detection and dependency structure mining, on top of providing accurate density estimation. Extensive empirical evidence shows that ABDA is a suitable tool for automatic exploratory analysis of mixed continuous and discrete tabular data.},
keywords = {isabel, project-robustgenerative},
pubstate = {published},
tppubtype = {inproceedings}
}
Making sense of a dataset in an automatic and unsupervised fashion is a challenging problem in statistics and AI. Classical approaches for exploratory data analysis are usually not flexible enough to deal with the uncertainty inherent to real-world data: they are often restricted to fixed latent interaction models and homogeneous likelihoods; they are sensitive to missing, corrupt and anomalous data; moreover, their expressiveness generally comes at the price of intractable inference. As a result, supervision from statisticians is usually needed to find the right model for the data. However, since domain experts are not necessarily also experts in statistics, we propose Automatic Bayesian Density Analysis (ABDA) to make exploratory data analysis accessible at large. Specifically, ABDA allows for automatic and efficient missing value estimation, statistical data type and likelihood discovery, anomaly detection and dependency structure mining, on top of providing accurate density estimation. Extensive empirical evidence shows that ABDA is a suitable tool for automatic exploratory analysis of mixed continuous and discrete tabular data.
Kilbertus, Niki; Gomez-Rodriguez, Manuel; Schölkopf, Bernhard; Muandet, Krikamol; Valera, Isabel
Improving Consequential Decision Making under Imperfect Predictions Journal Article
In: CoRR, vol. abs/1902.02979, 2019.
Abstract | Links | BibTeX | Tags: isabel
@article{DBLP:journals/corr/abs-1902-02979,
title = {Improving Consequential Decision Making under Imperfect Predictions},
author = {Niki Kilbertus and Manuel Gomez-Rodriguez and Bernhard Schölkopf and Krikamol Muandet and Isabel Valera},
url = {http://arxiv.org/abs/1902.02979},
year = {2019},
date = {2019-01-01},
urldate = {2019-01-01},
journal = {CoRR},
volume = {abs/1902.02979},
abstract = {Consequential decisions are increasingly informed by sophisticated data-driven predictive models. However, to consistently learn accurate predictive models, one needs access to ground truth labels. Unfortunately, in practice, labels may only exist conditional on certain decisions---if a loan is denied, there is not even an option for the individual to pay back the loan. Hence, the observed data distribution depends on how decisions are being made. In this paper, we show that in this selective labels setting, learning a predictor directly only from available labeled data is suboptimal in terms of both fairness and utility. To avoid this undesirable behavior, we propose to directly learn decision policies that maximize utility under fairness constraints and thereby take into account how decisions affect which data is observed in the future. Our results suggest the need for a paradigm shift in the context of fair machine learning from the currently prevalent idea of simply building predictive models from a single static dataset via risk minimization, to a more interactive notion of "learning to decide". In particular, such policies should not entirely neglect part of the input space, drawing connections to explore/exploit tradeoffs in reinforcement learning, data missingness, and potential outcomes in causal inference. Experiments on synthetic and real-world data illustrate the favorable properties of learning to decide in terms of utility and fairness.},
keywords = {isabel},
pubstate = {published},
tppubtype = {article}
}
Consequential decisions are increasingly informed by sophisticated data-driven predictive models. However, to consistently learn accurate predictive models, one needs access to ground truth labels. Unfortunately, in practice, labels may only exist conditional on certain decisions---if a loan is denied, there is not even an option for the individual to pay back the loan. Hence, the observed data distribution depends on how decisions are being made. In this paper, we show that in this selective labels setting, learning a predictor directly only from available labeled data is suboptimal in terms of both fairness and utility. To avoid this undesirable behavior, we propose to directly learn decision policies that maximize utility under fairness constraints and thereby take into account how decisions affect which data is observed in the future. Our results suggest the need for a paradigm shift in the context of fair machine learning from the currently prevalent idea of simply building predictive models from a single static dataset via risk minimization, to a more interactive notion of "learning to decide". In particular, such policies should not entirely neglect part of the input space, drawing connections to explore/exploit tradeoffs in reinforcement learning, data missingness, and potential outcomes in causal inference. Experiments on synthetic and real-world data illustrate the favorable properties of learning to decide in terms of utility and fairness.
2018
Ruiz, Francisco J. R.; Valera, Isabel; Svensson, Lennart; Pérez-Cruz, Fernando
Infinite Factorial Finite State Machine for Blind Multiuser Channel Estimation Journal Article
In: IEEE Trans. Cogn. Commun. Netw., vol. 4, no. 2, pp. 177–191, 2018.
Abstract | Links | BibTeX | Tags: isabel
@article{DBLP:journals/tccn/RuizVSP18,
title = {Infinite Factorial Finite State Machine for Blind Multiuser Channel Estimation},
author = {Francisco J. R. Ruiz and Isabel Valera and Lennart Svensson and Fernando Pérez-Cruz},
url = {https://doi.org/10.1109/TCCN.2018.2790976},
doi = {10.1109/TCCN.2018.2790976},
year = {2018},
date = {2018-01-01},
urldate = {2018-01-01},
journal = {IEEE Trans. Cogn. Commun. Netw.},
volume = {4},
number = {2},
pages = {177--191},
abstract = {New communication standards need to deal with machine-to-machine communications, in which users may start or stop transmitting at any time in an asynchronous manner. Thus, the number of users is an unknown and time-varying parameter that needs to be accurately estimated in order to properly recover the symbols transmitted by all users in the system. In this paper, we address the problem of joint channel parameter and data estimation in a multiuser communication channel in which the number of transmitters is not known. For that purpose, we develop the infinite factorial finite state machine model, a Bayesian nonparametric model based on the Markov Indian buffet that allows for an unbounded number of transmitters with arbitrary channel length. We propose an inference algorithm that makes use of slice sampling and particle Gibbs with ancestor sampling. Our approach is fully blind as it does not require a prior channel estimation step, prior knowledge of the number of transmitters, or any signaling information. Our experimental results, loosely based on the LTE random access channel, show that the proposed approach can effectively recover the data-generating process for a wide range of scenarios, with varying number of transmitters, number of receivers, constellation order, channel length, and signal-to-noise ratio.},
keywords = {isabel},
pubstate = {published},
tppubtype = {article}
}
New communication standards need to deal with machine-to-machine communications, in which users may start or stop transmitting at any time in an asynchronous manner. Thus, the number of users is an unknown and time-varying parameter that needs to be accurately estimated in order to properly recover the symbols transmitted by all users in the system. In this paper, we address the problem of joint channel parameter and data estimation in a multiuser communication channel in which the number of transmitters is not known. For that purpose, we develop the infinite factorial finite state machine model, a Bayesian nonparametric model based on the Markov Indian buffet that allows for an unbounded number of transmitters with arbitrary channel length. We propose an inference algorithm that makes use of slice sampling and particle Gibbs with ancestor sampling. Our approach is fully blind as it does not require a prior channel estimation step, prior knowledge of the number of transmitters, or any signaling information. Our experimental results, loosely based on the LTE random access channel, show that the proposed approach can effectively recover the data-generating process for a wide range of scenarios, with varying number of transmitters, number of receivers, constellation order, channel length, and signal-to-noise ratio.
Valera, Isabel; Singla, Adish; Rodriguez, Manuel Gomez
Enhancing the Accuracy and Fairness of Human Decision Making Proceedings Article
In: Bengio, Samy; Wallach, Hanna M.; Larochelle, Hugo; Grauman, Kristen; Cesa-Bianchi, Nicolò; Garnett, Roman (Ed.): Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pp. 1774–1783, 2018.
Abstract | Links | BibTeX | Tags: isabel, project-fairml
@inproceedings{DBLP:conf/nips/ValeraSR18,
title = {Enhancing the Accuracy and Fairness of Human Decision Making},
author = {Isabel Valera and Adish Singla and Manuel Gomez Rodriguez},
editor = {Samy Bengio and Hanna M. Wallach and Hugo Larochelle and Kristen Grauman and Nicolò Cesa-Bianchi and Roman Garnett},
url = {https://proceedings.neurips.cc/paper/2018/hash/0a113ef6b61820daa5611c870ed8d5ee-Abstract.html},
year = {2018},
date = {2018-01-01},
urldate = {2018-01-01},
booktitle = {Advances in Neural Information Processing Systems 31: Annual Conference
on Neural Information Processing Systems 2018, NeurIPS 2018, December
3-8, 2018, Montréal, Canada},
pages = {1774--1783},
abstract = {Societies often rely on human experts to take a wide variety of decisions affecting their members, from jail-or-release decisions taken by judges and stop-and-frisk decisions taken by police officers to accept-or-reject decisions taken by academics. In this context, each decision is taken by an expert who is typically chosen uniformly at random from a pool of experts. However, these decisions may be imperfect due to limited experience, implicit biases, or faulty probabilistic reasoning. Can we improve the accuracy and fairness of the overall decision making process by optimizing the assignment between experts and decisions?
In this paper, we address the above problem from the perspective of sequential decision making and show that, for different fairness notions from the literature, it reduces to a sequence of (constrained) weighted bipartite matchings, which can be solved efficiently using algorithms with approximation guarantees. Moreover, these algorithms also benefit from posterior sampling to actively trade off exploitation---selecting expert assignments which lead to accurate and fair decisions---and exploration---selecting expert assignments to learn about the experts' preferences and biases. We demonstrate the effectiveness of our algorithms on both synthetic and real-world data and show that they can significantly improve both the accuracy and fairness of the decisions taken by pools of experts.},
keywords = {isabel, project-fairml},
pubstate = {published},
tppubtype = {inproceedings}
}
Societies often rely on human experts to take a wide variety of decisions affecting their members, from jail-or-release decisions taken by judges and stop-and-frisk decisions taken by police officers to accept-or-reject decisions taken by academics. In this context, each decision is taken by an expert who is typically chosen uniformly at random from a pool of experts. However, these decisions may be imperfect due to limited experience, implicit biases, or faulty probabilistic reasoning. Can we improve the accuracy and fairness of the overall decision making process by optimizing the assignment between experts and decisions?
In this paper, we address the above problem from the perspective of sequential decision making and show that, for different fairness notions from the literature, it reduces to a sequence of (constrained) weighted bipartite matchings, which can be solved efficiently using algorithms with approximation guarantees. Moreover, these algorithms also benefit from posterior sampling to actively trade off exploitation---selecting expert assignments which lead to accurate and fair decisions---and exploration---selecting expert assignments to learn about the experts' preferences and biases. We demonstrate the effectiveness of our algorithms on both synthetic and real-world data and show that they can significantly improve both the accuracy and fairness of the decisions taken by pools of experts.
In this paper, we address the above problem from the perspective of sequential decision making and show that, for different fairness notions from the literature, it reduces to a sequence of (constrained) weighted bipartite matchings, which can be solved efficiently using algorithms with approximation guarantees. Moreover, these algorithms also benefit from posterior sampling to actively trade off exploitation---selecting expert assignments which lead to accurate and fair decisions---and exploration---selecting expert assignments to learn about the experts' preferences and biases. We demonstrate the effectiveness of our algorithms on both synthetic and real-world data and show that they can significantly improve both the accuracy and fairness of the decisions taken by pools of experts.
Locatello, Francesco; Dresdner, Gideon; Khanna, Rajiv; Valera, Isabel; Rätsch, Gunnar
Boosting Black Box Variational Inference Proceedings Article
In: Bengio, Samy; Wallach, Hanna M.; Larochelle, Hugo; Grauman, Kristen; Cesa-Bianchi, Nicolò; Garnett, Roman (Ed.): Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pp. 3405–3415, 2018.
Abstract | Links | BibTeX | Tags: isabel
@inproceedings{DBLP:conf/nips/LocatelloDKVR18,
title = {Boosting Black Box Variational Inference},
author = {Francesco Locatello and Gideon Dresdner and Rajiv Khanna and Isabel Valera and Gunnar Rätsch},
editor = {Samy Bengio and Hanna M. Wallach and Hugo Larochelle and Kristen Grauman and Nicolò Cesa-Bianchi and Roman Garnett},
url = {https://proceedings.neurips.cc/paper/2018/hash/74563ba21a90da13dacf2a73e3ddefa7-Abstract.html},
year = {2018},
date = {2018-01-01},
urldate = {2018-01-01},
booktitle = {Advances in Neural Information Processing Systems 31: Annual Conference
on Neural Information Processing Systems 2018, NeurIPS 2018, December
3-8, 2018, Montréal, Canada},
pages = {3405--3415},
abstract = {Approximating a probability density in a tractable manner is a central task in Bayesian statistics. Variational Inference (VI) is a popular technique that achieves tractability by choosing a relatively simple variational approximation. Borrowing ideas from the classic boosting framework, recent approaches attempt to emph{boost} VI by replacing the selection of a single density with an iteratively constructed mixture of densities. In order to guarantee convergence, previous works impose stringent assumptions that require significant effort for practitioners. Specifically, they require a custom implementation of the greedy step (called the LMO) for every probabilistic model with respect to an unnatural variational family of truncated distributions. Our work fixes these issues with novel theoretical and algorithmic insights. On the theoretical side, we show that boosting VI satisfies a relaxed smoothness assumption which is sufficient for the convergence of the functional Frank-Wolfe (FW) algorithm. Furthermore, we rephrase the LMO problem and propose to maximize the Residual ELBO (RELBO) which replaces the standard ELBO optimization in VI. These theoretical enhancements allow for black box implementation of the boosting subroutine. Finally, we present a stopping criterion drawn from the duality gap in the classic FW analyses and exhaustive experiments to illustrate the usefulness of our theoretical and algorithmic contributions.},
keywords = {isabel},
pubstate = {published},
tppubtype = {inproceedings}
}
Approximating a probability density in a tractable manner is a central task in Bayesian statistics. Variational Inference (VI) is a popular technique that achieves tractability by choosing a relatively simple variational approximation. Borrowing ideas from the classic boosting framework, recent approaches attempt to emph{boost} VI by replacing the selection of a single density with an iteratively constructed mixture of densities. In order to guarantee convergence, previous works impose stringent assumptions that require significant effort for practitioners. Specifically, they require a custom implementation of the greedy step (called the LMO) for every probabilistic model with respect to an unnatural variational family of truncated distributions. Our work fixes these issues with novel theoretical and algorithmic insights. On the theoretical side, we show that boosting VI satisfies a relaxed smoothness assumption which is sufficient for the convergence of the functional Frank-Wolfe (FW) algorithm. Furthermore, we rephrase the LMO problem and propose to maximize the Residual ELBO (RELBO) which replaces the standard ELBO optimization in VI. These theoretical enhancements allow for black box implementation of the boosting subroutine. Finally, we present a stopping criterion drawn from the duality gap in the classic FW analyses and exhaustive experiments to illustrate the usefulness of our theoretical and algorithmic contributions.
Nazábal, Alfredo; Olmos, Pablo M.; Ghahramani, Zoubin; Valera, Isabel
Handling Incomplete Heterogeneous Data using VAEs Journal Article
In: CoRR, vol. abs/1807.03653, 2018.
Abstract | Links | BibTeX | Tags: isabel
@article{DBLP:journals/corr/abs-1807-03653,
title = {Handling Incomplete Heterogeneous Data using VAEs},
author = {Alfredo Nazábal and Pablo M. Olmos and Zoubin Ghahramani and Isabel Valera},
url = {http://arxiv.org/abs/1807.03653},
year = {2018},
date = {2018-01-01},
urldate = {2018-01-01},
journal = {CoRR},
volume = {abs/1807.03653},
abstract = {Variational autoencoders (VAEs), as well as other generative models, have been shown to be efficient and accurate for capturing the latent structure of vast amounts of complex high-dimensional data. However, existing VAEs can still not directly handle data that are heterogenous (mixed continuous and discrete) or incomplete (with missing data at random), which is indeed common in real-world applications.
In this paper, we propose a general framework to design VAEs suitable for fitting incomplete heterogenous data. The proposed HI-VAE includes likelihood models for real-valued, positive real valued, interval, categorical, ordinal and count data, and allows accurate estimation (and potentially imputation) of missing data. Furthermore, HI-VAE presents competitive predictive performance in supervised tasks, outperforming supervised models when trained on incomplete data.},
keywords = {isabel},
pubstate = {published},
tppubtype = {article}
}
Variational autoencoders (VAEs), as well as other generative models, have been shown to be efficient and accurate for capturing the latent structure of vast amounts of complex high-dimensional data. However, existing VAEs can still not directly handle data that are heterogenous (mixed continuous and discrete) or incomplete (with missing data at random), which is indeed common in real-world applications.
In this paper, we propose a general framework to design VAEs suitable for fitting incomplete heterogenous data. The proposed HI-VAE includes likelihood models for real-valued, positive real valued, interval, categorical, ordinal and count data, and allows accurate estimation (and potentially imputation) of missing data. Furthermore, HI-VAE presents competitive predictive performance in supervised tasks, outperforming supervised models when trained on incomplete data.
In this paper, we propose a general framework to design VAEs suitable for fitting incomplete heterogenous data. The proposed HI-VAE includes likelihood models for real-valued, positive real valued, interval, categorical, ordinal and count data, and allows accurate estimation (and potentially imputation) of missing data. Furthermore, HI-VAE presents competitive predictive performance in supervised tasks, outperforming supervised models when trained on incomplete data.
2017
Zafar, Muhammad Bilal; Valera, Isabel; Gomez-Rodriguez, Manuel; Gummadi, Krishna P.
Fairness Constraints: Mechanisms for Fair Classification Proceedings Article
In: Singh, Aarti; Zhu, Xiaojin (Jerry) (Ed.): Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, 20-22 April 2017, Fort Lauderdale, FL, USA, pp. 962–970, PMLR, 2017.
Abstract | Links | BibTeX | Tags: isabel, project-fairml
@inproceedings{DBLP:conf/aistats/ZafarVGG17,
title = {Fairness Constraints: Mechanisms for Fair Classification},
author = {Muhammad Bilal Zafar and Isabel Valera and Manuel Gomez-Rodriguez and Krishna P. Gummadi},
editor = {Aarti Singh and Xiaojin (Jerry) Zhu},
url = {http://proceedings.mlr.press/v54/zafar17a.html},
year = {2017},
date = {2017-01-01},
urldate = {2017-01-01},
booktitle = {Proceedings of the 20th International Conference on Artificial Intelligence
and Statistics, AISTATS 2017, 20-22 April 2017, Fort Lauderdale,
FL, USA},
volume = {54},
pages = {962--970},
publisher = {PMLR},
series = {Proceedings of Machine Learning Research},
abstract = {Algorithmic decision making systems are ubiquitous across a wide variety of online as well as offline services. These systems rely on complex learning methods and vast amounts of data to optimize the service functionality, satisfaction of the end user and profitability. However, there is a growing concern that these automated decisions can lead, even in the absence of intent, to a lack of fairness, i.e., their outcomes can disproportionately hurt (or, benefit) particular groups of people sharing one or more sensitive attributes (e.g., race, sex). In this paper, we introduce a flexible mechanism to design fair classifiers by leveraging a novel intuitive measure of decision boundary (un)fairness. We instantiate this mechanism with two well-known classifiers, logistic regression and support vector machines, and show on real-world data that our mechanism allows for a fine-grained control on the degree of fairness, often at a small cost in terms of accuracy.},
keywords = {isabel, project-fairml},
pubstate = {published},
tppubtype = {inproceedings}
}
Algorithmic decision making systems are ubiquitous across a wide variety of online as well as offline services. These systems rely on complex learning methods and vast amounts of data to optimize the service functionality, satisfaction of the end user and profitability. However, there is a growing concern that these automated decisions can lead, even in the absence of intent, to a lack of fairness, i.e., their outcomes can disproportionately hurt (or, benefit) particular groups of people sharing one or more sensitive attributes (e.g., race, sex). In this paper, we introduce a flexible mechanism to design fair classifiers by leveraging a novel intuitive measure of decision boundary (un)fairness. We instantiate this mechanism with two well-known classifiers, logistic regression and support vector machines, and show on real-world data that our mechanism allows for a fine-grained control on the degree of fairness, often at a small cost in terms of accuracy.
Valera, Isabel; Ghahramani, Zoubin
Automatic Discovery of the Statistical Types of Variables in a Dataset Proceedings Article
In: Precup, Doina; Teh, Yee Whye (Ed.): Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, pp. 3521–3529, PMLR, 2017.
Abstract | Links | BibTeX | Tags: isabel
@inproceedings{DBLP:conf/icml/ValeraG17,
title = {Automatic Discovery of the Statistical Types of Variables in a Dataset},
author = {Isabel Valera and Zoubin Ghahramani},
editor = {Doina Precup and Yee Whye Teh},
url = {http://proceedings.mlr.press/v70/valera17a.html},
year = {2017},
date = {2017-01-01},
urldate = {2017-01-01},
booktitle = {Proceedings of the 34th International Conference on Machine Learning,
ICML 2017, Sydney, NSW, Australia, 6-11 August 2017},
volume = {70},
pages = {3521--3529},
publisher = {PMLR},
series = {Proceedings of Machine Learning Research},
abstract = {A common practice in statistics and machine learning is to assume that the statistical data types (e.g., ordinal, categorical or real-valued) of variables, and usually also the likelihood model, is known. However, as the availability of real-world data increases, this assumption becomes too restrictive. Data are often heterogeneous, complex, and improperly or incompletely documented. Surprisingly, despite their practical importance, there is still a lack of tools to automatically discover the statistical types of, as well as appropriate likelihood (noise) models for, the variables in a dataset. In this paper, we fill this gap by proposing a Bayesian method, which accurately discovers the statistical data types in both synthetic and real data.},
keywords = {isabel},
pubstate = {published},
tppubtype = {inproceedings}
}
A common practice in statistics and machine learning is to assume that the statistical data types (e.g., ordinal, categorical or real-valued) of variables, and usually also the likelihood model, is known. However, as the availability of real-world data increases, this assumption becomes too restrictive. Data are often heterogeneous, complex, and improperly or incompletely documented. Surprisingly, despite their practical importance, there is still a lack of tools to automatically discover the statistical types of, as well as appropriate likelihood (noise) models for, the variables in a dataset. In this paper, we fill this gap by proposing a Bayesian method, which accurately discovers the statistical data types in both synthetic and real data.
Zafar, Muhammad Bilal; Valera, Isabel; Gomez-Rodriguez, Manuel; Gummadi, Krishna P.; Weller, Adrian
From Parity to Preference-based Notions of Fairness in Classification Proceedings Article
In: Guyon, Isabelle; Luxburg, Ulrike; Bengio, Samy; Wallach, Hanna M.; Fergus, Rob; Vishwanathan, S. V. N.; Garnett, Roman (Ed.): Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp. 229–239, 2017.
Abstract | Links | BibTeX | Tags: isabel, project-fairml
@inproceedings{DBLP:conf/nips/ZafarVGGW17,
title = {From Parity to Preference-based Notions of Fairness in Classification},
author = {Muhammad Bilal Zafar and Isabel Valera and Manuel Gomez-Rodriguez and Krishna P. Gummadi and Adrian Weller},
editor = {Isabelle Guyon and Ulrike Luxburg and Samy Bengio and Hanna M. Wallach and Rob Fergus and S. V. N. Vishwanathan and Roman Garnett},
url = {https://proceedings.neurips.cc/paper/2017/hash/82161242827b703e6acf9c726942a1e4-Abstract.html},
year = {2017},
date = {2017-01-01},
urldate = {2017-01-01},
booktitle = {Advances in Neural Information Processing Systems 30: Annual Conference
on Neural Information Processing Systems 2017, December 4-9, 2017,
Long Beach, CA, USA},
pages = {229--239},
abstract = {The adoption of automated, data-driven decision making in an ever expanding range of applications has raised concerns about its potential unfairness towards certain social groups. In this context, a number of recent studies have focused on defining, detecting, and removing unfairness from data-driven decision systems. However, the existing notions of fairness, based on parity (equality) in treatment or outcomes for different social groups, tend to be quite stringent, limiting the overall decision making accuracy. In this paper, we draw inspiration from the fair-division and envy-freeness literature in economics and game theory and propose preference-based notions of fairness -- given the choice between various sets of decision treatments or outcomes, any group of users would collectively prefer its treatment or outcomes, regardless of the (dis)parity as compared to the other groups. Then, we introduce tractable proxies to design margin-based classifiers that satisfy these preference-based notions of fairness. Finally, we experiment with a variety of synthetic and real-world datasets and show that preference-based fairness allows for greater decision accuracy than parity-based fairness.},
keywords = {isabel, project-fairml},
pubstate = {published},
tppubtype = {inproceedings}
}
The adoption of automated, data-driven decision making in an ever expanding range of applications has raised concerns about its potential unfairness towards certain social groups. In this context, a number of recent studies have focused on defining, detecting, and removing unfairness from data-driven decision systems. However, the existing notions of fairness, based on parity (equality) in treatment or outcomes for different social groups, tend to be quite stringent, limiting the overall decision making accuracy. In this paper, we draw inspiration from the fair-division and envy-freeness literature in economics and game theory and propose preference-based notions of fairness -- given the choice between various sets of decision treatments or outcomes, any group of users would collectively prefer its treatment or outcomes, regardless of the (dis)parity as compared to the other groups. Then, we introduce tractable proxies to design margin-based classifiers that satisfy these preference-based notions of fairness. Finally, we experiment with a variety of synthetic and real-world datasets and show that preference-based fairness allows for greater decision accuracy than parity-based fairness.
Upadhyay, Utkarsh; Valera, Isabel; Gomez-Rodriguez, Manuel
Uncovering the Dynamics of Crowdlearning and the Value of Knowledge Proceedings Article
In: Rijke, Maarten; Shokouhi, Milad; Tomkins, Andrew; Zhang, Min (Ed.): Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, WSDM 2017, Cambridge, United Kingdom, February 6-10, 2017, pp. 61–70, ACM, 2017.
Abstract | Links | BibTeX | Tags: isabel
@inproceedings{DBLP:conf/wsdm/UpadhyayVG17,
title = {Uncovering the Dynamics of Crowdlearning and the Value of Knowledge},
author = {Utkarsh Upadhyay and Isabel Valera and Manuel Gomez-Rodriguez},
editor = {Maarten Rijke and Milad Shokouhi and Andrew Tomkins and Min Zhang},
url = {https://doi.org/10.1145/3018661.3018685},
doi = {10.1145/3018661.3018685},
year = {2017},
date = {2017-01-01},
urldate = {2017-01-01},
booktitle = {Proceedings of the Tenth ACM International Conference on Web Search
and Data Mining, WSDM 2017, Cambridge, United Kingdom, February
6-10, 2017},
pages = {61--70},
publisher = {ACM},
abstract = {Learning from the crowd has become increasingly popular in the Web and social media. There is a wide variety of crowdlearning sites in which, on the one hand, users learn from the knowledge that other users contribute to the site, and, on the other hand, knowledge is reviewed and curated by the same users using assessment measures such as upvotes or likes. In this paper, we present a probabilistic modeling framework of crowdlearning, which uncovers the evolution of a user's expertise over time by leveraging other users' assessments of her contributions. The model allows for both off-site and on-site learning and captures forgetting of knowledge. We then develop a scalable estimation method to fit the model parameters from millions of recorded learning and contributing events. We show the effectiveness of our model by tracing activity of ~25 thousand users in Stack Overflow over a 4.5 year period. We find that answers with high knowledge value are rare. Newbies and experts tend to acquire less knowledge than users in the middle range. Prolific learners tend to be also proficient contributors that post answers with high knowledge value.},
keywords = {isabel},
pubstate = {published},
tppubtype = {inproceedings}
}
Learning from the crowd has become increasingly popular in the Web and social media. There is a wide variety of crowdlearning sites in which, on the one hand, users learn from the knowledge that other users contribute to the site, and, on the other hand, knowledge is reviewed and curated by the same users using assessment measures such as upvotes or likes. In this paper, we present a probabilistic modeling framework of crowdlearning, which uncovers the evolution of a user's expertise over time by leveraging other users' assessments of her contributions. The model allows for both off-site and on-site learning and captures forgetting of knowledge. We then develop a scalable estimation method to fit the model parameters from millions of recorded learning and contributing events. We show the effectiveness of our model by tracing activity of ~25 thousand users in Stack Overflow over a 4.5 year period. We find that answers with high knowledge value are rare. Newbies and experts tend to acquire less knowledge than users in the middle range. Prolific learners tend to be also proficient contributors that post answers with high knowledge value.
Tabibian, Behzad; Valera, Isabel; Farajtabar, Mehrdad; Song, Le; Schölkopf, Bernhard; Gomez-Rodriguez, Manuel
Distilling Information Reliability and Source Trustworthiness from Digital Traces Proceedings Article
In: Barrett, Rick; Cummings, Rick; Agichtein, Eugene; Gabrilovich, Evgeniy (Ed.): Proceedings of the 26th International Conference on World Wide Web, WWW 2017, Perth, Australia, April 3-7, 2017, pp. 847–855, ACM, 2017.
Abstract | Links | BibTeX | Tags: isabel
@inproceedings{DBLP:conf/www/TabibianVFSSG17,
title = {Distilling Information Reliability and Source Trustworthiness from Digital Traces},
author = {Behzad Tabibian and Isabel Valera and Mehrdad Farajtabar and Le Song and Bernhard Schölkopf and Manuel Gomez-Rodriguez},
editor = {Rick Barrett and Rick Cummings and Eugene Agichtein and Evgeniy Gabrilovich},
url = {https://doi.org/10.1145/3038912.3052672},
doi = {10.1145/3038912.3052672},
year = {2017},
date = {2017-01-01},
urldate = {2017-01-01},
booktitle = {Proceedings of the 26th International Conference on World Wide Web,
WWW 2017, Perth, Australia, April 3-7, 2017},
pages = {847--855},
publisher = {ACM},
abstract = {Online knowledge repositories typically rely on their users or dedicated editors to evaluate the reliability of their contents. These explicit feedback mechanisms can be viewed as noisy measurements of both information reliability and information source trustworthiness. Can we leverage these noisy measurements, often biased, to distill a robust, unbiased and interpretable measure of both notions?
In this paper, we argue that the large volume of digital traces left by the users within knowledge repositories also reflect information reliability and source trustworthiness. In particular, we propose a temporal point process modeling framework which links the temporal behavior of the users to information reliability and source trustworthiness. Furthermore, we develop an efficient convex optimization procedure to learn the parameters of the model from historical traces of the evaluations provided by these users. Experiments on real-world data gathered from Wikipedia and Stack Overflow show that our modeling framework accurately predicts evaluation events, provides an interpretable measure of information reliability and source trustworthiness, and yields interesting insights about real-world events.},
keywords = {isabel},
pubstate = {published},
tppubtype = {inproceedings}
}
Online knowledge repositories typically rely on their users or dedicated editors to evaluate the reliability of their contents. These explicit feedback mechanisms can be viewed as noisy measurements of both information reliability and information source trustworthiness. Can we leverage these noisy measurements, often biased, to distill a robust, unbiased and interpretable measure of both notions?
In this paper, we argue that the large volume of digital traces left by the users within knowledge repositories also reflect information reliability and source trustworthiness. In particular, we propose a temporal point process modeling framework which links the temporal behavior of the users to information reliability and source trustworthiness. Furthermore, we develop an efficient convex optimization procedure to learn the parameters of the model from historical traces of the evaluations provided by these users. Experiments on real-world data gathered from Wikipedia and Stack Overflow show that our modeling framework accurately predicts evaluation events, provides an interpretable measure of information reliability and source trustworthiness, and yields interesting insights about real-world events.
In this paper, we argue that the large volume of digital traces left by the users within knowledge repositories also reflect information reliability and source trustworthiness. In particular, we propose a temporal point process modeling framework which links the temporal behavior of the users to information reliability and source trustworthiness. Furthermore, we develop an efficient convex optimization procedure to learn the parameters of the model from historical traces of the evaluations provided by these users. Experiments on real-world data gathered from Wikipedia and Stack Overflow show that our modeling framework accurately predicts evaluation events, provides an interpretable measure of information reliability and source trustworthiness, and yields interesting insights about real-world events.
Zafar, Muhammad Bilal; Valera, Isabel; Gomez-Rodriguez, Manuel; Gummadi, Krishna P.
Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment Proceedings Article
In: Barrett, Rick; Cummings, Rick; Agichtein, Eugene; Gabrilovich, Evgeniy (Ed.): Proceedings of the 26th International Conference on World Wide Web, WWW 2017, Perth, Australia, April 3-7, 2017, pp. 1171–1180, ACM, 2017.
Abstract | Links | BibTeX | Tags: isabel, project-fairml
@inproceedings{DBLP:conf/www/ZafarVGG17,
title = {Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment},
author = {Muhammad Bilal Zafar and Isabel Valera and Manuel Gomez-Rodriguez and Krishna P. Gummadi},
editor = {Rick Barrett and Rick Cummings and Eugene Agichtein and Evgeniy Gabrilovich},
url = {https://doi.org/10.1145/3038912.3052660},
doi = {10.1145/3038912.3052660},
year = {2017},
date = {2017-01-01},
urldate = {2017-01-01},
booktitle = {Proceedings of the 26th International Conference on World Wide Web,
WWW 2017, Perth, Australia, April 3-7, 2017},
pages = {1171--1180},
publisher = {ACM},
abstract = {Automated data-driven decision making systems are increasingly being used to assist, or even replace humans in many settings. These systems function by learning from historical decisions, often taken by humans. In order to maximize the utility of these systems (or, classifiers), their training involves minimizing the errors (or, misclassifications) over the given historical data. However, it is quite possible that the optimally trained classifier makes decisions for people belonging to different social groups with different misclassification rates (e.g., misclassification rates for females are higher than for males), thereby placing these groups at an unfair disadvantage. To account for and avoid such unfairness, in this paper, we introduce a new notion of unfairness, disparate mistreatment, which is defined in terms of misclassification rates. We then propose intuitive measures of disparate mistreatment for decision boundary-based classifiers, which can be easily incorporated into their formulation as convex-concave constraints. Experiments on synthetic as well as real world datasets show that our methodology is effective at avoiding disparate mistreatment, often at a small cost in terms of accuracy.},
keywords = {isabel, project-fairml},
pubstate = {published},
tppubtype = {inproceedings}
}
Automated data-driven decision making systems are increasingly being used to assist, or even replace humans in many settings. These systems function by learning from historical decisions, often taken by humans. In order to maximize the utility of these systems (or, classifiers), their training involves minimizing the errors (or, misclassifications) over the given historical data. However, it is quite possible that the optimally trained classifier makes decisions for people belonging to different social groups with different misclassification rates (e.g., misclassification rates for females are higher than for males), thereby placing these groups at an unfair disadvantage. To account for and avoid such unfairness, in this paper, we introduce a new notion of unfairness, disparate mistreatment, which is defined in terms of misclassification rates. We then propose intuitive measures of disparate mistreatment for decision boundary-based classifiers, which can be easily incorporated into their formulation as convex-concave constraints. Experiments on synthetic as well as real world datasets show that our methodology is effective at avoiding disparate mistreatment, often at a small cost in terms of accuracy.
Mavroforakis, Charalampos; Valera, Isabel; Gomez-Rodriguez, Manuel
Modeling the Dynamics of Learning Activity on the Web Proceedings Article
In: Barrett, Rick; Cummings, Rick; Agichtein, Eugene; Gabrilovich, Evgeniy (Ed.): Proceedings of the 26th International Conference on World Wide Web, WWW 2017, Perth, Australia, April 3-7, 2017, pp. 1421–1430, ACM, 2017.
Abstract | Links | BibTeX | Tags: isabel
@inproceedings{DBLP:conf/www/MavroforakisVG17,
title = {Modeling the Dynamics of Learning Activity on the Web},
author = {Charalampos Mavroforakis and Isabel Valera and Manuel Gomez-Rodriguez},
editor = {Rick Barrett and Rick Cummings and Eugene Agichtein and Evgeniy Gabrilovich},
url = {https://doi.org/10.1145/3038912.3052669},
doi = {10.1145/3038912.3052669},
year = {2017},
date = {2017-01-01},
urldate = {2017-01-01},
booktitle = {Proceedings of the 26th International Conference on World Wide Web,
WWW 2017, Perth, Australia, April 3-7, 2017},
pages = {1421--1430},
publisher = {ACM},
abstract = {People are increasingly relying on social media and the Web to find solutions to their problems in a wide range of domains. In this setting, closely related problems often lead to the same characteristic learning pattern --- people sharing a similar problem visit closely related pieces of information, perform almost identical queries or, more generally, take a series of similar actions at a similar pace. In this paper, we introduce a novel modeling framework for clustering continuous-time grouped streaming data, the Hierarchical Dirichlet Hawkes process (HDHP), which allows us to automatically uncover a wide variety of learning patterns from detailed traces of learning activity. Our model allows for efficient inference, scaling to millions of actions and thousands of users. Experiments on real data from Stack Overflow reveal that our framework recovers meaningful learning patterns, accurately tracks users' interests and goals over time and achieves better predictive performance than the state of the art.},
keywords = {isabel},
pubstate = {published},
tppubtype = {inproceedings}
}
People are increasingly relying on social media and the Web to find solutions to their problems in a wide range of domains. In this setting, closely related problems often lead to the same characteristic learning pattern --- people sharing a similar problem visit closely related pieces of information, perform almost identical queries or, more generally, take a series of similar actions at a similar pace. In this paper, we introduce a novel modeling framework for clustering continuous-time grouped streaming data, the Hierarchical Dirichlet Hawkes process (HDHP), which allows us to automatically uncover a wide variety of learning patterns from detailed traces of learning activity. Our model allows for efficient inference, scaling to millions of actions and thousands of users. Experiments on real data from Stack Overflow reveal that our framework recovers meaningful learning patterns, accurately tracks users' interests and goals over time and achieves better predictive performance than the state of the art.
Valera, Isabel; Pradier, Melanie F.; Ghahramani, Zoubin
General Latent Feature Modeling for Data Exploration Tasks Journal Article
In: CoRR, vol. abs/1707.08352, 2017.
Abstract | Links | BibTeX | Tags: isabel
@article{DBLP:journals/corr/ValeraPG17,
title = {General Latent Feature Modeling for Data Exploration Tasks},
author = {Isabel Valera and Melanie F. Pradier and Zoubin Ghahramani},
url = {http://arxiv.org/abs/1707.08352},
year = {2017},
date = {2017-01-01},
urldate = {2017-01-01},
journal = {CoRR},
volume = {abs/1707.08352},
abstract = {This paper introduces a general Bayesian non- parametric latent feature model suitable to per- form automatic exploratory analysis of heterogeneous datasets, where the attributes describing each object can be either discrete, continuous or mixed variables. The proposed model presents several important properties. First, it accounts for heterogeneous data while can be inferred in linear time with respect to the number of objects and attributes. Second, its Bayesian nonparametric nature allows us to automatically infer the model complexity from the data, i.e., the number of features necessary to capture the latent structure in the data. Third, the latent features in the model are binary-valued variables, easing the interpretability of the obtained latent features in data exploration tasks.},
keywords = {isabel},
pubstate = {published},
tppubtype = {article}
}
This paper introduces a general Bayesian non- parametric latent feature model suitable to per- form automatic exploratory analysis of heterogeneous datasets, where the attributes describing each object can be either discrete, continuous or mixed variables. The proposed model presents several important properties. First, it accounts for heterogeneous data while can be inferred in linear time with respect to the number of objects and attributes. Second, its Bayesian nonparametric nature allows us to automatically infer the model complexity from the data, i.e., the number of features necessary to capture the latent structure in the data. Third, the latent features in the model are binary-valued variables, easing the interpretability of the obtained latent features in data exploration tasks.
0000
Gros, Timo P.; M¨uller, Nicola J.; Fiˇser, Daniel; Valera, Isabel; Wolf, Verena; Hoffmann, J¨org
Per-Domain Generalizing Policies for Classical Planning: On Scaling Behavior and Validation Instances Proceedings Article
In: 0000.
@inproceedings{GrosPerDomainGP,
title = {Per-Domain Generalizing Policies for Classical Planning: On Scaling Behavior and Validation Instances},
author = {Timo P. Gros and Nicola J. M¨uller and Daniel Fiˇser and Isabel Valera and Verena Wolf and J¨org Hoffmann},
url = {https://api.semanticscholar.org/CorpusID:281213806},
keywords = {isabel},
pubstate = {published},
tppubtype = {inproceedings}
}