Machine Learning – Publications
1.
Javaloy, Adrián; Meghdadi, Maryam; Valera, Isabel
Mitigating Modality Collapse in Multimodal VAEs via Impartial Optimization Journal Article Spotlight
In: CoRR, vol. abs/2206.04496, 2022.
Abstract | Links | BibTeX | Tags: adrian, isabel, maryam, project-robustgenerative, spotlight, variational autoencoder
@article{DBLP:journals/corr/abs-2206-04496,
title = {Mitigating Modality Collapse in Multimodal VAEs via Impartial Optimization},
author = {Adrián Javaloy and Maryam Meghdadi and Isabel Valera},
url = {https://doi.org/10.48550/arXiv.2206.04496},
doi = {10.48550/arXiv.2206.04496},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {CoRR},
volume = {abs/2206.04496},
abstract = {A number of variational autoencoders (VAEs) have recently emerged with the aim of modeling multimodal data, e.g., to jointly model images and their corresponding captions. Still, multimodal VAEs tend to focus solely on a subset of the modalities, e.g., by fitting the image while neglecting the caption. We refer to this limitation as modality collapse. In this work, we argue that this effect is a consequence of conflicting gradients during multimodal VAE training. We show how to detect the sub-graphs in the computational graphs where gradients conflict (impartiality blocks), as well as how to leverage existing gradient-conflict solutions from multitask learning to mitigate modality collapse. That is, to ensure impartial optimization across modalities. We apply our training framework to several multimodal VAE models, losses and datasets from the literature, and empirically show that our framework significantly improves the reconstruction performance, conditional generation, and coherence of the latent space across modalities.},
keywords = {adrian, isabel, maryam, project-robustgenerative, spotlight, variational autoencoder},
pubstate = {published},
tppubtype = {article}
}
A number of variational autoencoders (VAEs) have recently emerged with the aim of modeling multimodal data, e.g., to jointly model images and their corresponding captions. Still, multimodal VAEs tend to focus solely on a subset of the modalities, e.g., by fitting the image while neglecting the caption. We refer to this limitation as modality collapse. In this work, we argue that this effect is a consequence of conflicting gradients during multimodal VAE training. We show how to detect the sub-graphs in the computational graphs where gradients conflict (impartiality blocks), as well as how to leverage existing gradient-conflict solutions from multitask learning to mitigate modality collapse. That is, to ensure impartial optimization across modalities. We apply our training framework to several multimodal VAE models, losses and datasets from the literature, and empirically show that our framework significantly improves the reconstruction performance, conditional generation, and coherence of the latent space across modalities.
2.
Rateike, Miriam; Majumdar, Ayan; Mineeva, Olga; Gummadi, Krishna P.; Valera, Isabel
Don't Throw it Away! The Utility of Unlabeled Data in Fair Decision Making Proceedings Article
In: FAccT '22: 2022 ACM Conference on Fairness, Accountability, and Transparency, Seoul, Republic of Korea, June 21 - 24, 2022, pp. 1421–1433, ACM, 2022.
Abstract | Links | BibTeX | Tags: ayanm, decision making, fair representation, fairness, isabel, label bias, miriam, project-fairml, selection bias, variational autoencoder
@inproceedings{DBLP:conf/fat/RateikeMMGV22,
title = {Don't Throw it Away! The Utility of Unlabeled Data in Fair Decision Making},
author = {Miriam Rateike and Ayan Majumdar and Olga Mineeva and Krishna P. Gummadi and Isabel Valera},
url = {https://doi.org/10.1145/3531146.3533199},
doi = {10.1145/3531146.3533199},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {FAccT '22: 2022 ACM Conference on Fairness, Accountability, and
Transparency, Seoul, Republic of Korea, June 21 - 24, 2022},
pages = {1421–1433},
publisher = {ACM},
abstract = {unbiased, i.e., equally distributed across socially salient groups. In many practical settings, the ground-truth cannot be directly observed, and instead, we have to rely on a biased proxy measure of the ground-truth, i.e., biased labels, in the data. In addition, data is often selectively labeled, i.e., even the biased labels are only observed for a small fraction of the data that received a positive decision. To overcome label and selection biases, recent work proposes to learn stochastic, exploring decision policies via i) online training of new policies at each time-step and ii) enforcing fairness as a constraint on performance. However, the existing approach uses only labeled data, disregarding a large amount of unlabeled data, and thereby suffers from high instability and variance in the learned decision policies at different times. In this paper, we propose a novel method based on a variational autoencoder for practical fair decision-making. Our method learns an unbiased data representation leveraging both labeled and unlabeled data and uses the representations to learn a policy in an online process. Using synthetic data, we empirically validate that our method converges to the optimal (fair) policy according to the ground-truth with low variance. In real-world experiments, we further show that our training approach not only offers a more stable learning process but also yields policies with higher fairness as well as utility than previous approaches.},
keywords = {ayanm, decision making, fair representation, fairness, isabel, label bias, miriam, project-fairml, selection bias, variational autoencoder},
pubstate = {published},
tppubtype = {inproceedings}
}
unbiased, i.e., equally distributed across socially salient groups. In many practical settings, the ground-truth cannot be directly observed, and instead, we have to rely on a biased proxy measure of the ground-truth, i.e., biased labels, in the data. In addition, data is often selectively labeled, i.e., even the biased labels are only observed for a small fraction of the data that received a positive decision. To overcome label and selection biases, recent work proposes to learn stochastic, exploring decision policies via i) online training of new policies at each time-step and ii) enforcing fairness as a constraint on performance. However, the existing approach uses only labeled data, disregarding a large amount of unlabeled data, and thereby suffers from high instability and variance in the learned decision policies at different times. In this paper, we propose a novel method based on a variational autoencoder for practical fair decision-making. Our method learns an unbiased data representation leveraging both labeled and unlabeled data and uses the representations to learn a policy in an online process. Using synthetic data, we empirically validate that our method converges to the optimal (fair) policy according to the ground-truth with low variance. In real-world experiments, we further show that our training approach not only offers a more stable learning process but also yields policies with higher fairness as well as utility than previous approaches.