Robust generative models for real-world data
A core goal of our research is to develop robust ML methods that can properly handle realistic assumptions and, especially, the complex nature of real-world data. In this context, we have extensively worked on ML for mixed continuous and discrete data (e.g., tabular data), as well as with missing data (NeurIPS’15; ICML’17; AAAI’19; JMLR’20)—e.g., to the best of my knowledge, HI-VAE (Pattern Recognition’20) was the first deep generative model to handle both heterogeneous and missing data.
Specifically, we focus on developing generative models, based on latent variable models for heterogenous data (e.g., tabular datasets). Our extensive work in this area include bayesian non-parametric models (see “General Latent Feature Models for Heterogeneous Datasets”), sum and product networks (SPNs) and variational autoencoders (VAEs)) (see “Automatic Bayesian Density Analysis”); and our applications include automatic and efficient missing value estimation, statistical data type and likelihood discovery, anomaly detection and dependency structure mining, etc.
Our first line of ongoing research in the area of latent variable models have extended to account not only for heterogenous dataset (e.g., tabular datasets) but also multimodal data (containing, e.g., e.g., images and their corresponding captions), especially on those models relying on neural networks and thus gradient-based optimization, see “Boosting heterogeneous VAEs via multi-objective optimization}”. In particular, we aim to mitigate that issues similar to negative transfer in multi-task learning (as a consequence of the gradient competition across tasks) prevent the model to accurately fit the marginal distribution of, and the statistical dependencies among, all features.
In a second line of ongoing research, we are also investigating how to leverage expressive probabilistic models to perform causal reasoning under limited causal assumptions. Our initial results in this line allows us to evaluate the effect of causal interventions and estimate the value of structure counterfactuals assuming access to only observational data and the underlying causal graph (but not the structural equations) (NeurIPS’20, AAAI’22).
References
2022
Mitigating Modality Collapse in Multimodal VAEs via Impartial Optimization Journal Article
In: CoRR, vol. abs/2206.04496, 2022.
2021
Boosting heterogeneous VAEs via multi-objective optimization Workshop
2021.
2020
General Latent Feature Models for Heterogeneous Datasets Journal Article
In: J. Mach. Learn. Res., vol. 21, pp. 100:1–100:49, 2020.
2019
Automatic Bayesian Density Analysis Proceedings Article
In: The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI
2019, The Thirty-First Innovative Applications of Artificial Intelligence
Conference, IAAI 2019, The Ninth AAAI Symposium on Educational
Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii,
USA, January 27 - February 1, 2019, pp. 5207–5215, AAAI Press, 2019.
2022
Mitigating Modality Collapse in Multimodal VAEs via Impartial Optimization Journal Article
In: CoRR, vol. abs/2206.04496, 2022.
2021
Boosting heterogeneous VAEs via multi-objective optimization Workshop
2021.
2020
General Latent Feature Models for Heterogeneous Datasets Journal Article
In: J. Mach. Learn. Res., vol. 21, pp. 100:1–100:49, 2020.
2019
Automatic Bayesian Density Analysis Proceedings Article
In: The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019, pp. 5207–5215, AAAI Press, 2019.