People
Adrián Javaloy
About me
Hi there 👋 My name is Adrián, and I am a PhD student at the Probabilistic Machine Learning group under the supervision of Isabel Valera at Saarland University. I am also an alumnus of the Empirical Inference department at the Max Planck Institute for Intelligent Systems. Before pursuing my PhD, I completed a Master in Computer Science and a double Bachelor Degree in Computer Science Engineering and Mathematics at the University of Murcia.
I am mostly interested in principled research that aims to improve machine learning methods, making them easier to use and more robust. Specifically, my current research focuses on improving models that optimize different tasks simultaneously. This includes apparently unrelated areas such as multitask learning (MTL) and deep probabilistic generative models (e.g. VAEs) under heterogeneous environments.
However, I am a curiosity-driven, and I am always happy to discuss interesting ideas even if they lie outside my comfort zone. Feel free to reach me out via Twitter or email.
Publications
2024
Sánchez-Martin, Pablo; Utz, Sonja; Valera, Isabel
Exploring the Boundaries of Ambient Awareness in Twitter Journal Article
In: CoRR, vol. abs/2403.17776, 2024.
Abstract | Links | BibTeX | Tags: isabel, pablo
@article{DBLP:journals/corr/abs-2403-17776,
title = {Exploring the Boundaries of Ambient Awareness in Twitter},
author = {Pablo Sánchez-Martin and Sonja Utz and Isabel Valera},
url = {https://doi.org/10.48550/arXiv.2403.17776},
doi = {10.48550/ARXIV.2403.17776},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {CoRR},
volume = {abs/2403.17776},
abstract = {Ambient awareness refers to the ability of social media users to obtain knowledge about who knows what (i.e., users' expertise) in their network, by simply being exposed to other users' content (e.g, tweets on Twitter). Previous work, based on user surveys, reveals that individuals self-report ambient awareness only for parts of their networks. However, it is unclear whether it is their limited cognitive capacity or the limited exposure to diagnostic tweets (i.e., online content) that prevents people from developing ambient awareness for their complete network. In this work, we focus on in-wall ambient awareness (IWAA) in Twitter and conduct a two-step data-driven analysis, that allows us to explore to which extent IWAA is likely, or even possible. First, we rely on reactions (e.g., likes), as strong evidence of users being aware of experts in Twitter. Unfortunately, such strong evidence can be only measured for active users, which represent the minority in the network. Thus to study the boundaries of IWAA to a larger extent, in the second part of our analysis, we instead focus on the passive exposure to content generated by other users -- which we refer to as in-wall visibility. This analysis shows that (in line with citet{levordashka2016ambient}) only for a subset of users IWAA is plausible, while for the majority it is unlikely, if even possible, to develop IWAA. We hope that our methodology paves the way for the emergence of data-driven approaches for the study of ambient awareness.},
keywords = {isabel, pablo},
pubstate = {published},
tppubtype = {article}
}
Ambient awareness refers to the ability of social media users to obtain knowledge about who knows what (i.e., users' expertise) in their network, by simply being exposed to other users' content (e.g, tweets on Twitter). Previous work, based on user surveys, reveals that individuals self-report ambient awareness only for parts of their networks. However, it is unclear whether it is their limited cognitive capacity or the limited exposure to diagnostic tweets (i.e., online content) that prevents people from developing ambient awareness for their complete network. In this work, we focus on in-wall ambient awareness (IWAA) in Twitter and conduct a two-step data-driven analysis, that allows us to explore to which extent IWAA is likely, or even possible. First, we rely on reactions (e.g., likes), as strong evidence of users being aware of experts in Twitter. Unfortunately, such strong evidence can be only measured for active users, which represent the minority in the network. Thus to study the boundaries of IWAA to a larger extent, in the second part of our analysis, we instead focus on the passive exposure to content generated by other users -- which we refer to as in-wall visibility. This analysis shows that (in line with citet{levordashka2016ambient}) only for a subset of users IWAA is plausible, while for the majority it is unlikely, if even possible, to develop IWAA. We hope that our methodology paves the way for the emergence of data-driven approaches for the study of ambient awareness.
Sánchez-Martin, Pablo; Khan, Kinaan Aamir; Valera, Isabel
Improving the interpretability of GNN predictions through conformal-based graph sparsification Journal Article
In: CoRR, vol. abs/2404.12356, 2024.
Abstract | Links | BibTeX | Tags: isabel, pablo
@article{DBLP:journals/corr/abs-2404-12356,
title = {Improving the interpretability of GNN predictions through conformal-based graph sparsification},
author = {Pablo Sánchez-Martin and Kinaan Aamir Khan and Isabel Valera},
url = {https://doi.org/10.48550/arXiv.2404.12356},
doi = {10.48550/ARXIV.2404.12356},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {CoRR},
volume = {abs/2404.12356},
abstract = {Graph Neural Networks (GNNs) have achieved state-of-the-art performance in solving graph classification tasks. However, most GNN architectures aggregate information from all nodes and edges in a graph, regardless of their relevance to the task at hand, thus hindering the interpretability of their predictions. In contrast to prior work, in this paper we propose a GNN emph{training} approach that jointly i) finds the most predictive subgraph by removing edges and/or nodes -- -emph{without making assumptions about the subgraph structure} -- while ii) optimizing the performance of the graph classification task. To that end, we rely on reinforcement learning to solve the resulting bi-level optimization with a reward function based on conformal predictions to account for the current in-training uncertainty of the classifier. Our empirical results on nine different graph classification datasets show that our method competes in performance with baselines while relying on significantly sparser subgraphs, leading to more interpretable GNN-based predictions.},
keywords = {isabel, pablo},
pubstate = {published},
tppubtype = {article}
}
Graph Neural Networks (GNNs) have achieved state-of-the-art performance in solving graph classification tasks. However, most GNN architectures aggregate information from all nodes and edges in a graph, regardless of their relevance to the task at hand, thus hindering the interpretability of their predictions. In contrast to prior work, in this paper we propose a GNN emph{training} approach that jointly i) finds the most predictive subgraph by removing edges and/or nodes -- -emph{without making assumptions about the subgraph structure} -- while ii) optimizing the performance of the graph classification task. To that end, we rely on reinforcement learning to solve the resulting bi-level optimization with a reward function based on conformal predictions to account for the current in-training uncertainty of the classifier. Our empirical results on nine different graph classification datasets show that our method competes in performance with baselines while relying on significantly sparser subgraphs, leading to more interpretable GNN-based predictions.
2023
Javaloy, Adrián; Sánchez-Mart'ın, Pablo; Valera, Isabel
Causal normalizing flows: from theory to practice Journal Article Oral
In: CoRR, vol. abs/2306.05415, 2023.
Abstract | Links | BibTeX | Tags: adrian, isabel, oral, pablo
@article{DBLP:journals/corr/abs-2306-05415,
title = {Causal normalizing flows: from theory to practice},
author = {Adrián Javaloy and Pablo Sánchez-Mart'ın and Isabel Valera},
url = {https://doi.org/10.48550/arXiv.2306.05415},
doi = {10.48550/arXiv.2306.05415},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2306.05415},
abstract = {In this work, we deepen on the use of normalizing flows for causal reasoning. Specifically, we first leverage recent results on non-linear ICA to show that causal models are identifiable from observational data given a causal ordering, and thus can be recovered using autoregressive normalizing flows (NFs). Second, we analyze different design and learning choices for causal normalizing flows to capture the underlying causal data-generating process. Third, we describe how to implement the do-operator in causal NFs, and thus, how to answer interventional and counterfactual questions. Finally, in our experiments, we validate our design and training choices through a comprehensive ablation study; compare causal NFs to other approaches for approximating causal models; and empirically demonstrate that causal NFs can be used to address real-world problems—where the presence of mixed discrete-continuous data and partial knowledge on the causal graph is the norm. The code for this work can be found at https://github.com/psanch21/causal-flows.},
keywords = {adrian, isabel, oral, pablo},
pubstate = {published},
tppubtype = {article}
}
In this work, we deepen on the use of normalizing flows for causal reasoning. Specifically, we first leverage recent results on non-linear ICA to show that causal models are identifiable from observational data given a causal ordering, and thus can be recovered using autoregressive normalizing flows (NFs). Second, we analyze different design and learning choices for causal normalizing flows to capture the underlying causal data-generating process. Third, we describe how to implement the do-operator in causal NFs, and thus, how to answer interventional and counterfactual questions. Finally, in our experiments, we validate our design and training choices through a comprehensive ablation study; compare causal NFs to other approaches for approximating causal models; and empirically demonstrate that causal NFs can be used to address real-world problems—where the presence of mixed discrete-continuous data and partial knowledge on the causal graph is the norm. The code for this work can be found at https://github.com/psanch21/causal-flows.
Javaloy, Adrián; Sánchez-Mart'ın, Pablo; Levi, Amit; Valera, Isabel
Learnable Graph Convolutional Attention Networks Proceedings Article
In: The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023, OpenReview.net, 2023.
Abstract | Links | BibTeX | Tags: adrian, isabel, pablo
@inproceedings{DBLP:conf/iclr/JavaloySLV23,
title = {Learnable Graph Convolutional Attention Networks},
author = {Adrián Javaloy and Pablo Sánchez-Mart'ın and Amit Levi and Isabel Valera},
url = {https://openreview.net/pdf?id=WsUMeHPo-2},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
booktitle = {The Eleventh International Conference on Learning Representations,
ICLR 2023, Kigali, Rwanda, May 1-5, 2023},
publisher = {OpenReview.net},
abstract = {Existing Graph Neural Networks (GNNs) compute the message exchange between nodes by either aggregating uniformly (convolving) the features of all the neighbor- ing nodes, or by applying a non-uniform score (attending) to the features. Recent works have shown the strengths and weaknesses of the resulting GNN architectures, respectively, GCNs and GATs. In this work, we aim at exploiting the strengths of both approaches to their full extent. To this end, we first introduce the graph convolutional attention layer (CAT), which relies on convolutions to compute the attention scores. Unfortunately, as in the case of GCNs and GATs, we show that there exists no clear winner between the three—neither theoretically nor in practice—as their performance directly depends on the nature of the data (i.e., of the graph and features). This result brings us to the main contribution of our work, the learnable graph convolutional attention network (L-CAT): a GNN architecture that automatically interpolates between GCN, GAT and CAT in each layer, by adding only two scalar parameters. Our results demonstrate that L-CAT is able to efficiently combine different GNN layers along the network, outperforming competing methods in a wide range of datasets, and resulting in a more robust model that reduces the need of cross-validating.},
keywords = {adrian, isabel, pablo},
pubstate = {published},
tppubtype = {inproceedings}
}
Existing Graph Neural Networks (GNNs) compute the message exchange between nodes by either aggregating uniformly (convolving) the features of all the neighbor- ing nodes, or by applying a non-uniform score (attending) to the features. Recent works have shown the strengths and weaknesses of the resulting GNN architectures, respectively, GCNs and GATs. In this work, we aim at exploiting the strengths of both approaches to their full extent. To this end, we first introduce the graph convolutional attention layer (CAT), which relies on convolutions to compute the attention scores. Unfortunately, as in the case of GCNs and GATs, we show that there exists no clear winner between the three—neither theoretically nor in practice—as their performance directly depends on the nature of the data (i.e., of the graph and features). This result brings us to the main contribution of our work, the learnable graph convolutional attention network (L-CAT): a GNN architecture that automatically interpolates between GCN, GAT and CAT in each layer, by adding only two scalar parameters. Our results demonstrate that L-CAT is able to efficiently combine different GNN layers along the network, outperforming competing methods in a wide range of datasets, and resulting in a more robust model that reduces the need of cross-validating.
Koyuncu, Batuhan; Sánchez-Mart'ın, Pablo; Peis, Ignacio; Olmos, Pablo M.; Valera, Isabel
Variational Mixture of HyperGenerators for Learning Distributions Over Functions Journal Article
In: CoRR, vol. abs/2302.06223, 2023.
Abstract | Links | BibTeX | Tags: batu, isabel, pablo
@article{DBLP:journals/corr/abs-2302-06223,
title = {Variational Mixture of HyperGenerators for Learning Distributions Over Functions},
author = {Batuhan Koyuncu and Pablo Sánchez-Mart'ın and Ignacio Peis and Pablo M. Olmos and Isabel Valera},
url = {https://doi.org/10.48550/arXiv.2302.06223},
doi = {10.48550/arXiv.2302.06223},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2302.06223},
abstract = {Recent approaches build on implicit neural representations (INRs) to propose generative models over function spaces. However, they are computationally costly when dealing with inference tasks, such as missing data imputation, or directly cannot tackle them. In this work, we propose a novel deep generative model, named VAMoH. VAMoH combines the capabilities of modeling continuous functions using INRs and the inference capabilities of Variational Autoencoders (VAEs). In addition, VAMoH relies on a normalizing flow to define the prior, and a mixture of hypernetworks to parametrize the data log-likelihood. This gives VAMoH a high expressive capability and interpretability. Through experiments on a diverse range of data types, such as images, voxels, and climate data, we show that VAMoH can effectively learn rich distributions over continuous functions. Furthermore, it can perform inference-related tasks, such as conditional super-resolution generation and in-painting, as well or better than previous approaches, while being less computationally demanding.},
keywords = {batu, isabel, pablo},
pubstate = {published},
tppubtype = {article}
}
Recent approaches build on implicit neural representations (INRs) to propose generative models over function spaces. However, they are computationally costly when dealing with inference tasks, such as missing data imputation, or directly cannot tackle them. In this work, we propose a novel deep generative model, named VAMoH. VAMoH combines the capabilities of modeling continuous functions using INRs and the inference capabilities of Variational Autoencoders (VAEs). In addition, VAMoH relies on a normalizing flow to define the prior, and a mixture of hypernetworks to parametrize the data log-likelihood. This gives VAMoH a high expressive capability and interpretability. Through experiments on a diverse range of data types, such as images, voxels, and climate data, we show that VAMoH can effectively learn rich distributions over continuous functions. Furthermore, it can perform inference-related tasks, such as conditional super-resolution generation and in-painting, as well or better than previous approaches, while being less computationally demanding.
2022
Sánchez-Mart'ın, Pablo; Rateike, Miriam; Valera, Isabel
VACA: Designing Variational Graph Autoencoders for Causal Queries Proceedings Article
In: Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Conference on Innovative Applications of Artificial Intelligence, IAAI 2022, The Twelveth Symposium on Educational Advances in Artificial Intelligence, EAAI 2022 Virtual Event, February 22 - March 1, 2022, pp. 8159–8168, AAAI Press, 2022.
Abstract | Links | BibTeX | Tags: isabel, miriam, pablo
@inproceedings{DBLP:conf/aaai/Sanchez-MartinR22b,
title = {VACA: Designing Variational Graph Autoencoders for Causal Queries},
author = {Pablo Sánchez-Mart'ın and Miriam Rateike and Isabel Valera},
url = {https://ojs.aaai.org/index.php/AAAI/article/view/20789},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI
2022, Thirty-Fourth Conference on Innovative Applications of Artificial
Intelligence, IAAI 2022, The Twelveth Symposium on Educational Advances
in Artificial Intelligence, EAAI 2022 Virtual Event, February 22
- March 1, 2022},
pages = {8159–8168},
publisher = {AAAI Press},
abstract = {In this paper, we introduce VACA, a novel class of variational graph autoencoders for causal inference in the absence of hidden confounders, when only observational data and the causal graph are available. Without making any parametric assumptions, VACA mimics the necessary properties of a Structural Causal Model (SCM) to provide a flexible and practical framework for approximating interventions (do-operator) and abduction-action-prediction steps. As a result, and as shown by our empirical results, VACA accurately approximates the interventional and counterfactual distributions on diverse SCMs. Finally, we apply VACA to evaluate counterfactual fairness in fair classification problems, as well as to learn fair classifiers without compromising performance.},
keywords = {isabel, miriam, pablo},
pubstate = {published},
tppubtype = {inproceedings}
}
In this paper, we introduce VACA, a novel class of variational graph autoencoders for causal inference in the absence of hidden confounders, when only observational data and the causal graph are available. Without making any parametric assumptions, VACA mimics the necessary properties of a Structural Causal Model (SCM) to provide a flexible and practical framework for approximating interventions (do-operator) and abduction-action-prediction steps. As a result, and as shown by our empirical results, VACA accurately approximates the interventional and counterfactual distributions on diverse SCMs. Finally, we apply VACA to evaluate counterfactual fairness in fair classification problems, as well as to learn fair classifiers without compromising performance.