People
Kavya Gupta
Saarland Informatics Campus
Building E1.1, Room 2.26
About me
I am a Postdoctoral researcher with Prof. Isabel Valera at Saarland University since March 2024. I recieved my Ph.D. from University Paris-Saclay in 2023 under the supervision of Prof. Jean-Christophe Pesquet. I am an alumna of IIITD, India. I am currently working on Society-Aware Machine Learning.
My research focuses on various aspects of understanding vulnerabilities in neural networks. I study the robustness of neural networks: understanding their behavior and providing formal guarantees on the robustness using Lipschitz bounds for making AI systems work in their safe set throughout their life cycle. I am focused on making AI systems secure and trustworthy with scalable and efficient methods. My research interests include robust machine learning, optimization, computer vision.
I am interested in research which leads to innovative as well as practical and real outcomes. Feel free to reach out on twitter or via email.
Publications
2026
Majumdar, Ayan; Kanubala, Deborah Dormah; Gupta, Kavya; Valera, Isabel
A Causal Framework to Measure and Mitigate Non-binary Treatment Discrimination Journal Article
In: CoRR, vol. abs/2503.22454, 2026.
@article{DBLP:journals/corr/abs-2503-22454,
title = {A Causal Framework to Measure and Mitigate Non-binary Treatment Discrimination},
author = {Ayan Majumdar and Deborah Dormah Kanubala and Kavya Gupta and Isabel Valera},
url = {https://doi.org/10.48550/arXiv.2503.22454},
doi = {10.48550/ARXIV.2503.22454},
year = {2026},
date = {2026-03-19},
urldate = {2026-03-19},
journal = {CoRR},
volume = {abs/2503.22454},
abstract = {Fairness studies of algorithmic decision-making systems often simplify complex decision processes, such as bail or lending decisions, into binary classification tasks (e.g., approve or not approve). However, these approaches overlook that such decisions are not inherently binary; they also involve non-binary treatment decisions (e.g., loan or bail terms) that can influence the downstream outcomes (e.g., loan repayment or reoffending). We argue that treatment decisions are integral to the decision-making process and, therefore, should be central to fairness analyses. Consequently, we propose a causal framework that extends and complements existing fairness notions by explicitly distinguishing between decision-subjects’ covariates and the treatment decisions. Our framework leverages path-specific counterfactual reasoning to: (i) measure treatment disparity and its downstream effects in historical data; and (ii) mitigate the impact of past unfair treatment decisions when automating decision-making. We use our framework to empirically analyze four widely used loan approval datasets to reveal potential disparity in non-binary treatment decisions and their discriminatory impact on outcomes, highlighting the need to incorporate treatment decisions in fairness assessments. Finally, by intervening in treatment decisions, we show that our framework effectively mitigates treatment discrimination from historical loan approval data to ensure fair risk score estimation and (non-binary) decision-making processes that benefit all stakeholders.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Fairness studies of algorithmic decision-making systems often simplify complex decision processes, such as bail or lending decisions, into binary classification tasks (e.g., approve or not approve). However, these approaches overlook that such decisions are not inherently binary; they also involve non-binary treatment decisions (e.g., loan or bail terms) that can influence the downstream outcomes (e.g., loan repayment or reoffending). We argue that treatment decisions are integral to the decision-making process and, therefore, should be central to fairness analyses. Consequently, we propose a causal framework that extends and complements existing fairness notions by explicitly distinguishing between decision-subjects’ covariates and the treatment decisions. Our framework leverages path-specific counterfactual reasoning to: (i) measure treatment disparity and its downstream effects in historical data; and (ii) mitigate the impact of past unfair treatment decisions when automating decision-making. We use our framework to empirically analyze four widely used loan approval datasets to reveal potential disparity in non-binary treatment decisions and their discriminatory impact on outcomes, highlighting the need to incorporate treatment decisions in fairness assessments. Finally, by intervening in treatment decisions, we show that our framework effectively mitigates treatment discrimination from historical loan approval data to ensure fair risk score estimation and (non-binary) decision-making processes that benefit all stakeholders.
2025
Kalampalikis, Nektarios; Gupta, Kavya; Vitanov, Georgi; Valera, Isabel
Towards Reasonable Concept Bottleneck Models Journal Article
In: CoRR, vol. abs/2506.05014, 2025.
@article{DBLP:journals/corr/abs-2506-05014,
title = {Towards Reasonable Concept Bottleneck Models},
author = {Nektarios Kalampalikis and Kavya Gupta and Georgi Vitanov and Isabel Valera},
url = {https://doi.org/10.48550/arXiv.2506.05014},
doi = {10.48550/ARXIV.2506.05014},
year = {2025},
date = {2025-01-01},
urldate = {2025-01-01},
journal = {CoRR},
volume = {abs/2506.05014},
abstract = {In this paper, we propose textbf{C}oncept textbf{REA}soning textbf{M}odels (CREAM), a novel family of Concept Bottleneck Models (CBMs) that: (i) explicitly encodes concept-concept ({texttt{C-C}}) and concept-task ({texttt{C$rightarrow$Y}}) relationships to enforce a desired model reasoning; and (ii) use a regularized side-channel to achieve competitive task performance, while keeping high concept importance. Specifically, CREAM architecturally embeds (bi)directed concept-concept, and concept to task relationships specified by a human expert, while severing undesired information flows (e.g., to handle mutually exclusive concepts). Moreover, CREAM integrates a black-box side-channel that is regularized to encourage task predictions to be grounded in the relevant concepts, thereby utilizing the side-channel only when necessary to enhance performance. Our experiments show that: (i) CREAM mainly relies on concepts while achieving task performance on par with black-box models; and (ii) the embedded {texttt{C-C}} and {texttt{C$rightarrow$Y}} relationships ease model interventions and mitigate concept leakage.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
In this paper, we propose textbf{C}oncept textbf{REA}soning textbf{M}odels (CREAM), a novel family of Concept Bottleneck Models (CBMs) that: (i) explicitly encodes concept-concept ({texttt{C-C}}) and concept-task ({texttt{C$rightarrow$Y}}) relationships to enforce a desired model reasoning; and (ii) use a regularized side-channel to achieve competitive task performance, while keeping high concept importance. Specifically, CREAM architecturally embeds (bi)directed concept-concept, and concept to task relationships specified by a human expert, while severing undesired information flows (e.g., to handle mutually exclusive concepts). Moreover, CREAM integrates a black-box side-channel that is regularized to encourage task predictions to be grounded in the relevant concepts, thereby utilizing the side-channel only when necessary to enhance performance. Our experiments show that: (i) CREAM mainly relies on concepts while achieving task performance on par with black-box models; and (ii) the embedded {texttt{C-C}} and {texttt{C$rightarrow$Y}} relationships ease model interventions and mitigate concept leakage.
2024
Kanubala, Deborah Dormah; Valera, Isabel; Gupta, Kavya
Fairness Beyond Binary Decisions: a Case Study on German Credit Proceedings Article
In: Cerrato, Mattia; Coronel, Alesia Vallenas; Ahrweiler, Petra; Loi, Michele; Pechenizkiy, Mykola; Tamò-Larrieux, Aurelia (Ed.): Proceedings of the 3rd European Workshop on Algorithmic Fairness, Mainz, Germany, July 1st to 3rd, 2024, CEUR-WS.org, 2024.
@inproceedings{DBLP:conf/ewaf/KanubalaVG24,
title = {Fairness Beyond Binary Decisions: a Case Study on German Credit},
author = {Deborah Dormah Kanubala and Isabel Valera and Kavya Gupta},
editor = {Mattia Cerrato and Alesia Vallenas Coronel and Petra Ahrweiler and Michele Loi and Mykola Pechenizkiy and Aurelia Tamò-Larrieux},
url = {https://ceur-ws.org/Vol-3908/paper_15.pdf},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
booktitle = {Proceedings of the 3rd European Workshop on Algorithmic Fairness,
Mainz, Germany, July 1st to 3rd, 2024},
volume = {3908},
publisher = {CEUR-WS.org},
series = {CEUR Workshop Proceedings},
abstract = {Data-driven approaches are increasingly used to (partially) automate decision-making in credit scoring
by predicting whether an applicant is “creditworthy or not” based on a set of features about the applicant,
such as age and income, along with what we refer here to as treatment decisions, e.g., loan amount and
duration. Existing data-driven approaches for automating and evaluating the accuracy and fairness of
such credit decisions ignore that treatment decisions (here, loan terms) are part of the decision and
thus may be subject to discrimination. This discrimination can propagate to the final outcome (repaid
or not) of positive decisions (granted loans). In this extended abstract, we rely on causal reasoning
and a broadly studied fair machine-learning dataset, the German credit, to i) show that the current fair
data-driven approach neglects discrimination in treatment decisions (i.e., loan terms) and its downstream
consequences on the decision outcome (i.e., ability to repay); and ii) argue for the need to move beyond
binary decisions in fair data-driven decision-making in consequential settings like credit scoring},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Data-driven approaches are increasingly used to (partially) automate decision-making in credit scoring
by predicting whether an applicant is “creditworthy or not” based on a set of features about the applicant,
such as age and income, along with what we refer here to as treatment decisions, e.g., loan amount and
duration. Existing data-driven approaches for automating and evaluating the accuracy and fairness of
such credit decisions ignore that treatment decisions (here, loan terms) are part of the decision and
thus may be subject to discrimination. This discrimination can propagate to the final outcome (repaid
or not) of positive decisions (granted loans). In this extended abstract, we rely on causal reasoning
and a broadly studied fair machine-learning dataset, the German credit, to i) show that the current fair
data-driven approach neglects discrimination in treatment decisions (i.e., loan terms) and its downstream
consequences on the decision outcome (i.e., ability to repay); and ii) argue for the need to move beyond
binary decisions in fair data-driven decision-making in consequential settings like credit scoring
by predicting whether an applicant is “creditworthy or not” based on a set of features about the applicant,
such as age and income, along with what we refer here to as treatment decisions, e.g., loan amount and
duration. Existing data-driven approaches for automating and evaluating the accuracy and fairness of
such credit decisions ignore that treatment decisions (here, loan terms) are part of the decision and
thus may be subject to discrimination. This discrimination can propagate to the final outcome (repaid
or not) of positive decisions (granted loans). In this extended abstract, we rely on causal reasoning
and a broadly studied fair machine-learning dataset, the German credit, to i) show that the current fair
data-driven approach neglects discrimination in treatment decisions (i.e., loan terms) and its downstream
consequences on the decision outcome (i.e., ability to repay); and ii) argue for the need to move beyond
binary decisions in fair data-driven decision-making in consequential settings like credit scoring
2023
Gupta, Kavya
University of Paris-Saclay, France, 2023.
@phdthesis{DBLP:phd/hal/Gupta23,
title = {Stability Quantification of Neural Networks. (Quantification de la stabilité des réseaux de neurones)},
author = {Kavya Gupta},
url = {https://tel.archives-ouvertes.fr/tel-04047901},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
school = {University of Paris-Saclay, France},
abstract = {Artificial neural networks are at the core of recent advances in Artificial Intelligence. One of the main challenges faced today, especially by companies likeThales designing advanced industrial systems is to ensure the safety of newgenerations of products using these technologies. In 2013 in a key observation, neural networks were shown to be sensitive to adversarial perturbations, raising serious concerns about their applicability in critically safe environments. In the last years, publications studying the various aspects of this robustness of neural networks, and rising questions such as "Why adversarial attacks occur?", "How can we make the neural network more robust to adversarial noise?", "How to generate stronger attacks?" etc., have grown exponentially. The contributions of this thesis aim to tackle such problems. The adversarial machine learning community concentrates majorly on classification scenarios, whereas studies on regression tasks are scarce. Our contributions bridge this significant gap between adversarial machine learning and regression applications.The first contribution in Chapter 3 proposes a white-box attackers designed to attack regression models. The presented adversarial attacker is derived from the algebraic properties of the Jacobian of the network. We show that our attacker successfully fools the neural network and measure its effectiveness in reducing the estimation performance. We present our results on various open-source and real industrial tabular datasets. Our analysis relies on the quantification of the fooling error as well as different error metrics. Another noteworthy feature of our attacker is that it allows us to optimally attack a subset of inputs, which may help to analyze the sensitivity of some specific inputs. We also, show the effect of this attacker on spectrally normalised trained models which are known to be more robust in handling attacks.The second contribution of this thesis (Chapter 4) presents a multivariate Lipschitz constant analysis of neural networks. The Lipschitz constant is widely used in the literature to study the internal properties of neural networks. But most works do a single parametric analysis, which do not allow to quantify the effect of individual inputs on the output. We propose a multivariate Lipschitz constant-based stability analysis of fully connected neural networks allowing us to capture the influence of each input or group of inputs on the neural network stability. Our approach relies on a suitable re-normalization of the input space, intending to perform a more precise analysis than the one provided by a global Lipschitz constant. We display the results of this analysis by a new representation designed for machine learning practitioners and safety engineers termed as a Lipschitz star. We perform experiments on various open-access tabular datasets and an actual Thales Air Mobility industrial application subject to certification requirements.The use of spectral normalization in designing a stability control loop is discussed in Chapter 5. A critical part of the optimal model is to behave according to specified performance and stability targets while in operation. But imposing tight Lipschitz constant constraints while training the models usually leads to a reduction of their accuracy. Hence, we design an algorithm to train "stable-by-design" neural network models using our spectral normalization approach, which optimizes the model by taking into account both performance and stability targets. We focus on Small Unmanned Aerial Vehicles (UAVs). More specifically, we present a novel application of neural networks to detect in real-time elevon positioning faults to allow the remote pilot to take necessary actions to ensure safety.},
keywords = {},
pubstate = {published},
tppubtype = {phdthesis}
}
Artificial neural networks are at the core of recent advances in Artificial Intelligence. One of the main challenges faced today, especially by companies likeThales designing advanced industrial systems is to ensure the safety of newgenerations of products using these technologies. In 2013 in a key observation, neural networks were shown to be sensitive to adversarial perturbations, raising serious concerns about their applicability in critically safe environments. In the last years, publications studying the various aspects of this robustness of neural networks, and rising questions such as "Why adversarial attacks occur?", "How can we make the neural network more robust to adversarial noise?", "How to generate stronger attacks?" etc., have grown exponentially. The contributions of this thesis aim to tackle such problems. The adversarial machine learning community concentrates majorly on classification scenarios, whereas studies on regression tasks are scarce. Our contributions bridge this significant gap between adversarial machine learning and regression applications.The first contribution in Chapter 3 proposes a white-box attackers designed to attack regression models. The presented adversarial attacker is derived from the algebraic properties of the Jacobian of the network. We show that our attacker successfully fools the neural network and measure its effectiveness in reducing the estimation performance. We present our results on various open-source and real industrial tabular datasets. Our analysis relies on the quantification of the fooling error as well as different error metrics. Another noteworthy feature of our attacker is that it allows us to optimally attack a subset of inputs, which may help to analyze the sensitivity of some specific inputs. We also, show the effect of this attacker on spectrally normalised trained models which are known to be more robust in handling attacks.The second contribution of this thesis (Chapter 4) presents a multivariate Lipschitz constant analysis of neural networks. The Lipschitz constant is widely used in the literature to study the internal properties of neural networks. But most works do a single parametric analysis, which do not allow to quantify the effect of individual inputs on the output. We propose a multivariate Lipschitz constant-based stability analysis of fully connected neural networks allowing us to capture the influence of each input or group of inputs on the neural network stability. Our approach relies on a suitable re-normalization of the input space, intending to perform a more precise analysis than the one provided by a global Lipschitz constant. We display the results of this analysis by a new representation designed for machine learning practitioners and safety engineers termed as a Lipschitz star. We perform experiments on various open-access tabular datasets and an actual Thales Air Mobility industrial application subject to certification requirements.The use of spectral normalization in designing a stability control loop is discussed in Chapter 5. A critical part of the optimal model is to behave according to specified performance and stability targets while in operation. But imposing tight Lipschitz constant constraints while training the models usually leads to a reduction of their accuracy. Hence, we design an algorithm to train "stable-by-design" neural network models using our spectral normalization approach, which optimizes the model by taking into account both performance and stability targets. We focus on Small Unmanned Aerial Vehicles (UAVs). More specifically, we present a novel application of neural networks to detect in real-time elevon positioning faults to allow the remote pilot to take necessary actions to ensure safety.
Gupta, Kavya; Verma, Sagar
CertViT: Certified Robustness of Pre-Trained Vision Transformers Journal Article
In: CoRR, vol. abs/2302.10287, 2023.
@article{DBLP:journals/corr/abs-2302-10287,
title = {CertViT: Certified Robustness of Pre-Trained Vision Transformers},
author = {Kavya Gupta and Sagar Verma},
url = {https://doi.org/10.48550/arXiv.2302.10287},
doi = {10.48550/ARXIV.2302.10287},
year = {2023},
date = {2023-01-01},
urldate = {2023-01-01},
journal = {CoRR},
volume = {abs/2302.10287},
abstract = {Lipschitz bounded neural networks are certifiably robust and have a good trade-off between clean and certified accuracy. Existing Lipschitz bounding methods train from scratch and are limited to moderately sized networks (< 6M parameters). They require a fair amount of hyper-parameter tuning and are computationally prohibitive for large networks like Vision Transformers (5M to 660M parameters). Obtaining certified robustness of transformers is not feasible due to the non-scalability and inflexibility of the current methods. This work presents CertViT, a two-step proximal-projection method to achieve certified robustness from pre-trained weights. The proximal step tries to lower the Lipschitz bound and the projection step tries to maintain the clean accuracy of pre-trained weights. We show that CertViT networks have better certified accuracy than state-of-the-art Lipschitz trained networks. We apply CertViT on several variants of pre-trained vision transformers and show adversarial robustness using standard attacks},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Lipschitz bounded neural networks are certifiably robust and have a good trade-off between clean and certified accuracy. Existing Lipschitz bounding methods train from scratch and are limited to moderately sized networks (< 6M parameters). They require a fair amount of hyper-parameter tuning and are computationally prohibitive for large networks like Vision Transformers (5M to 660M parameters). Obtaining certified robustness of transformers is not feasible due to the non-scalability and inflexibility of the current methods. This work presents CertViT, a two-step proximal-projection method to achieve certified robustness from pre-trained weights. The proximal step tries to lower the Lipschitz bound and the projection step tries to maintain the clean accuracy of pre-trained weights. We show that CertViT networks have better certified accuracy than state-of-the-art Lipschitz trained networks. We apply CertViT on several variants of pre-trained vision transformers and show adversarial robustness using standard attacks
2022
Gupta, Kavya; Kaakai, Fateh; Pesquet-Popescu, Béatrice; Pesquet, Jean-Christophe
Safe Design of Stable Neural Networks for Fault Detection in Small UAVs Proceedings Article
In: Trapp, Mario; Schoitsch, Erwin; Guiochet, Jérémie; Bitsch, Friedemann (Ed.): Computer Safety, Reliability, and Security. SAFECOMP 2022 Workshops - DECSoS, DepDevOps, SASSUR, SENSEI, USDAI, and WAISE, Munich, Germany, September 6-9, 2022, Proceedings, pp. 263–275, Springer, 2022.
@inproceedings{DBLP:conf/safecomp/GuptaKPP22,
title = {Safe Design of Stable Neural Networks for Fault Detection in Small UAVs},
author = {Kavya Gupta and Fateh Kaakai and Béatrice Pesquet-Popescu and Jean-Christophe Pesquet},
editor = {Mario Trapp and Erwin Schoitsch and Jérémie Guiochet and Friedemann Bitsch},
url = {https://doi.org/10.1007/978-3-031-14862-0_19},
doi = {10.1007/978-3-031-14862-0_19},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
booktitle = {Computer Safety, Reliability, and Security. SAFECOMP 2022 Workshops
- DECSoS, DepDevOps, SASSUR, SENSEI, USDAI, and WAISE, Munich, Germany,
September 6-9, 2022, Proceedings},
volume = {13415},
pages = {263–275},
publisher = {Springer},
series = {Lecture Notes in Computer Science},
abstract = {Stability of a machine learning model is the extent to which a model can continue to operate correctly despite small perturbations in its inputs. A formal method to measure stability is the Lipschitz constant of the model which allows to evaluate how small perturbations in the inputs impact the output variations. Variations in the outputs may lead to high errors for regression tasks or unintended changes in the classes for classification tasks. Verification of the stability of ML models is crucial in many industrial domains such as aeronautics, space, automotive etc. It has been recognized that data-driven models are intrinsically extremely sensitive to small perturbation of the inputs. Therefore, the need to design methods for verifying the stability of ML models is of importance for manufacturers developing safety critical products.
In this work, we focus on Small Unmanned Aerial Vehicles (UAVs) which are in the frontage of new technology solutions for intelligent systems. However, real-time fault detection/diagnosis in such UAVs remains a challenge from data collection to prediction tasks. This work presents application of neural networks to detect in real-time elevon positioning faults. We show the efficiency of a formal method based on the Lipschitz constant for quantifying the stability of neural network models. We also present how this method can be coupled with spectral normalization constraints at the design phase to control the internal parameters of the model and make it more stable while keeping a high level of performance (accuracy-stability trade-off).},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Stability of a machine learning model is the extent to which a model can continue to operate correctly despite small perturbations in its inputs. A formal method to measure stability is the Lipschitz constant of the model which allows to evaluate how small perturbations in the inputs impact the output variations. Variations in the outputs may lead to high errors for regression tasks or unintended changes in the classes for classification tasks. Verification of the stability of ML models is crucial in many industrial domains such as aeronautics, space, automotive etc. It has been recognized that data-driven models are intrinsically extremely sensitive to small perturbation of the inputs. Therefore, the need to design methods for verifying the stability of ML models is of importance for manufacturers developing safety critical products.
In this work, we focus on Small Unmanned Aerial Vehicles (UAVs) which are in the frontage of new technology solutions for intelligent systems. However, real-time fault detection/diagnosis in such UAVs remains a challenge from data collection to prediction tasks. This work presents application of neural networks to detect in real-time elevon positioning faults. We show the efficiency of a formal method based on the Lipschitz constant for quantifying the stability of neural network models. We also present how this method can be coupled with spectral normalization constraints at the design phase to control the internal parameters of the model and make it more stable while keeping a high level of performance (accuracy-stability trade-off).
In this work, we focus on Small Unmanned Aerial Vehicles (UAVs) which are in the frontage of new technology solutions for intelligent systems. However, real-time fault detection/diagnosis in such UAVs remains a challenge from data collection to prediction tasks. This work presents application of neural networks to detect in real-time elevon positioning faults. We show the efficiency of a formal method based on the Lipschitz constant for quantifying the stability of neural network models. We also present how this method can be coupled with spectral normalization constraints at the design phase to control the internal parameters of the model and make it more stable while keeping a high level of performance (accuracy-stability trade-off).
Gupta, Kavya; Kaakai, Fateh; Pesquet-Popescu, Beatrice; Pesquet, Jean-Christophe; Malliaros, Fragkiskos D.
Multivariate Lipschitz Analysis of the Stability of Neural Networks Journal Article
In: Frontiers in Signal Processing, vol. Volume 2 - 2022, 2022, ISSN: 2673-8198.
@article{10.3389/frsip.2022.794469,
title = {Multivariate Lipschitz Analysis of the Stability of Neural Networks},
author = {Kavya Gupta and Fateh Kaakai and Beatrice Pesquet-Popescu and Jean-Christophe Pesquet and Fragkiskos D. Malliaros},
url = {https://www.frontiersin.org/journals/signal-processing/articles/10.3389/frsip.2022.794469},
doi = {10.3389/frsip.2022.794469},
issn = {2673-8198},
year = {2022},
date = {2022-01-01},
urldate = {2022-01-01},
journal = {Frontiers in Signal Processing},
volume = {Volume 2 - 2022},
abstract = {The stability of neural networks with respect to adversarial perturbations has been extensively studied. One of the main strategies consist of quantifying the Lipschitz regularity of neural networks. In this paper, we introduce a multivariate Lipschitz constant-based stability analysis of fully connected neural networks allowing us to capture the influence of each input or group of inputs on the neural network stability. Our approach relies on a suitable re-normalization of the input space, with the objective to perform a more precise analysis than the one provided by a global Lipschitz constant. We investigate the mathematical properties of the proposed multivariate Lipschitz analysis and show its usefulness in better understanding the sensitivity of the neural network with regard to groups of inputs. We display the results of this analysis by a new representation designed for machine learning practitioners and safety engineers termed as a Lipschitz star. The Lipschitz star is a graphical and practical tool to analyze the sensitivity of a neural network model during its development, with regard to different combinations of inputs. By leveraging this tool, we show that it is possible to build robust-by-design models using spectral normalization techniques for controlling the stability of a neural network, given a safety Lipschitz target. Thanks to our multivariate Lipschitz analysis, we can also measure the efficiency of adversarial training in inference tasks. We perform experiments on various open access tabular datasets, and also on a real Thales Air Mobility industrial application subject to certification requirements.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The stability of neural networks with respect to adversarial perturbations has been extensively studied. One of the main strategies consist of quantifying the Lipschitz regularity of neural networks. In this paper, we introduce a multivariate Lipschitz constant-based stability analysis of fully connected neural networks allowing us to capture the influence of each input or group of inputs on the neural network stability. Our approach relies on a suitable re-normalization of the input space, with the objective to perform a more precise analysis than the one provided by a global Lipschitz constant. We investigate the mathematical properties of the proposed multivariate Lipschitz analysis and show its usefulness in better understanding the sensitivity of the neural network with regard to groups of inputs. We display the results of this analysis by a new representation designed for machine learning practitioners and safety engineers termed as a Lipschitz star. The Lipschitz star is a graphical and practical tool to analyze the sensitivity of a neural network model during its development, with regard to different combinations of inputs. By leveraging this tool, we show that it is possible to build robust-by-design models using spectral normalization techniques for controlling the stability of a neural network, given a safety Lipschitz target. Thanks to our multivariate Lipschitz analysis, we can also measure the efficiency of adversarial training in inference tasks. We perform experiments on various open access tabular datasets, and also on a real Thales Air Mobility industrial application subject to certification requirements.
2021
Lassau, Nathalie; Ammari, Samy; Chouzenoux, Emilie; Gortais, Hugo; Herent, Paul; Devilder, Matthieu; Soliman, Samer; Meyrignac, Olivier; Talabard, Marie-Pauline; Lamarque, Jean-Philippe; Dubois, Remy; Loiseau, Nicolas; Trichelair, Paul; Bendjebbar, Etienne; Garcia, Gabriel; Balleyguier, Corinne; Merad, Mansouria; Stoclin, Annabelle; Jegou, Simon; Griscelli, Franck; Tetelboum, Nicolas; Li, Yingping; Verma, Sagar; Terris, Matthieu; Dardouri, Tasnim; Gupta, Kavya; Neacsu, Ana; Chemouni, Frank; Sefta, Meriem; Jehanno, Paul; Bousaid, Imad; Boursin, Yannick; Planchet, Emmanuel; Azoulay, Mikael; Dachary, Jocelyn; Brulport, Fabien; Gonzalez, Adrian; Dehaene, Olivier; Schiratti, Jean-Baptiste; Schutte, Kathryn; Pesquet, Jean-Christophe; Talbot, Hugues; Pronier, Elodie; Wainrib, Gilles; Clozel, Thomas; Barlesi, Fabrice; Bellin, Marie-France; Blum, Michael G. B.
Integrating deep learning CT-scan model, biological and clinical variables to predict severity of COVID-19 patients Journal Article
In: Nat Commun, vol. 12, no. 1, 2021, ISSN: 2041-1723.
@article{Lassau2021,
title = {Integrating deep learning CT-scan model, biological and clinical variables to predict severity of COVID-19 patients},
author = {Nathalie Lassau and Samy Ammari and Emilie Chouzenoux and Hugo Gortais and Paul Herent and Matthieu Devilder and Samer Soliman and Olivier Meyrignac and Marie-Pauline Talabard and Jean-Philippe Lamarque and Remy Dubois and Nicolas Loiseau and Paul Trichelair and Etienne Bendjebbar and Gabriel Garcia and Corinne Balleyguier and Mansouria Merad and Annabelle Stoclin and Simon Jegou and Franck Griscelli and Nicolas Tetelboum and Yingping Li and Sagar Verma and Matthieu Terris and Tasnim Dardouri and Kavya Gupta and Ana Neacsu and Frank Chemouni and Meriem Sefta and Paul Jehanno and Imad Bousaid and Yannick Boursin and Emmanuel Planchet and Mikael Azoulay and Jocelyn Dachary and Fabien Brulport and Adrian Gonzalez and Olivier Dehaene and Jean-Baptiste Schiratti and Kathryn Schutte and Jean-Christophe Pesquet and Hugues Talbot and Elodie Pronier and Gilles Wainrib and Thomas Clozel and Fabrice Barlesi and Marie-France Bellin and Michael G. B. Blum},
doi = {10.1038/s41467-020-20657-4},
issn = {2041-1723},
year = {2021},
date = {2021-12-00},
urldate = {2021-12-00},
journal = {Nat Commun},
volume = {12},
number = {1},
publisher = {Springer Science and Business Media LLC},
abstract = {The SARS-COV-2 pandemic has put pressure on intensive care units, so that identifying predictors of disease severity is a priority. We collect 58 clinical and biological variables, and chest CT scan data, from 1003 coronavirus-infected patients from two French hospitals. We train a deep learning model based on CT scans to predict severity. We then construct the multimodal AI-severity score that includes 5 clinical and biological variables (age, sex, oxygenation, urea, platelet) in addition to the deep learning model. We show that neural network analysis of CT-scans brings unique prognosis information, although it is correlated with other markers of severity (oxygenation, LDH, and CRP) explaining the measurable but limited 0.03 increase of AUC obtained when adding CT-scan information to clinical variables. Here, we show that when comparing AI-severity with 11 existing severity scores, we find significantly improved prognosis performance; AI-severity can therefore rapidly become a reference scoring approach.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The SARS-COV-2 pandemic has put pressure on intensive care units, so that identifying predictors of disease severity is a priority. We collect 58 clinical and biological variables, and chest CT scan data, from 1003 coronavirus-infected patients from two French hospitals. We train a deep learning model based on CT scans to predict severity. We then construct the multimodal AI-severity score that includes 5 clinical and biological variables (age, sex, oxygenation, urea, platelet) in addition to the deep learning model. We show that neural network analysis of CT-scans brings unique prognosis information, although it is correlated with other markers of severity (oxygenation, LDH, and CRP) explaining the measurable but limited 0.03 increase of AUC obtained when adding CT-scan information to clinical variables. Here, we show that when comparing AI-severity with 11 existing severity scores, we find significantly improved prognosis performance; AI-severity can therefore rapidly become a reference scoring approach.
Gupta, Kavya; Pesquet-Popescu, Béatrice; Kaakai, Fateh; Pesquet, Jean-Christophe
A Quantitative Analysis Of The Robustness Of Neural Networks For Tabular Data Proceedings Article
In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2021, Toronto, ON, Canada, June 6-11, 2021, pp. 8057–8061, IEEE, 2021.
@inproceedings{DBLP:conf/icassp/GuptaPKP21,
title = {A Quantitative Analysis Of The Robustness Of Neural Networks For Tabular Data},
author = {Kavya Gupta and Béatrice Pesquet-Popescu and Fateh Kaakai and Jean-Christophe Pesquet},
url = {https://doi.org/10.1109/ICASSP39728.2021.9413858},
doi = {10.1109/ICASSP39728.2021.9413858},
year = {2021},
date = {2021-01-01},
urldate = {2021-01-01},
booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing,
ICASSP 2021, Toronto, ON, Canada, June 6-11, 2021},
pages = {8057–8061},
publisher = {IEEE},
abstract = {This paper presents a quantitative approach to demonstrate the robustness of neural networks for tabular data. These data form the backbone of the data structures found in most industrial applications. We analyse the effect of various widely used techniques we encounter in neural network practice, such as regularization of weights, addition of noise to the data, and positivity constraints. This analysis is performed by using three state-of-the-art techniques, which provide mathematical proofs of robustness in terms of Lipschitz constant for feed-forward networks. The experiments are carried out on two prediction tasks and one classification task. Our work brings insights into building robust neural network architectures for safety critical systems that require certification or approval from a competent authority.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
This paper presents a quantitative approach to demonstrate the robustness of neural networks for tabular data. These data form the backbone of the data structures found in most industrial applications. We analyse the effect of various widely used techniques we encounter in neural network practice, such as regularization of weights, addition of noise to the data, and positivity constraints. This analysis is performed by using three state-of-the-art techniques, which provide mathematical proofs of robustness in terms of Lipschitz constant for feed-forward networks. The experiments are carried out on two prediction tasks and one classification task. Our work brings insights into building robust neural network architectures for safety critical systems that require certification or approval from a competent authority.
Gupta, Kavya; Pesquet, Jean-Christophe; Pesquet-Popescu, Béatrice; Kaakai, Fateh; Malliaros, Fragkiskos D.
An Adversarial Attacker for Neural Networks in Regression Problems Proceedings Article
In: Espinoza, Huáscar; McDermid, John A.; Huang, Xiaowei; Castillo-Effen, Mauricio; Chen, Xin Cynthia; Hernández-Orallo, José; hÉigeartaigh, Seán Ó; Mallah, Richard; Pedroza, Gabriel (Ed.): Proceedings of the Workshop on Artificial Intelligence Safety 2021 co-located with the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI 2021), Virtual, August, 2021, CEUR-WS.org, 2021.
@inproceedings{DBLP:conf/ijcai/GuptaPPKM21,
title = {An Adversarial Attacker for Neural Networks in Regression Problems},
author = {Kavya Gupta and Jean-Christophe Pesquet and Béatrice Pesquet-Popescu and Fateh Kaakai and Fragkiskos D. Malliaros},
editor = {Huáscar Espinoza and John A. McDermid and Xiaowei Huang and Mauricio Castillo-Effen and Xin Cynthia Chen and José Hernández-Orallo and Seán Ó hÉigeartaigh and Richard Mallah and Gabriel Pedroza},
url = {https://ceur-ws.org/Vol-2916/paper_17.pdf},
year = {2021},
date = {2021-01-01},
urldate = {2021-01-01},
booktitle = {Proceedings of the Workshop on Artificial Intelligence Safety 2021
co-located with the Thirtieth International Joint Conference on Artificial
Intelligence (IJCAI 2021), Virtual, August, 2021},
volume = {2916},
publisher = {CEUR-WS.org},
series = {CEUR Workshop Proceedings},
abstract = {Adversarial attacks against neural networks and their defenses have been mostly investigated in classification scenarios. However, adversarial attacks in a regression setting remain understudied, although they play a critical role in a large portion of safety-critical applications. In this work, we present an adversarial attacker for regression tasks, derived from the algebraic properties of the Jacobian of the network. We show that our attacker successfully fools the neural network, and we measure its effectiveness in reducing the estimation performance. We present a white-box adversarial attacker to support engineers in designing safety-critical regression machine learning models. We present our results on various open-source and real industrial tabular datasets. In particular, the proposed adversarial attacker outperforms attackers based on random perturbations of the inputs. Our analysis relies on the quantification of the fooling error as well as various error metrics. A noteworthy feature of our attacker is that it allows us to optimally attack a subset of inputs, which may be helpful to analyse the sensitivity of some specific inputs.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Adversarial attacks against neural networks and their defenses have been mostly investigated in classification scenarios. However, adversarial attacks in a regression setting remain understudied, although they play a critical role in a large portion of safety-critical applications. In this work, we present an adversarial attacker for regression tasks, derived from the algebraic properties of the Jacobian of the network. We show that our attacker successfully fools the neural network, and we measure its effectiveness in reducing the estimation performance. We present a white-box adversarial attacker to support engineers in designing safety-critical regression machine learning models. We present our results on various open-source and real industrial tabular datasets. In particular, the proposed adversarial attacker outperforms attackers based on random perturbations of the inputs. Our analysis relies on the quantification of the fooling error as well as various error metrics. A noteworthy feature of our attacker is that it allows us to optimally attack a subset of inputs, which may be helpful to analyse the sensitivity of some specific inputs.
2020
Neacsu, Ana; Gupta, Kavya; Pesquet, Jean-Christophe; Burileanu, Corneliu
Signal Denoising Using a New Class of Robust Neural Networks Proceedings Article
In: 28th European Signal Processing Conference, EUSIPCO 2020, Amsterdam, Netherlands, January 18-21, 2021, pp. 1492–1496, IEEE, 2020.
@inproceedings{DBLP:conf/eusipco/NeacsuGPB20,
title = {Signal Denoising Using a New Class of Robust Neural Networks},
author = {Ana Neacsu and Kavya Gupta and Jean-Christophe Pesquet and Corneliu Burileanu},
url = {https://doi.org/10.23919/Eusipco47968.2020.9287630},
doi = {10.23919/EUSIPCO47968.2020.9287630},
year = {2020},
date = {2020-01-01},
urldate = {2020-01-01},
booktitle = {28th European Signal Processing Conference, EUSIPCO 2020, Amsterdam,
Netherlands, January 18-21, 2021},
pages = {1492–1496},
publisher = {IEEE},
abstract = {In this work, we propose a novel neural network architecture, called Adaptive Convolutional Neural Network (ACNN), which can be viewed as an intermediate solution between a standard convolutional network and a fully connected one. A constrained training strategy is developed to learn the parameters of such a network. The proposed algorithm allows us to control the Lipschitz constant of our ACNN to secure its robustness to adversarial noise. The resulting learning approach is evaluated for signal denoising based on a database of music recordings. Both qualitative and quantitative results show that the designed network is successful in removing Gaussian noise with unknown variance.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
In this work, we propose a novel neural network architecture, called Adaptive Convolutional Neural Network (ACNN), which can be viewed as an intermediate solution between a standard convolutional network and a fully connected one. A constrained training strategy is developed to learn the parameters of such a network. The proposed algorithm allows us to control the Lipschitz constant of our ACNN to secure its robustness to adversarial noise. The resulting learning approach is evaluated for signal denoising based on a database of music recordings. Both qualitative and quantitative results show that the designed network is successful in removing Gaussian noise with unknown variance.
Lassau, Nathalie; Ammari, Samy; Chouzenoux, Emilie; Gortais, Hugo; Herent, Paul; Devilder, Matthieu; Soliman, Samer; Meyrignac, Olivier; Talabard, Marie-Pauline; Lamarque, Jean-Philippe; Dubois, Remy; Loiseau, Nicolas; Trichelair, Paul; Bendjebbar, Etienne; Garcia, Gabriel; Balleyguier, Corinne; Merad, Mansouria; Stoclin, Annabelle; Jegou, Simon; Griscelli, Franck; Tetelboum, Nicolas; Li, Yingping; Verma, Sagar; Terris, Matthieu; Dardouri, Tasnim; Gupta, Kavya; Neacsu, Ana; Chemouni, Frank; Sefta, Meriem; Jehanno, Paul; Bousaid, Imad; Boursin, Yannick; Planchet, Emmanuel; Azoulay, Mikael; Dachary, Jocelyn; Brulport, Fabien; Gonzalez, Adrian; Dehaene, Olivier; Schiratti, Jean-Baptiste; Schutte, Kathryn; Pesquet, Jean-Christophe; Talbot, Hugues; Pronier, Elodie; Wainrib, Gilles; Clozel, Thomas; Barlesi, Fabrice; Bellin, Marie-France; Blum, Michael G. B.
Integration of clinical characteristics, lab tests and a deep learning CT scan analysis to predict severity of hospitalized COVID-19 patients Journal Article
In: medRxiv, 2020.
@article{Lassau2020.05.14.20101972,
title = {Integration of clinical characteristics, lab tests and a deep learning CT scan analysis to predict severity of hospitalized COVID-19 patients},
author = {Nathalie Lassau and Samy Ammari and Emilie Chouzenoux and Hugo Gortais and Paul Herent and Matthieu Devilder and Samer Soliman and Olivier Meyrignac and Marie-Pauline Talabard and Jean-Philippe Lamarque and Remy Dubois and Nicolas Loiseau and Paul Trichelair and Etienne Bendjebbar and Gabriel Garcia and Corinne Balleyguier and Mansouria Merad and Annabelle Stoclin and Simon Jegou and Franck Griscelli and Nicolas Tetelboum and Yingping Li and Sagar Verma and Matthieu Terris and Tasnim Dardouri and Kavya Gupta and Ana Neacsu and Frank Chemouni and Meriem Sefta and Paul Jehanno and Imad Bousaid and Yannick Boursin and Emmanuel Planchet and Mikael Azoulay and Jocelyn Dachary and Fabien Brulport and Adrian Gonzalez and Olivier Dehaene and Jean-Baptiste Schiratti and Kathryn Schutte and Jean-Christophe Pesquet and Hugues Talbot and Elodie Pronier and Gilles Wainrib and Thomas Clozel and Fabrice Barlesi and Marie-France Bellin and Michael G. B. Blum},
url = {https://www.medrxiv.org/content/early/2020/10/06/2020.05.14.20101972},
doi = {10.1101/2020.05.14.20101972},
year = {2020},
date = {2020-01-01},
urldate = {2020-01-01},
journal = {medRxiv},
publisher = {Cold Spring Harbor Laboratory Press},
abstract = {The SARS-COV-2 pandemic has put pressure on Intensive Care Units, so that identifying predictors of disease severity is a priority. We collected 58 clinical and biological variables, chest CT scan data (506,341 images), and radiology reports from 1,003 coronavirus-infected patients from two French hospitals. We trained a deep learning model based on CT scans to predict severity; this model was more discriminative than a radiologist quantification of disease extent. We showed that neural network analysis of CT-scan brings unique prognosis information, although it is correlated with other markers of severity (oxygenation, LDH, and CRP). To provide a multimodal severity score, we developed AI-severity that includes 5 clinical and biological variables (age, sex, oxygenation, urea, platelet) as well as the CT deep learning model. When comparing AI-severity with 11 existing scores for severity, we find significantly improved prognosis performance; AI-severity can therefore rapidly become a reference scoring approach.Competing Interest StatementThe authors declare the following competing interests: Employment: Michael Blum, Paul Herent, Rémy Dubois, Nicolas Loiseau, Paul Trichelair, Etienne Bendjebbar, Simon Jégou, Meriem Sefta, Paul Jehanno, Fabien Brulport, Olivier Dehaene, Jean-Baptiste Schiratti, Kathryn Schutte, Elodie Pronier, Jocelyn Dachary, Adrian Gonzalez, employed by Owkin Co-founders of Owkin Inc : Thomas Clozel, Gilles Wainrib. Funding StatementOwkin employees are paid by Owkin, Inc.Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:This study has received approval of ethic committees from the two hospitals and authors submitted a declaration to the National Commission of Data Processing and Liberties (Number INDS MR5413020420, CNIL) in order to get registered in the medical studies database and respect the General Regulation on Data Protection (RGPD) requirements.All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.YesThe dataset of patients hospitalized at Kremlin-Bicetre (KB) and Institut Gustave Roussy (IGR) are stored on a server at Institut Gustave Roussy (IGR). The data are available from the first author upon request subject to ethical review.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The SARS-COV-2 pandemic has put pressure on Intensive Care Units, so that identifying predictors of disease severity is a priority. We collected 58 clinical and biological variables, chest CT scan data (506,341 images), and radiology reports from 1,003 coronavirus-infected patients from two French hospitals. We trained a deep learning model based on CT scans to predict severity; this model was more discriminative than a radiologist quantification of disease extent. We showed that neural network analysis of CT-scan brings unique prognosis information, although it is correlated with other markers of severity (oxygenation, LDH, and CRP). To provide a multimodal severity score, we developed AI-severity that includes 5 clinical and biological variables (age, sex, oxygenation, urea, platelet) as well as the CT deep learning model. When comparing AI-severity with 11 existing scores for severity, we find significantly improved prognosis performance; AI-severity can therefore rapidly become a reference scoring approach.Competing Interest StatementThe authors declare the following competing interests: Employment: Michael Blum, Paul Herent, Rémy Dubois, Nicolas Loiseau, Paul Trichelair, Etienne Bendjebbar, Simon Jégou, Meriem Sefta, Paul Jehanno, Fabien Brulport, Olivier Dehaene, Jean-Baptiste Schiratti, Kathryn Schutte, Elodie Pronier, Jocelyn Dachary, Adrian Gonzalez, employed by Owkin Co-founders of Owkin Inc : Thomas Clozel, Gilles Wainrib. Funding StatementOwkin employees are paid by Owkin, Inc.Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:This study has received approval of ethic committees from the two hospitals and authors submitted a declaration to the National Commission of Data Processing and Liberties (Number INDS MR5413020420, CNIL) in order to get registered in the medical studies database and respect the General Regulation on Data Protection (RGPD) requirements.All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.YesThe dataset of patients hospitalized at Kremlin-Bicetre (KB) and Institut Gustave Roussy (IGR) are stored on a server at Institut Gustave Roussy (IGR). The data are available from the first author upon request subject to ethical review.
2019
Biswas, Sandika; Sinha, Sanjana; Gupta, Kavya; Bhowmick, Brojeshwar
Lifting 2d Human Pose to 3d : A Weakly Supervised Approach Miscellaneous
2019.
@misc{biswas2019lifting2dhumanpose,
title = {Lifting 2d Human Pose to 3d : A Weakly Supervised Approach},
author = {Sandika Biswas and Sanjana Sinha and Kavya Gupta and Brojeshwar Bhowmick},
url = {https://arxiv.org/abs/1905.01047},
year = {2019},
date = {2019-01-01},
urldate = {2019-01-01},
abstract = {Estimating 3d human pose from monocular images is a challenging problem due to the variety and complexity of human poses and the inherent ambiguity in recovering depth from the single view. Recent deep learning based methods show promising results by using supervised learning on 3d pose annotated datasets. However, the lack of large-scale 3d annotated training data captured under in-the-wild settings makes the 3d pose estimation difficult for in-the-wild poses. Few approaches have utilized training images from both 3d and 2d pose datasets in a weakly-supervised manner for learning 3d poses in unconstrained settings. In this paper, we propose a method which can effectively predict 3d human pose from 2d pose using a deep neural network trained in a weakly-supervised manner on a combination of ground-truth 3d pose and ground-truth 2d pose. Our method uses re-projection error minimization as a constraint to predict the 3d locations of body joints, and this is crucial for training on data where the 3d ground-truth is not present. Since minimizing re-projection error alone may not guarantee an accurate 3d pose, we also use additional geometric constraints on skeleton pose to regularize the pose in 3d. We demonstrate the superior generalization ability of our method by cross-dataset validation on a challenging 3d benchmark dataset MPI-INF-3DHP containing in the wild 3d poses.},
keywords = {},
pubstate = {published},
tppubtype = {misc}
}
Estimating 3d human pose from monocular images is a challenging problem due to the variety and complexity of human poses and the inherent ambiguity in recovering depth from the single view. Recent deep learning based methods show promising results by using supervised learning on 3d pose annotated datasets. However, the lack of large-scale 3d annotated training data captured under in-the-wild settings makes the 3d pose estimation difficult for in-the-wild poses. Few approaches have utilized training images from both 3d and 2d pose datasets in a weakly-supervised manner for learning 3d poses in unconstrained settings. In this paper, we propose a method which can effectively predict 3d human pose from 2d pose using a deep neural network trained in a weakly-supervised manner on a combination of ground-truth 3d pose and ground-truth 2d pose. Our method uses re-projection error minimization as a constraint to predict the 3d locations of body joints, and this is crucial for training on data where the 3d ground-truth is not present. Since minimizing re-projection error alone may not guarantee an accurate 3d pose, we also use additional geometric constraints on skeleton pose to regularize the pose in 3d. We demonstrate the superior generalization ability of our method by cross-dataset validation on a challenging 3d benchmark dataset MPI-INF-3DHP containing in the wild 3d poses.
2018
Chowdhury, Arijit; Kimbahune, Sanjay; Gupta, Kavya; Bhowmick, Brojeshwar; Mukhopadhyay, Shalini; B.S., Mithun; Shinde, Sujit; Bhavsar, Karan
Non-destructive method to detect artificially ripened banana using hyperspectral sensing and RGB imaging Bachelor Thesis
2018.
@bachelorthesis{Chowdhury2018,
title = {Non-destructive method to detect artificially ripened banana using hyperspectral sensing and RGB imaging},
author = {Arijit Chowdhury and Sanjay Kimbahune and Kavya Gupta and Brojeshwar Bhowmick and Shalini Mukhopadhyay and Mithun B.S. and Sujit Shinde and Karan Bhavsar},
editor = {Moon S. Kim and Byoung-Kwan Cho and Bryan A. Chin and Kuanglin Chao},
doi = {10.1117/12.2306367},
year = {2018},
date = {2018-05-15},
urldate = {2018-05-15},
publisher = {SPIE},
abstract = {Fruits provide essential nutrition in most natural form suitable for human beings. They are best when ripened naturally. However, industrialization has provided many ways for quick ripening and for extended shelf life of fruits. Detection of artificial ripening could be done by sophisticated methods like chemical analysis in lab or visual inspection by experts, which may not be feasible all the time. Of all the fruits, banana is the most consumed fruit around the world. Adulteration of banana can have devastating effects on masses on scale. It is figured, bananas are potentially ripened using carcinogens like Calcium Carbide(CaC2). In this paper, we propose and devise a novel and automatic method to classify the naturally and artificially ripened banana using spectral and RGB data. Our results show that using a Deep Learning (Neural Network) on RGB data, we achieve accuracy of up-to 90%.and using Random Forest and Multilayer Perceptron (MLP) feed forward Neural Network as classifiers on spectral data we can achieve accuracies of up-to 98.74% and 89.49% respectively.},
keywords = {},
pubstate = {published},
tppubtype = {bachelorthesis}
}
Fruits provide essential nutrition in most natural form suitable for human beings. They are best when ripened naturally. However, industrialization has provided many ways for quick ripening and for extended shelf life of fruits. Detection of artificial ripening could be done by sophisticated methods like chemical analysis in lab or visual inspection by experts, which may not be feasible all the time. Of all the fruits, banana is the most consumed fruit around the world. Adulteration of banana can have devastating effects on masses on scale. It is figured, bananas are potentially ripened using carcinogens like Calcium Carbide(CaC2). In this paper, we propose and devise a novel and automatic method to classify the naturally and artificially ripened banana using spectral and RGB data. Our results show that using a Deep Learning (Neural Network) on RGB data, we achieve accuracy of up-to 90%.and using Random Forest and Multilayer Perceptron (MLP) feed forward Neural Network as classifiers on spectral data we can achieve accuracies of up-to 98.74% and 89.49% respectively.
Gupta, Kavya; Majumdar, Angshul
Imposing Class-Wise Feature Similarity in Stacked Autoencoders by Nuclear Norm Regularization Journal Article
In: Neural Process. Lett., vol. 48, no. 1, pp. 615–629, 2018.
@article{DBLP:journals/npl/GuptaM18,
title = {Imposing Class-Wise Feature Similarity in Stacked Autoencoders by Nuclear Norm Regularization},
author = {Kavya Gupta and Angshul Majumdar},
url = {https://doi.org/10.1007/s11063-017-9731-2},
doi = {10.1007/S11063-017-9731-2},
year = {2018},
date = {2018-01-01},
urldate = {2018-01-01},
journal = {Neural Process. Lett.},
volume = {48},
number = {1},
pages = {615–629},
abstract = {This work proposes a new formulation for supervised stacked autoencoder. We argue that features from the same class should be similar to each other and hence linearly dependent. This means that, when stacked as columns, the feature matrix for each class will be rank deficient (low-rank). We impose this constraint into the stacked autoencoder formulation in the form of nuclear norm penalties on class-wise feature matrices at each level. The nuclear norm penalty is the convex surrogate of rank, and promotes a low-rank solution as desired by our proposal. Owing to the nuclear norm penalties, our formulation is non-smooth; hence cannot be solved using gradient descent based techniques like backpropagation directly. Moreover we learn the stacked autoencoder in one go, without the usual pre-training followed by fine-tuning regime. Both the ends (non-smooth cost function and single stage training for all the layers simultaneously) are met by employing the variable splitting followed by augmented Lagrangian method of alternating directions. Two sets of experiments have been carried out. The first set is on a variety of benchmark datasets. Our method excels over other deep learning models compared against—class sparse stacked autoencoder, deep belief network and discriminative deep belief network. The second experiment is on the brain computer classification problem; we find that our method outperforms prior deep learning based solutions utilized for this task.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
This work proposes a new formulation for supervised stacked autoencoder. We argue that features from the same class should be similar to each other and hence linearly dependent. This means that, when stacked as columns, the feature matrix for each class will be rank deficient (low-rank). We impose this constraint into the stacked autoencoder formulation in the form of nuclear norm penalties on class-wise feature matrices at each level. The nuclear norm penalty is the convex surrogate of rank, and promotes a low-rank solution as desired by our proposal. Owing to the nuclear norm penalties, our formulation is non-smooth; hence cannot be solved using gradient descent based techniques like backpropagation directly. Moreover we learn the stacked autoencoder in one go, without the usual pre-training followed by fine-tuning regime. Both the ends (non-smooth cost function and single stage training for all the layers simultaneously) are met by employing the variable splitting followed by augmented Lagrangian method of alternating directions. Two sets of experiments have been carried out. The first set is on a variety of benchmark datasets. Our method excels over other deep learning models compared against—class sparse stacked autoencoder, deep belief network and discriminative deep belief network. The second experiment is on the brain computer classification problem; we find that our method outperforms prior deep learning based solutions utilized for this task.
Gupta, Kavya; Bhowmick, Brojeshwar
Coupled Autoencoder Based Reconstruction of Images from Compressively Sampled Measurements Proceedings Article
In: 26th European Signal Processing Conference, EUSIPCO 2018, Roma, Italy, September 3-7, 2018, pp. 1067–1071, IEEE, 2018.
@inproceedings{DBLP:conf/eusipco/GuptaB18,
title = {Coupled Autoencoder Based Reconstruction of Images from Compressively Sampled Measurements},
author = {Kavya Gupta and Brojeshwar Bhowmick},
url = {https://doi.org/10.23919/EUSIPCO.2018.8553263},
doi = {10.23919/EUSIPCO.2018.8553263},
year = {2018},
date = {2018-01-01},
urldate = {2018-01-01},
booktitle = {26th European Signal Processing Conference, EUSIPCO 2018, Roma,
Italy, September 3-7, 2018},
pages = {1067–1071},
publisher = {IEEE},
abstract = {This work addresses the problem of reconstructing images from their lower dimensional random projections using Coupled Autoencoder (CAE). Traditionally, Compressed Sensing (CS) based techniques have been employed for this task. CS based techniques are iterative in nature; hence inversion process is time-consuming and cannot be deployed for the real-time reconstruction process. These inversion processes are transductive in nature. With the recent development in deep learning - auto encoders, CNN based architectures have been used for learning inversion in an inductive setup. The training period for inductive learning is large but is very fast during application. But these approaches work only on the signal domain and not on the measurement domain. We show the application of CAE, which can work directly from the measurement domain. We compare CAE with a Dictionary learning based coupling setup and a recently proposed CNN based CS reconstruction algorithm. We show reconstruction capability of CAE in terms of PSNR and SSIM on a standard set of images with measurement rates of 0.04 and 0.25.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
This work addresses the problem of reconstructing images from their lower dimensional random projections using Coupled Autoencoder (CAE). Traditionally, Compressed Sensing (CS) based techniques have been employed for this task. CS based techniques are iterative in nature; hence inversion process is time-consuming and cannot be deployed for the real-time reconstruction process. These inversion processes are transductive in nature. With the recent development in deep learning - auto encoders, CNN based architectures have been used for learning inversion in an inductive setup. The training period for inductive learning is large but is very fast during application. But these approaches work only on the signal domain and not on the measurement domain. We show the application of CAE, which can work directly from the measurement domain. We compare CAE with a Dictionary learning based coupling setup and a recently proposed CNN based CS reconstruction algorithm. We show reconstruction capability of CAE in terms of PSNR and SSIM on a standard set of images with measurement rates of 0.04 and 0.25.
Gupta, Kavya; Bhowmick, Brojeshwar; Majumdar, Angshul
Coupled Analysis Dictionary Learning to inductively learn inversion: Application to real-time reconstruction of Biomedical signals Proceedings Article
In: 2018 International Joint Conference on Neural Networks, IJCNN 2018, Rio de Janeiro, Brazil, July 8-13, 2018, pp. 1–8, IEEE, 2018.
@inproceedings{DBLP:conf/ijcnn/GuptaBM18,
title = {Coupled Analysis Dictionary Learning to inductively learn inversion: Application to real-time reconstruction of Biomedical signals},
author = {Kavya Gupta and Brojeshwar Bhowmick and Angshul Majumdar},
url = {https://doi.org/10.1109/IJCNN.2018.8489148},
doi = {10.1109/IJCNN.2018.8489148},
year = {2018},
date = {2018-01-01},
urldate = {2018-01-01},
booktitle = {2018 International Joint Conference on Neural Networks, IJCNN 2018,
Rio de Janeiro, Brazil, July 8-13, 2018},
pages = {1–8},
publisher = {IEEE},
abstract = {This work addresses the problem of reconstructing biomedical signals from their lower dimensional projections. Traditionally Compressed Sensing (CS) based techniques have been employed for this task. These are transductive inversion processes; the problem with these approaches is that the inversion is time-consuming and hence not suitable for real-time applications. With the recent advent of deep learning, Stacked Sparse Denoising Autoencoder (SSDAE) has been used for learning inversion in an inductive setup. The training period for inductive learning is large but is very fast during application - capable of real-time speed. This work proposes a new approach for inductive learning of the inversion process. It is based on Coupled Analysis Dictionary Learning. Results on Biomedical signal reconstruction show that our proposed approach is very fast and yields result far better than CS and SSDAE.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
This work addresses the problem of reconstructing biomedical signals from their lower dimensional projections. Traditionally Compressed Sensing (CS) based techniques have been employed for this task. These are transductive inversion processes; the problem with these approaches is that the inversion is time-consuming and hence not suitable for real-time applications. With the recent advent of deep learning, Stacked Sparse Denoising Autoencoder (SSDAE) has been used for learning inversion in an inductive setup. The training period for inductive learning is large but is very fast during application - capable of real-time speed. This work proposes a new approach for inductive learning of the inversion process. It is based on Coupled Analysis Dictionary Learning. Results on Biomedical signal reconstruction show that our proposed approach is very fast and yields result far better than CS and SSDAE.
Gupta, Kavya; Bhowmick, Brojeshwar; Majumdar, Angshul
Motion Blur removal via Coupled Autoencoder Journal Article
In: CoRR, vol. abs/1812.09888, 2018.
@article{DBLP:journals/corr/abs-1812-09888,
title = {Motion Blur removal via Coupled Autoencoder},
author = {Kavya Gupta and Brojeshwar Bhowmick and Angshul Majumdar},
url = {http://arxiv.org/abs/1812.09888},
year = {2018},
date = {2018-01-01},
urldate = {2018-01-01},
journal = {CoRR},
volume = {abs/1812.09888},
abstract = {In this paper a joint optimization technique has been proposed for coupled autoencoder which learns the autoencoder weights and coupling map (between source and target) simultaneously. The technique is applicable to any transfer learning problem. In this work, we propose a new formulation that recasts deblurring as a transfer learning problem, it is solved using the proposed coupled autoencoder. The proposed technique can operate on-the-fly, since it does not require solving any costly inverse problem. Experiments have been carried out on state-of-the-art techniques, our method yields better quality images in shorter operating times.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
In this paper a joint optimization technique has been proposed for coupled autoencoder which learns the autoencoder weights and coupling map (between source and target) simultaneously. The technique is applicable to any transfer learning problem. In this work, we propose a new formulation that recasts deblurring as a transfer learning problem, it is solved using the proposed coupled autoencoder. The proposed technique can operate on-the-fly, since it does not require solving any costly inverse problem. Experiments have been carried out on state-of-the-art techniques, our method yields better quality images in shorter operating times.
2017
Gupta, Kavya; Majumdar, Angshul
Learning autoencoders with low-rank weights Proceedings Article
In: 2017 IEEE International Conference on Image Processing, ICIP 2017, Beijing, China, September 17-20, 2017, pp. 3899–3903, IEEE, 2017.
@inproceedings{DBLP:conf/icip/GuptaM17,
title = {Learning autoencoders with low-rank weights},
author = {Kavya Gupta and Angshul Majumdar},
url = {https://doi.org/10.1109/ICIP.2017.8297013},
doi = {10.1109/ICIP.2017.8297013},
year = {2017},
date = {2017-01-01},
urldate = {2017-01-01},
booktitle = {2017 IEEE International Conference on Image Processing, ICIP 2017,
Beijing, China, September 17-20, 2017},
pages = {3899–3903},
publisher = {IEEE},
abstract = {In this work we propose to regularize the encoding and decoding weights of an autoencoder using low-rank penalty in the form of nuclear norm. Such a formulation models redundancy in the network. We show that our proposed method yields better classification accuracy (on an average) and denoising results than other stochastic and deterministic regularization techniques used in deep autoencoders. Our method is also considerably faster compared to these techniques. The experiments have been carried out on benchmark deep learning datasets.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
In this work we propose to regularize the encoding and decoding weights of an autoencoder using low-rank penalty in the form of nuclear norm. Such a formulation models redundancy in the network. We show that our proposed method yields better classification accuracy (on an average) and denoising results than other stochastic and deterministic regularization techniques used in deep autoencoders. Our method is also considerably faster compared to these techniques. The experiments have been carried out on benchmark deep learning datasets.
2016
Gupta, Kavya; Raj, Ankita; Majumdar, Angshul
Analysis and Synthesis Prior Greedy Algorithms for Non-linear Sparse Recovery Proceedings Article
In: Bilgin, Ali; Marcellin, Michael W.; Serra-Sagristà, Joan; Storer, James A. (Ed.): 2016 Data Compression Conference, DCC 2016, Snowbird, UT, USA, March 30 - April 1, 2016, pp. 599, IEEE, 2016.
@inproceedings{DBLP:conf/dcc/GuptaRM16,
title = {Analysis and Synthesis Prior Greedy Algorithms for Non-linear Sparse Recovery},
author = {Kavya Gupta and Ankita Raj and Angshul Majumdar},
editor = {Ali Bilgin and Michael W. Marcellin and Joan Serra-Sagristà and James A. Storer},
url = {https://doi.org/10.1109/DCC.2016.36},
doi = {10.1109/DCC.2016.36},
year = {2016},
date = {2016-01-01},
urldate = {2016-01-01},
booktitle = {2016 Data Compression Conference, DCC 2016, Snowbird, UT, USA, March
30 - April 1, 2016},
pages = {599},
publisher = {IEEE},
abstract = {In this work we address the problem of recovering sparse solutions to non-linear inverse problems. We look at two variants of the basic problem - the synthesis prior problem when the solution is sparse and the analysis prior problem where the solution is co-sparse in some linear basis. For the first problem, we propose non-linear variants of the Orthogonal Matching Pursuit (OMP) and CoSamp algorithms, for the second problem we propose a non-linear variant of the Greedy Analysis Pursuit (GAP) algorithm. We empirically test the success rates of our algorithms on exponential and logarithmic functions.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
In this work we address the problem of recovering sparse solutions to non-linear inverse problems. We look at two variants of the basic problem - the synthesis prior problem when the solution is sparse and the analysis prior problem where the solution is co-sparse in some linear basis. For the first problem, we propose non-linear variants of the Orthogonal Matching Pursuit (OMP) and CoSamp algorithms, for the second problem we propose a non-linear variant of the Greedy Analysis Pursuit (GAP) algorithm. We empirically test the success rates of our algorithms on exponential and logarithmic functions.
Mehta, Janki; Gupta, Kavya; Gogna, Anupriya; Majumdar, Angshul; Anand, Saket
Stacked Robust Autoencoder for Classification Proceedings Article
In: Hirose, Akira; Ozawa, Seiichi; Doya, Kenji; Ikeda, Kazushi; Lee, Minho; Liu, Derong (Ed.): Neural Information Processing - 23rd International Conference, ICONIP 2016, Kyoto, Japan, October 16-21, 2016, Proceedings, Part III, pp. 600–607, 2016.
@inproceedings{DBLP:conf/iconip/MehtaGGMA16,
title = {Stacked Robust Autoencoder for Classification},
author = {Janki Mehta and Kavya Gupta and Anupriya Gogna and Angshul Majumdar and Saket Anand},
editor = {Akira Hirose and Seiichi Ozawa and Kenji Doya and Kazushi Ikeda and Minho Lee and Derong Liu},
url = {https://doi.org/10.1007/978-3-319-46675-0_66},
doi = {10.1007/978-3-319-46675-0_66},
year = {2016},
date = {2016-01-01},
urldate = {2016-01-01},
booktitle = {Neural Information Processing - 23rd International Conference, ICONIP
2016, Kyoto, Japan, October 16-21, 2016, Proceedings, Part III},
volume = {9949},
pages = {600–607},
series = {Lecture Notes in Computer Science},
abstract = {In this work we propose an l p -norm data fidelity constraint for training the autoencoder. Usually the Euclidean distance is used for this purpose; we generalize the l 2 -norm to the l p -norm; smaller values of p make the problem robust to outliers. The ensuing optimization problem is solved using the Augmented Lagrangian approach. The proposed l p -norm Autoencoder has been tested on benchmark deep learning datasets – MNIST, CIFAR-10 and SVHN. We have seen that the proposed robust autoencoder yields better results than the standard autoencoder (l 2 -norm) and deep belief network for all of these problems.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
In this work we propose an l p -norm data fidelity constraint for training the autoencoder. Usually the Euclidean distance is used for this purpose; we generalize the l 2 -norm to the l p -norm; smaller values of p make the problem robust to outliers. The ensuing optimization problem is solved using the Augmented Lagrangian approach. The proposed l p -norm Autoencoder has been tested on benchmark deep learning datasets – MNIST, CIFAR-10 and SVHN. We have seen that the proposed robust autoencoder yields better results than the standard autoencoder (l 2 -norm) and deep belief network for all of these problems.
Gupta, Kavya; Majumdar, Angshul
Sparsely connected autoencoder Proceedings Article
In: 2016 International Joint Conference on Neural Networks, IJCNN 2016, Vancouver, BC, Canada, July 24-29, 2016, pp. 1940–1947, IEEE, 2016.
@inproceedings{DBLP:conf/ijcnn/GuptaM16,
title = {Sparsely connected autoencoder},
author = {Kavya Gupta and Angshul Majumdar},
url = {https://doi.org/10.1109/IJCNN.2016.7727437},
doi = {10.1109/IJCNN.2016.7727437},
year = {2016},
date = {2016-01-01},
urldate = {2016-01-01},
booktitle = {2016 International Joint Conference on Neural Networks, IJCNN 2016,
Vancouver, BC, Canada, July 24-29, 2016},
pages = {1940–1947},
publisher = {IEEE},
abstract = {This work proposes to learn autoencoders with sparse connections. Prior studies on autoencoders enforced sparsity on the neuronal activity; these are different from our proposed approach - we learn sparse connections. Sparsity in connections helps in learning (and keeping) the important relations while trimming the irrelevant ones. We have tested the performance of our proposed method on two tasks - classification and denoising. For classification we have compared against stacked autneencoders, contractive autoencoders, deep belief network, sparse deep neural network and optimal brain damage neural network; the denoising performance was compared against denoising autoencoder and sparse (activity) autoencoder. In both the tasks our proposed method yields superior results.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
This work proposes to learn autoencoders with sparse connections. Prior studies on autoencoders enforced sparsity on the neuronal activity; these are different from our proposed approach - we learn sparse connections. Sparsity in connections helps in learning (and keeping) the important relations while trimming the irrelevant ones. We have tested the performance of our proposed method on two tasks - classification and denoising. For classification we have compared against stacked autneencoders, contractive autoencoders, deep belief network, sparse deep neural network and optimal brain damage neural network; the denoising performance was compared against denoising autoencoder and sparse (activity) autoencoder. In both the tasks our proposed method yields superior results.
2015
Gupta, Kavya; Raj, Ankita; Majumdar, Angshul
Analysis and Synthesis Prior Greedy Algorithms for Non-linear Sparse Recovery Journal Article
In: CoRR, vol. abs/1512.07709, 2015.
@article{DBLP:journals/corr/GuptaRM15,
title = {Analysis and Synthesis Prior Greedy Algorithms for Non-linear Sparse Recovery},
author = {Kavya Gupta and Ankita Raj and Angshul Majumdar},
url = {http://arxiv.org/abs/1512.07709},
year = {2015},
date = {2015-01-01},
urldate = {2015-01-01},
journal = {CoRR},
volume = {abs/1512.07709},
abstract = {In this work we address the problem of recovering sparse solutions to non linear inverse problems. We look at two variants of the basic problem, the synthesis prior problem when the solution is sparse and the analysis prior problem where the solution is cosparse in some linear basis. For the first problem, we propose non linear variants of the Orthogonal Matching Pursuit (OMP) and CoSamp algorithms; for the second problem we propose a non linear variant of the Greedy Analysis Pursuit (GAP) algorithm. We empirically test the success rates of our algorithms on exponential and logarithmic functions. We model speckle denoising as a non linear sparse recovery problem and apply our technique to solve it. Results show that our method outperforms state of the art methods in ultrasound speckle denoising.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
In this work we address the problem of recovering sparse solutions to non linear inverse problems. We look at two variants of the basic problem, the synthesis prior problem when the solution is sparse and the analysis prior problem where the solution is cosparse in some linear basis. For the first problem, we propose non linear variants of the Orthogonal Matching Pursuit (OMP) and CoSamp algorithms; for the second problem we propose a non linear variant of the Greedy Analysis Pursuit (GAP) algorithm. We empirically test the success rates of our algorithms on exponential and logarithmic functions. We model speckle denoising as a non linear sparse recovery problem and apply our technique to solve it. Results show that our method outperforms state of the art methods in ultrasound speckle denoising.