Thomas Fel

Hey there! 👋

Thomas Fel here, a 2nd year French PhD student working on Explainable AI advised by Thomas Serre at ANITI (Toulouse, France) & Brown University (Boston, USA). I am also part of the incredible DEEL core team, relentlessly working towards making AI systems certifiable.

I'm interested in Explainability (XAI) for Vision, Robustness, Optimization and Certification of neural network systems. Most of my research involves reverse engineering existing neural networks to understand their critical strategies and contributing to open source. In particular, I am the author of Xplique .  /  Google Scholar  /  Twitter  /  Github

profile photo
  • 05/2023 : Invited talk at Harvard (Viégas & Wattenberg lab)
  • 04/2023 : Diffusion model vs Human paper accepted at ICML (Oral)!
  • 04/2023 : COCKATIEL accepted at ACL!
  • 03/2023 : 2 (first-author) papers (CRAFT, EVA) accepted at CVPR 🎉!
  • 03/2023 : Invited talk at AIRBUS
  • 02/2023 : Invited talk at ENS-Paris Saclay
  • 01/2023 : One-month road trip in Argentina 🦙

  • 12/2022 : Best thesis (both public & jury): "my thesis in 180 seconds" at SNCF
  • 12/2022 : Art & AI exhibition in Paris
  • 11/2022 : Invited talk at Brown DSCOV
  • 09/2022 : 3 papers (Harmonization, HSIC, Meta-predictor) accepted at NeurIPS 🎉!
  • 07/2022 : Oral presentation of EVA at ICML (Workshop)
  • 06/2022 : Presentation of Xplique at CVPR (Workshop)
  • 02/2022 : Presentation of the Sobol paper at the Mathematical Institute of Toulouse

  • 09/2021 : Sobol accepted at NeurIPS!
  • 04/2021 : Official start of the thesis
CRAFT: Concept Recursive Activation FacTorization
Thomas Fel*, Agustin Picard*, Louis Béthune*, Thibaut Boissin*,
David Vigouroux, Julien Colin, Rémi Cadène, Thomas Serre
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023

We proposed an automated method to extract concepts from neural networks that explains a specific class, tackling the "What" challenge in explainability. We go beyond understanding where the model looks, delving into what it sees at the precise location of interest. Taking it a step further, we enhance this method by (1) effectively estimating the importance of discovered concepts, and (2) generating localized heatmaps, revealing concept locations within each image.

A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimation
Thomas Fel*, Victor Boutin*, Mazda Moayeri, Rémi Cadène, Louis Béthune, Léo Andeol, Mathieu Chalvidal, Thomas Serre
Preprint (05/2023), Under review

In this article we demonstrate that all concept extraction methods can be viewed as dictionary learning methods. We leverage this common framework to develop a comprehensive framework for comparing and improving concept extraction methods. Furthermore, we extensively investigate the estimation of concept importance and show that it is possible to determine optimal importance estimation formulas in certain cases. We also highlight the significance of local concept importance in addressing a crucial question in Explainable Artificial Intelligence (XAI): identifying data points classified based on similar reasons.

Unlocking Feature Visualization for Deeper Networks with Magnitude Constrained Optimization
Thomas Fel*, Thibaut Boissin*, Victor Boutin*, Agustin Picard*, Paul Novello*, Julien Colin, Drew Linsley, Tom Rousseau, Rémi Cadène, Laurent Gardes, Thomas Serre
Preprint (05/2023), Under review

Since the remarkable work of Chris Olah and the Clarity team at OpenAI, feature visualization techniques have stagnated since 2017, and the methods proposed at the time are very difficult to make work on modern models (e.g. Vision Transformers). In this article, we propose a simple technique to revive feature visualization on modern models. Our method is based on a magnitude constraint, which ensures that the generated images have a magnitude similar to real images while avoiding the need for managing an additional hyperparameter.

Harmonizing the object recognition strategies of deep neural networks with humans
Thomas Fel*, Ivan Felipe*, Drew Linsley*, Thomas Serre
Proceedings of the Conference on Neural Information Processing Systems, NeurIPS 2022

In this article, we begin by demonstrating the existence of a trade-off between alignment and accuracy: the more accurate models are on ImageNet (84 state-of-the-art models), the less they are aligned with humans in terms of saliency maps. To address this trade-off, we propose a training routine : the harmonization loss. This routine aligns deep neural networks (DNNs) with human visual strategies (almost down to the human-human leve), even improving categorization accuracy.

''What I Cannot Predict, I Do Not Understand''
A Human-Centered Evaluation Framework for Explainability Methods
Julien Colin*, Thomas Fel*, Rémi Cadène, Thomas Serre.
Proceedings of the Conference on Neural Information Processing Systems, NeurIPS 2022

In this work, we conducted psychophysics experiments at scale to evaluate the ability of human participants to leverage representative attribution methods for understanding the behavior of different image classifiers representing three real-world scenarios: (1) identifying bias, (2) characterizing novel strategy and (3) understanding failure cases. Our results demonstrate that the degree to which individual attribution methods help human participants better understand an AI system varied widely across these scenarios. This suggests a critical need for the field to move past quantitative improvements of current attribution methods.

Making Sense of Dependence: Efficient Black-box Explanations Using Dependence Measure
Paul Novello, Thomas Fel, David Vigouroux
Proceedings of the Conference on Neural Information Processing Systems, NeurIPS 2022

This paper could be seen as a follow-up to Look at the variance and further improves the efficiency of black-box methods using Hilbert-Schmidt Independence Criterion (HSIC). HSIC measures the dependence between regions of an input image and the output of a model based on kernel embeddings of distributions. Our experiments show that HSIC is up to 8 times faster than the previous best black-box attribution methods while being as faithful. Importantly, we show that these advances can be transposed to efficiently and faithfully explain object detection models such as YOLOv4.

Don't Lie to Me! Robust and Efficient Explainability with Verified Perturbation Analysis
Thomas Fel*, Melanie Ducoffe*, David Vigouroux*, Remi Cadene, Mikael Capelle, Claire Nicodeme, Thomas Serre
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023

In this work, we propose the first explainability method based on formal method and therefore able to meet certification requirements. Specifically, we introduce EVA (Explaining using Verified perturbation Analysis) -- the first explainability method guarantee to have an exhaustive exploration of a perturbation space. We leverage the beneficial properties of verified perturbation analysis -- time efficiency, tractability and guaranteed complete coverage of a manifold -- to efficiently characterize the input variables that are most likely to drive the model decision. We evaluate the approach systematically and demonstrate state-of-the-art results on multiple benchmarks.

When adversarial attacks become interpretable counterfactual explanations
Mathieu Serrurier, Franck Mamalet, Thomas Fel, Louis Béthune, Thibaut Boissin
Under review, 2022

We argue that, when learning a 1-Lipschitz neural network with the dual loss of an optimal transportation problem, the gradient of the model is both the direction of the transportation plan and the direction to the closest adversarial attack. Traveling along the gradient to the decision boundary is no more an adversarial attack but becomes a counterfactual explanation, explicitly transporting from one class to the other. Through extensive experiments on XAI metrics, we find that the simple saliency map method, applied on such networks, becomes a reliable explanation! The proposed networks were already known to be certifiably robust, and we prove that they are also tailored for explainability.

Xplique: A Deep Learning Explainability Toolbox
Thomas Fel*, Lucas Hervier*, David Vigouroux, Antonin Poche, Justin Plakoo, Remi Cadene, Mathieu Chalvidal, Julien Colin, Thibaut Boissin, Louis Bethune, Agustin Picard, Claire Nicodeme, Laurent Gardes, Gregory Flandin, Thomas Serre
Workshop on Explainable Artificial Intelligence for Computer Vision, CVPR 2022

For more than a year, I worked alongside my thesis to implement more than fifty explainability papers. Xplique is the result of this work, it is a library that I develop and maintain. The library is composed of several modules: (1) the Attributions Methods module, (2) The Feature Visualization module, (3) The Concepts module and (4) the Metrics module.

Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity Analysis
Thomas Fel*, Rémi Cadène*, Mathieu Chalvidal, Matthieu Cord, David Vigouroux, Thomas Serre.
Proceedings of the Conference on Neural Information Processing Systems, NeurIPS 2021

We describe a novel and efficient black-box attribution method which is grounded in Sensitivity Analysis and uses Sobol indices. Beyond modeling the individual contributions of image regions, Sobol indices provide an efficient way to capture higher-order interactions between image regions and their contributions to a neural network's prediction through the lens of variance.

How Good is your Explanation? Algorithmic Stability Measures to Assess the Quality of Explanations for Deep Neural Networks
Thomas Fel, David Vigouroux, Rémi Cadène, Thomas Serre.
Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2022

We propose two new measures to evaluate explanations borrowed from the field of algorithmic stability: mean generalizability MeGe and relative consistency ReCo. We conduct extensive experiments on different network architectures, common explainability methods, and several image datasets to demonstrate the benefits of the proposed metrics and show that popular fidelity measures are not sufficient to guarantee trustworthy explanations. Finally, we found that 1-Lipschitz networks produce explanations with higher MeGe and ReCo than common neural networks.