Thomas Fel

Research Fellow at Kempner Institute, Harvard.

Ph.D. in Explainable AI.

Boston (US)

profile photo

About

Hi ! My name is Thomas, I am a Research fellow at the Kempner Institute, Harvard, working on Explainable AI. My main interest is using Explainable AI to better understand intelligence. I approach this through an interdisciplinary lens that combines computational science, mathematics, and neuroscience principles.

Previously, I was a Ph.D. student in the (awesome) Thomas Serre lab at Brown University, a core team member of DEEL (ANITI) and have interned at Google and GoPro.

News

Research

LENS Project
Thomas Fel, Thomas Serre

This tool is designed to offer a concise summary of the key concepts (termed as a 'dictionary of features') used by a large vision model across each of the 1,000 ImageNet classes. Each concept is accompanied by an importance score. This demonstration leverages insights from three of my prior projects, illustrating the potential synergy between attribution methods, concept analysis, and feature visualization.

Sparks of Explainability : Recent Advancements in Explaining Large Vision Models
Thomas Fel
Thesis

This doctoral thesis advances explainability in computer vision, developing tools to better understand the features used by deep neural networks. It explores attribution methods (saliency maps) and introduces new approaches like Sobol indices and EVA, which provide formal guarantees. The thesis finds current methods insufficient in complex scenarios and proposes aligning models with human reasoning and optimizing models using Human data (Harmonization project) or within constrained spaces (e.g., 1-Lipschitz functions). It further shifts focus from "where" models attend to "what" they perceive using concept-based explainability. The work culminates in a unified framework and interactive exploration for visualizing important concepts in models like ResNet.

Understanding Visual Feature Reliance through the Lens of Complexity
Thomas Fel*, Louis BĂ©thune*, Andrew Kyle Lampinen, Thomas Serre, Katherine Hermann
Proceedings of the Conference on Neural Information Processing Systems, NeurIPS 2024

This work introduces a metric based on 𝒱-information to quantify feature complexity in deep learning models. Analyzing 10,000 features from an ImageNet-trained model, our study finds a spectrum from simple to complex features, with simpler ones emerging early in training. Interestingly, simpler features are often more important for decision-making and tend to bypass the visual hierarchy via residual connections. Finally, we found that important features follow a Sedimentation process, becoming accessible early and building the foundation for the model’s learning.

CRAFT: Concept Recursive Activation FacTorization
Thomas Fel*, Agustin Picard*, Louis Béthune*, Thibaut Boissin*, David Vigouroux, Julien Colin, Rémi Cadène, Thomas Serre
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023

We proposed an automated method to extract concepts from neural networks that explains a specific class, tackling the "What" challenge in explainability. We go beyond understanding where the model looks, delving into what it sees at the precise location of interest. Taking it a step further, we enhance this method by (1) effectively estimating the importance of discovered concepts, and (2) generating localized heatmaps, revealing concept locations within each image.

A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimation
Thomas Fel*, Victor Boutin*, Mazda Moayeri, Rémi Cadène, Louis Béthune, Léo Andeol, Mathieu Chalvidal, Thomas Serre
Proceedings of the Conference on Neural Information Processing Systems, NeurIPS 2023 (Spotlight)

In this article we demonstrate that all concept extraction methods can be viewed as dictionary learning methods. We leverage this common framework to develop a comprehensive framework for comparing and improving concept extraction methods. Furthermore, we extensively investigate the estimation of concept importance and show that it is possible to determine optimal importance estimation formulas in certain cases. We also highlight the significance of local concept importance in addressing a crucial question in Explainable Artificial Intelligence (XAI): identifying data points classified based on similar reasons.

Unlocking Feature Visualization for Deeper Networks with Magnitude Constrained Optimization
Thomas Fel*, Thibaut Boissin*, Victor Boutin*, Agustin Picard*, Paul Novello*, Julien Colin, Drew Linsley, Tom Rousseau, Rémi Cadène, Laurent Gardes, Thomas Serre
Proceedings of the Conference on Neural Information Processing Systems, NeurIPS 2023

Since the remarkable work of Chris Olah and the Clarity team at OpenAI, feature visualization techniques have stagnated since 2017, and the methods proposed at the time are very difficult to make work on modern models (e.g. Vision Transformers). In this article, we propose a simple technique to revive feature visualization on modern models. Our method is based on a magnitude constraint, which ensures that the generated images have a magnitude similar to real images while avoiding the need for managing an additional hyperparameter.

Feature Accentuation
Chris Hamblin, Thomas Fel, Srijani Saha, Talia Konkle, George A. Alvarez
Under review

In this article, we introduce a novel interpretability tool called 'feature accentuation,' which addresses the need to understand both 'where' and 'what' neural network vision models recognize in arbitrary input images. By utilizing image-seeded feature visualization, we offer a comprehensive approach that provides naturalistic visualizations, shedding light on the spatial and semantic facets governing feature responses, ultimately enhancing our grasp of these complex models.

Harmonizing the object recognition strategies of deep neural networks with humans
Thomas Fel*, Ivan Felipe*, Drew Linsley*, Thomas Serre
Proceedings of the Conference on Neural Information Processing Systems, NeurIPS 2022

In this article, we begin by demonstrating the existence of a trade-off between alignment and accuracy: the more accurate models are on ImageNet (84 state-of-the-art models), the less they are aligned with humans in terms of saliency maps. To address this trade-off, we propose a training routine : the harmonization loss. This routine aligns deep neural networks (DNNs) with human visual strategies (almost down to the human-human leve), even improving categorization accuracy.

''What I Cannot Predict, I Do Not Understand''
A Human-Centered Evaluation Framework for Explainability Methods
Julien Colin*, Thomas Fel*, Rémi Cadène, Thomas Serre.
Proceedings of the Conference on Neural Information Processing Systems, NeurIPS 2022

In this work, we conducted psychophysics experiments at scale to evaluate the ability of human participants to leverage representative attribution methods for understanding the behavior of different image classifiers representing three real-world scenarios: (1) identifying bias, (2) characterizing novel strategy and (3) understanding failure cases. Our results demonstrate that the degree to which individual attribution methods help human participants better understand an AI system varied widely across these scenarios. This suggests a critical need for the field to move past quantitative improvements of current attribution methods.

On the explainable properties of 1-Lipschitz Neural Networks: An Optimal Transport Perspective
Mathieu Serrurier, Franck Mamalet, Thomas Fel, Louis BĂ©thune, Thibaut Boissin
Proceedings of the Conference on Neural Information Processing Systems, NeurIPS 2023

We argue that, when learning a 1-Lipschitz neural network with the dual loss of an optimal transportation problem, the gradient of the model is both the direction of the transportation plan and the direction to the closest adversarial attack. Traveling along the gradient to the decision boundary is no more an adversarial attack but becomes a counterfactual explanation, explicitly transporting from one class to the other. Through extensive experiments on XAI metrics, we find that the simple saliency map method, applied on such networks, becomes a reliable explanation! The proposed networks were already known to be certifiably robust, and we prove that they are also tailored for explainability.

Making Sense of Dependence: Efficient Black-box Explanations Using Dependence Measure
Paul Novello, Thomas Fel, David Vigouroux
Proceedings of the Conference on Neural Information Processing Systems, NeurIPS 2022

This paper could be seen as a follow-up to Look at the variance and further improves the efficiency of black-box methods using Hilbert-Schmidt Independence Criterion (HSIC). HSIC measures the dependence between regions of an input image and the output of a model based on kernel embeddings of distributions. Our experiments show that HSIC is up to 8 times faster than the previous best black-box attribution methods while being as faithful. Importantly, we show that these advances can be transposed to efficiently and faithfully explain object detection models such as YOLOv4.

Don't Lie to Me! Robust and Efficient Explainability with Verified Perturbation Analysis
Thomas Fel*, Melanie Ducoffe*, David Vigouroux*, Remi Cadene, Mikael Capelle, Claire Nicodeme, Thomas Serre
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023

In this work, we propose the first explainability method based on formal method and therefore able to meet certification requirements. Specifically, we introduce EVA (Explaining using Verified perturbation Analysis) -- the first explainability method guarantee to have an exhaustive exploration of a perturbation space. We leverage the beneficial properties of verified perturbation analysis -- time efficiency, tractability and guaranteed complete coverage of a manifold -- to efficiently characterize the input variables that are most likely to drive the model decision. We evaluate the approach systematically and demonstrate state-of-the-art results on multiple benchmarks.

Xplique: A Deep Learning Explainability Toolbox
Thomas Fel*, Lucas Hervier*, David Vigouroux, Antonin Poche, Justin Plakoo, Remi Cadene, Mathieu Chalvidal, Julien Colin, Thibaut Boissin, Louis Bethune, Agustin Picard, Claire Nicodeme, Laurent Gardes, Gregory Flandin, Thomas Serre
Workshop on Explainable Artificial Intelligence for Computer Vision, CVPRW 2022

For more than a year, I worked alongside my thesis to implement more than fifty explainability papers. Xplique is the result of this work, it is a library that I develop and maintain. The library is composed of several modules: (1) the Attributions Methods module, (2) The Feature Visualization module, (3) The Concepts module and (4) the Metrics module.

Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity Analysis
Thomas Fel*, Rémi Cadène*, Mathieu Chalvidal, Matthieu Cord, David Vigouroux, Thomas Serre.
Proceedings of the Conference on Neural Information Processing Systems, NeurIPS 2021

We describe a novel and efficient black-box attribution method which is grounded in Sensitivity Analysis and uses Sobol indices. Beyond modeling the individual contributions of image regions, Sobol indices provide an efficient way to capture higher-order interactions between image regions and their contributions to a neural network's prediction through the lens of variance.

How Good is your Explanation? Algorithmic Stability Measures to Assess the Quality of Explanations for Deep Neural Networks
Thomas Fel, David Vigouroux, Rémi Cadène, Thomas Serre.
Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2022

We propose two new measures to evaluate explanations borrowed from the field of algorithmic stability: mean generalizability MeGe and relative consistency ReCo. We conduct extensive experiments on different network architectures, common explainability methods, and several image datasets to demonstrate the benefits of the proposed metrics and show that popular fidelity measures are not sufficient to guarantee trustworthy explanations. Finally, we found that 1-Lipschitz networks produce explanations with higher MeGe and ReCo than common neural networks.