Wavelet-Based Mechanistic Interpretability of Vision Transformers via Frequency-Aware Ablations
Authors:
- 11 June 2025
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 4830-4834
Abstract
We explore a wavelet-based interpretability framework for Vision Transformers (ViT), aiming to analyze their reliance on frequency-specific representations. Through systematic ablations of wavelet subbands, we assess how different frequency components contribute to latent representations and attention mechanisms. Our empirical study on CIFAR-10 reveals that high-frequency details, particularly those captured by Haar wavelets, influence reconstruction fidelity and attention distributions. While preliminary findings suggest a frequency-dependent behavior in ViT representations, further investigation is needed to generalize across datasets and architectures. This study highlights the potential of frequency-based interpretability but also underscores the need for more rigorous evaluation in larger, more diverse settings. To encourage further exploration, all the experimentation and method code can be found on our GitHub repository11https://github.com/sabraha2/Wavelet-Based-Mechanistic-Interpretability-of-Vision-Transformers-in-a-Latent-Diffusion-Setting.Keywords
Funding Information
- DEVCOM Army Research Laboratory (W911NF-20-2-0218)
This publication has 10 references indexed in Scilit:
- Efficient image generation with Contour Wavelet DiffusionComputers & Graphics, 2024
- WaveDM: Wavelet-Based Diffusion Models for Image RestorationIEEE Transactions on Multimedia, 2024
- Wavelet Diffusion Models are fast and scalable Image Generators2023
- A Survey of Visual TransformersIEEE Transactions on Neural Networks and Learning Systems, 2023
- Wave-ViT: Unifying Wavelet and Transformers for Visual Representation Learning2022
- Quantifying Attention Flow in Transformers2020
- ImageNet: A large-scale hierarchical image database2009
- Diffusion mapsApplied and Computational Harmonic Analysis, 2006
- Nonlinear multiscale wavelet diffusion for speckle suppression and edge enhancement in ultrasound imagesIEEE Transactions on Medical Imaging, 2006
- An introduction to waveletsIEEE Computational Science and Engineering, 1995