SV-SAE: Layer-Wise Pruning for Autoencoder Based on Link Contributions
Open Access
- 29 April 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 13, 75666-75678
Abstract
Autoencoders are a type of deep neural network and are widely used for unsupervised learning, particularly in tasks that require feature extraction and dimensionality reduction. While most research focuses on compressing input data, less attention has been given to reducing the size and complexity of the autoencoder model itself, which is crucial for deployment on resource-constrained edge devices. This paper introduces a layer-wise pruning algorithm specifically for multilayer perceptron-based autoencoder. The resulting pruned model is referred to as a Shapley Value-based Sparse AutoEncoder (SV-SAE). Using cooperative game theory, the proposed algorithm models the autoencoder as a coalition of interconnected units and links, where the Shapley value quantifies their individual contributions to overall performance. This enables the selective removal of less important components, achieving an optimal balance between sparsity and accuracy. Experimental results confirm that the SV-SAE reaches an accuracy of 99.25%, utilizing only 10% of the original links. Notably, the SV-SAE remains robust under high sparsity levels with minimal performance degradation whereas other algorithms experience sharp declines as the pruning ratio increases. Designed for edge environments, the SV-SAE offers an interpretable framework for controlling layer-wise sparsity while preserving essential features in latent representations. The results highlight its potential for efficient deployment in resource-constrained scenarios, where model size and inference speed are critical factors.Keywords
Funding Information
- Institute of Information and Communications Technology Planning and Evaluation
- Korean Government (MSIT) through the Development of Distributed/Cooperative AI-Based 5G+ Network Data Analytics Functions and Control Technology (RS-2021-II210739)
- Development of 6G Network Architecture and Industrial Convergence Standards Technologies and Standardization (RS-2024-00344830)
- Ewha Womans University Research Grant of 2025
This publication has 27 references indexed in Scilit:
- Autoencoders and their applications in machine learning: a surveyArtificial Intelligence Review, 2024
- A Dynamic Pruning Method on Multiple Sparse Structures in Deep Neural NetworksIEEE Access, 2023
- A comprehensive survey on design and application of autoencoder in deep learningApplied Soft Computing, 2023
- A brain-inspired algorithm for training highly sparse neural networksMachine Learning, 2022
- A Novel Approach to Uncertainty Quantification in Groundwater Table Modeling by Automated Predictive Deep LearningNatural Resources Research, 2022
- Roulette: A Pruning Framework to Train a Sparse Neural Network From ScratchIEEE Access, 2021
- Comparison of feature importance measures as explanations for classification modelsSN Applied Sciences, 2021
- From local explanations to global understanding with explainable AI for treesNature Machine Intelligence, 2020
- Auto-encoder based dimensionality reductionNeurocomputing, 2016
- A value for n-person games1988