Layer Sequence Extraction of Optimized DNNs Using Side-Channel Information Leaks
- 16 April 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
- Vol. 43 (10) , 3102-3115
Abstract
Deep neural network (DNN) intellectual property (IP) models must be kept undisclosed to avoid revealing trade secrets. Recent works have devised machine learning techniques that leverage on side-channel information leakage of the target platform to reverse engineer DNN architectures. However, these works fail to perform successful attacks on DNNs that have undergone performance optimizations (i.e., operator fusion) using DNN compilers, e.g., Apache tensor virtual machine (TVM). We propose a two-phase attack framework to infer the layer sequences of optimized DNNs through side-channel information leakage. In the first phase, we use a recurrent network with multihead attention components to learn the intra and interlayer fusion patterns from GPU traces of TVM-optimized DNNs, in order to accurately predict the operation distribution. The second phase uses a model to learn the run-time temporal correlations between operations and layers, which enables the prediction of layer sequence. An encoding strategy is proposed to overcome the convergence issues faced by existing learning-based methods when inferring the layer sequences of optimized DNNs. Extensive experiments show that our learning-based framework outperforms state-of-the-art DNN model extraction techniques. Our framework is also the first to effectively reverse engineer both convolutional neural networks (CNNs) and recurrent neural networks (RNNs) using side-channel leakage.Keywords
Funding Information
- NTU-DESAY SV Research Program (2018-0980)
- Ministry of Education, Singapore, under its Academic Research Fund Tier 2 (MOE-T2EP20121-0008)
This publication has 22 references indexed in Scilit:
- NeuroUnlock: Unlocking the Architecture of Obfuscated Deep Neural Networks2022
- NeurObfuscator: A Full-stack Obfuscation Tool to Mitigate Neural Architecture Stealing2021
- Swin Transformer: Hierarchical Vision Transformer using Shifted Windows2021
- DeepEM: Deep Neural Networks Model Recovery through EM Side-Channel Information Leakage2020
- Sparsity Turns Adversarial: Energy and Latency Attacks on Deep Neural NetworksIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2020
- Deep learning methods for forecasting COVID-19 time-Series data: A Comparative studyChaos, Solitons & Fractals, 2020
- Leaky DNN: Stealing Deep-Learning Model Secret with GPU Context-Switching Side-Channel2020
- Practical Attacks on Deep Neural Networks by Memory TrojaningIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2020
- Side Channel Attacks in Computation Offloading Systems with GPU Virtualization2019
- Reverse Engineering Convolutional Neural Networks Through Side-channel Information Leaks2018