Improving Knowledge Distillation for Non-Intrusive Load Monitoring Through Explainability Guided Learning
- 4 June 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-5
Abstract
Knowledge distillation (KD) is a machine learning technique widely used in recent years for the task of domain adaptation and complexity reduction. It relies on a Student-Teacher mechanism to transfer the knowledge of a large and complex Teacher network into a smaller Student model. Given the inherent complexity of large Deep Neural Network (DNN) models, and the need for deployment on edge devices with limited resources, complexity reduction techniques have become a hot topic in the Non-intrusive Load Monitoring (NILM) community. Recent literature in NILM has devoted increased effort to domain adaptation and architecture reduction via KD. However, the mechanism behind the transfer of knowledge from the Teacher to the Student is not clearly understood. In this work, we aim to address the aforementioned issue by placing the KD NILM approach in a framework of explainable AI (XAI). We identify the main inconsistency in the transfer of explainable knowledge, and exploit this information to propose a method for improvement of KD through explainability guided learning. We evaluate our approach on a variety of appliances and domain adaptation scenarios and demonstrate that solving inconsistencies in the transfer of explainable knowledge can lead to improvement in predictive performance.Keywords
Funding Information
- European Commission
This publication has 17 references indexed in Scilit:
- Hierarchical distillation learning for scalable person searchPattern Recognition, 2021
- Review on Deep Neural Networks Applied to Low-Frequency NILMEnergies, 2021
- Lightweight Non-Intrusive Load Monitoring Employing Pruned Sequence-to-Point Learning2020
- EdgeNILM2020
- The global landscape of AI ethics guidelinesNature Machine Intelligence, 2019
- Unmasking Clever Hans predictors and assessing what machines really learnNature Communications, 2019
- European Union Regulations on Algorithmic Decision Making and a “Right to Explanation”AI Magazine, 2017
- An electrical load measurements dataset of United Kingdom households from a two-year longitudinal studyScientific Data, 2017
- The UK-DALE dataset, domestic appliance-level electricity demand and whole-house demand from five UK homesScientific Data, 2015
- Is disaggregation the holy grail of energy efficiency? The case of electricityEnergy Policy, 2013