XimSwap: Many-to-Many Face Swapping for TinyML
Open Access
- 11 May 2024
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Embedded Computing Systems
- Vol. 23 (3) , 1-16
Abstract
The unprecedented development of deep learning approaches for video processing has caused growing privacy concerns. To ensure data analysis while maintaining privacy, it is essential to address how to protect individuals’ identities. One solution is to anonymize data at the source, avoiding the transmission or storage of information that could lead to identification. This study introduces XimSwap, a novel deep learning technique for real-time video anonymization, which can remove facial identification features directly on edge devices with minimal computational resources. Our approach offers a comprehensive solution that guarantees privacy by design. This novel method for implementing face-swapping ensures that the pose and expression of a target face remain unchanged and can be used on embedded devices with very limited computational resources. By incorporating style transfer layers into convolutional ones and optimizing the network’s operation, we achieved a reduction of over 98% in the required operations and parameters compared with state-of-the-art architectures. Our approach also significantly reduces RAM usage, making it possible to implement the anonymization process on tiny edge devices, including microcontrollers, such as the STM32H743.Keywords
Funding Information
- European Union’s Horizon 2020 Research and Innovation Program (957337)
- NextGenerationEU (PE00000013)
This publication has 29 references indexed in Scilit:
- GHOST—A New Face Swap Approach for Image and Video DomainsIEEE Access, 2022
- Graph-Based Generative Face Anonymisation with Pose Preservation2022
- Face re-identification challenge: Are face recognition models good enough?Pattern Recognition, 2020
- Face Inpainting via Nested Generative Adversarial NetworksIEEE Access, 2019
- DeepPrivacy: A Generative Adversarial Network for Face Anonymization2019
- An efficient privacy protection scheme for data security in video surveillanceJournal of Visual Communication and Image Representation, 2019
- A Hybrid Model for Identity Obfuscation by Face Replacement2018
- A Sub-mW IoT-Endnode for Always-On Visual Monitoring and Smart TriggeringIEEE Internet of Things Journal, 2017
- Pixelated image abstraction with integrated user constraintsComputers & Graphics, 2013
- Privacy by DesignIdentity in the Information Society, 2010