Lightweight Optimal-Transport Harmonization on Edge Devices
Preprint
Abstract
Color harmonization adjusts the colors of an inserted object so that it perceptually matches the surrounding image, resulting in a seamless composite. The harmonization problem naturally arises in augmented reality (AR), yet harmonization algorithms are not currently integrated into AR pipelines because real-time solutions are scarce. In this work, we address color harmonization for AR by proposing a lightweight approach that supports on-device inference. For this, we leverage classical optimal transport theory by training a compact encoder to predict the Monge-Kantorovich transport map. We benchmark our MKL-Harmonizer algorithm against state-of-the-art methods and demonstrate that for real composite AR images our method achieves the best aggregated score. We release our dedicated AR dataset of composite images with pixel-accurate masks and data-gathering toolkit to support further data acquisition by researchers.Keywords
All Related Versions
This publication has 6 references indexed in Scilit:
- Dense Pixel-to-Pixel Harmonization via Continuous Image RepresentationIEEE Transactions on Circuits and Systems for Video Technology, 2023
- Harmonizer: Learning to Perform White-Box Image and Video Harmonization2022
- Example-based video color gradingACM Transactions on Graphics, 2013
- The linear Monge-Kantorovitch linear colour mapping for example-based colour transfer2007
- Color transfer between imagesIEEE Computer Graphics and Applications, 2001
- Environment Mapping and Other Applications of World ProjectionsIEEE Computer Graphics and Applications, 1986