Semantically Robust Unpaired Image Translation for Data with Unmatched Semantics Statistics
Authors:
Open Access
- 1 October 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 14253-14263
Abstract
Many applications of unpaired image-to-image translation require the input contents to be preserved semantically during translations. Unaware of the inherently unmatched semantics distributions between source and target domains, existing distribution matching methods (i.e., GAN-based) can give undesired solutions. In particular, although producing visually reasonable outputs, the learned models usually flip the semantics of the inputs. To tackle this without using extra supervisions, we propose to enforce the translated outputs to be semantically invariant w.r.t. small perceptual variations of the inputs, a property we call "semantic robustness". By optimizing a robustness loss w.r.t. multi-scale feature space perturbations of the inputs, our method effectively reduces semantics flipping and produces translations that outperform existing methods both quantitatively and qualitatively.Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- Unsupervised Image-to-Image Translation with Stacked Cycle-Consistent Adversarial Networks2018
- Deflecting Adversarial Attacks with Pixel Deflection2018
- Threat of Adversarial Attacks on Deep Learning in Computer Vision: A SurveyIEEE Access, 2018
- Dilated Residual Networks2017
- Learning from Simulated and Unsupervised Images through Adversarial Training2017
- Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks2017
- Robust Large Margin Deep Neural NetworksIEEE Transactions on Signal Processing, 2017
- The Cityscapes Dataset for Semantic Urban Scene Understanding2016
- Fully convolutional networks for semantic segmentation2015
- Robustness and generalizationMachine Learning, 2011