MicroAST: Towards Super-fast Ultra-Resolution Arbitrary Style Transfer
Open Access
- 26 June 2023
- journal article
- research article
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 37 (3) , 2742-2750
Abstract
Arbitrary style transfer (AST) transfers arbitrary artistic styles onto content images. Despite the recent rapid progress, existing AST methods are either incapable or too slow to run at ultra-resolutions (e.g., 4K) with limited resources, which heavily hinders their further applications. In this paper, we tackle this dilemma by learning a straightforward and lightweight model, dubbed MicroAST. The key insight is to completely abandon the use of cumbersome pre-trained Deep Convolutional Neural Networks (e.g., VGG) at inference. Instead, we design two micro encoders (content and style encoders) and one micro decoder for style transfer. The content encoder aims at extracting the main structure of the content image. The style encoder, coupled with a modulator, encodes the style image into learnable dual-modulation signals that modulate both intermediate features and convolutional filters of the decoder, thus injecting more sophisticated and flexible style signals to guide the stylizations. In addition, to boost the ability of the style encoder to extract more distinct and representative style signals, we also introduce a new style signal contrastive loss in our model. Compared to the state of the art, our MicroAST not only produces visually superior results but also is 5-73 times smaller and 6-18 times faster, for the first time enabling super-fast (about 0.5 seconds) AST at 4K ultra-resolutions.Keywords
All Related Versions
This publication has 5 references indexed in Scilit:
- Evaluate and improve the quality of neural style transferComputer Vision and Image Understanding, 2021
- Creative and diverse artwork generation using adversarial networksIET Computer Vision, 2020
- GLStyleNet: exquisite style transfer combining global and local pyramid featuresIET Computer Vision, 2020
- Neural Style Transfer: A ReviewIEEE Transactions on Visualization and Computer Graphics, 2019
- Wiki Art Gallery, Inc.: A Case for Critical ThinkingIssues in Accounting Education, 2011