Creating Word Paintings Jointly Considering Semantics, Attention, and Aesthetics
Open Access
- 31 July 2022
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Applied Perception
- Vol. 19 (3) , 1-21
Abstract
In this article, we present a content-aware method for generating a word painting. Word painting is a composite artwork made from the assemblage of words extracted from a given text, which carries similar semantics and visual features to a given source image. However, word painting, usually created by skilled artists, involves tedious manual processes, especially when generating streamlines and laying out text. Hence, we provide an easy method to create word paintings for users. How to design textural layout that simultaneously conveys the input image and enables easy access to the semantic theme is the key challenge to generating a visually pleasing word painting. To address this issue, given an image and its content-related text, we first decompose the input image into several regions and approximate each region with a smooth vector field. At the same time, by analyzing the input text, we extract some weighted keywords as the graphic elements. Then, to measure the likelihood of positions in the input image that attract the observers’ attention, we generate a saliency map with our trained visual attention model. Finally, jointly considering visual attention and aesthetic rules, we propose an energy-based optimization framework to arrange extracted keywords into the decomposed regions and synthesize a word painting. Experimental results and user studies show that this method is able to generate a fashionable and appealing word painting.Keywords
Funding Information
- National Nature Science Foundation of China (61772440, and 61772463)
This publication has 22 references indexed in Scilit:
- Morphable Word Clouds for Time-Varying Text Data VisualizationIEEE Transactions on Visualization and Computer Graphics, 2015
- PicWords: Render a Picture by Packing KeywordsIEEE Transactions on Multimedia, 2014
- Automatic generation of social media snippets for mobile browsing2013
- Recommendation system for automatic design of magazine covers2013
- Synthesizing open worlds with constraints using locally annealed reversible jump MCMCACM Transactions on Graphics, 2012
- Semantic Wordification of Document CollectionsComputer Graphics Forum, 2012
- Digital micrographyACM Transactions on Graphics, 2011
- Structure-based ASCII artACM Transactions on Graphics, 2010
- Computational visual attention systems and their cognitive foundationsACM Transactions on Applied Perception, 2010
- Artistic Surface Rendering Using Layout of TextComputer Graphics Forum, 2002