Kota Yamaguchi

AI-powered tools have transformed creative workflows, including graphic design assessment, where designers receive AI-generated scores to refine their work. In commercial design, these scores often reflect expected business outcomes rather than purely aesthetic quality. However, the black-box nature of AI-generated feedback often leaves designers uncertain about how to improve their work, leading to inefficient iterations and reduced satisfaction. To address this challenge, we explore how eXplainable AI (XAI) can make score-based feedback more interpretable and actionable. We developed two complementary forms of XAI feedback: highlight-based feedback, which identifies design elements contributing to the score, and adjustment feedback, which presents design variants to visualize how small changes in specific properties (e.g., size, position, or color) affect scores.We conducted a user study with 18 professional designersworking on a commercial advertisement design task, comparing three systems: 1) a baseline score-only system, 2) a prototype system with highlight-based feedback, and 3) a combined system integrating both feedback types. Results showthat both XAI-enhanced systems significantly improved designers’ perceived usefulness, explainability, actionability, and creativity, while task performance metrics (completion time, iteration count, and design diversity) remained comparable to the baseline. Only the combined system significantly improved overall satisfaction, as designers adopted an effective two-phase workflow balancing exploration and score-driven refinement. These findings demonstrate that integrating complementary XAI feedback can support informed and satisfying creative decision-making in commercial design workflows.

In layout design for advertisements and web pages, balancing visual appeal and communication efficiency is crucial. This study examines automated text box placement in incomplete layouts, comparing a standard Transformer-based method, a small Vision and Language Model (Phi3.5-vision), a large pretrained VLM (Gemini), and an extended Transformer that processes multiple images. Evaluations on the Crello dataset show the standard Transformer-based models generally outperform VLM-based approaches, particularly when incorporating richer appearance information. However, all methods face challenges with very small text or densely populated layouts. These findings highlight the benefits of task-specific architectures and suggest avenues for further improvement in automated layout design.

Explainable AI (XAI) attempts to produce interpretable results from highly complex AI systems, but its form and effectiveness vary depending on the application domain. In this paper, we explore how XAI techniques can help graphic designers work on advertising materials. A creative domain such as graphic design is often characterized by a weak connection between the individual work and the business goal; e.g., a small change in the design of a banner can result in a huge difference in the audience’s reaction. We develop an XAI system for designers that provides visual feedback explaining which component of the design is likely to affect the business metric. Our user study shows that with our system, designers complete the project in fewer iterations and in less time to achieve the desired quality of work compared to naive score-based feedback. These findings highlight the benefits of leveraging XAI in creative domains.

We propose an inference procedure for deep convolutional neural networks (CNNs) when partial evidence is available. Our method consists of a general feedback-based propagation approach (feedback-prop) that boosts the prediction accuracy for an arbitrary set of unknown target labels when the values for a non-overlapping arbitrary set of target labels are known. We show that existing models trained in a multi-label or multi-task setting can readily take advantage of feedback-prop without any retraining or fine-tuning. Our feedback-prop inference procedure is general, simple, reliable, and works on different challenging visual recognition tasks. We present two variants of feedback-prop based on layer-wise and residual iterative updates. We experiment using several multi-task models and show that feedback-prop is effective in all of them. Our results unveil a previously unreported but interesting dynamic property of deep CNNs. We also present an associated technical approach that takes advantage of this property for inference under partial evidence in general visual recognition tasks.