Excitement-Inducing Commentary Text-to-Speech System for Fighting Game Video Scenes

Authors:
Open Access
Publisher website
Google Scholar
Abstract
In recent years, video games have become a spectator activity, with e-sports and live streaming attracting large audiences. In e-sports, human commentators can enhance viewer excitement by accurately describing match situations and adjusting their speaking style to match exciting scenes. While automation of text generation from video games has been researched, generation of commentary speech that excites the audience has received little attention. To this end, we focus on modeling an e-sports commentator’s speaking style. Specifically, we propose a text-to-speech (TTS) system that can adjust the excitement level of synthetic speech on the basis of the gameplay videos. Our system consists of a TTS model and an excitement score predictor. The TTS model is trained to predict commentary speech from text and a binary excitement label. The excitement score predictor rates the excitement level of gameplay videos and feeds the binarized excitement score to the TTS model. We validate the effectiveness of our system using gameplay videos of Super Smash Bros. Ultimate. Subjective evaluations demonstrate that 1) our TTS model effectively controls excitement levels and 2) the TTS system enhanced the audience’s engagement and entertainment compared to a baseline TTS system.
Funding Information
  • Japan Society for the Promotion of Science (JSPS) Kakenhi (11K17945)