CATT: Complexity-Aware Thinker–Talker Orchestration for Low-Latency Spoken Responses in Digital Humans
Open Access
- 1 November 2026
- journal article
- research article
- Published by MDPI AG in Electronics
- Vol. 15 (21) , 5064
Abstract
Digital humans must speak promptly, but thinking-enabled large language models (LLMs) answer only after reasoning. CATT (Complexity-Aware Thinker–Talker) routes easy turns to a direct reply; hard turns get an answer-free acknowledgment, progress speech from background reasoning, then the final answer. It needs no fine-tuning and is plug-and-play: its orchestration code runs unchanged when the model endpoint, router, or speech and playback layer is replaced. Evaluated on the transcript-to-speech subsystem (local Qwen3-32B, 738 isolated turns, and virtual playback), CATT starts speaking about 6.3 s earlier than a prompt-matched thinking baseline. The first substantive content comes about 10.5 s earlier on closed-form math and science questions and 3.0 s earlier on open-ended requests (about 1 s later when routed to reasoning). The final closed-form answer arrives about 3.2 s later at the same accuracy. An LLM-judged listening rating, a proxy that includes waiting, places CATT about 1 point above the thinking baseline but 0.42 below a non-thinking baseline on open-ended requests. Both latency gains held for three model families accessed through hosted APIs and in an end-to-end avatar case study. A limited ReVerT reproduction on the same reasoning started speaking 8.4 s after CATT. Real users, microphones, networks and multi-user load remain untested.Keywords
This publication has 23 references indexed in Scilit:
- SynapseRoute: an auto-route switching framework on dual-state large language model for medical question answeringBMJ Innovations, 2026
- Fly-Swat or Cannon? Cost-Effective Language Model Choice via Meta-Modeling2024
- Efficient Memory Management for Large Language Model Serving with PagedAttention2023
- Turn-taking in Conversational Systems and Human-Robot Interaction: A ReviewComputer Speech & Language, 2021
- Pauses, gaps and overlaps in conversationsJournal of Phonetics, 2010
- Universals and cultural variation in turn-taking in conversationProceedings of the National Academy of Sciences of the United States of America, 2009
- Creating interactive virtual humans: some assembly requiredIEEE Intelligent Systems, 2002
- Intraclass correlations: Uses in assessing rater reliability.Psychological Bulletin, 1979
- Note on the Sampling Error of the Difference Between Correlated Proportions or PercentagesPsychometrika, 1947
- Probable Inference, the Law of Succession, and Statistical InferenceJournal of the American Statistical Association, 1927