What makes Reasoning Models Different? Follow the Reasoning Leader for Efficient Decoding
Authors:
- Ming LiMLMing Li
- Dianqi LiDLDianqi Li
- Kevin LinKLKevin Lin
Preprint
Abstract
Large reasoning models (LRMs) achieve strong reasoning performance by emitting long chains of thought. Yet, these verbose traces slow down inference and often drift into unnecessary detail, known as the overthinking phenomenon. To better understand LRMs' behavior, we systematically analyze the token-level misalignment between reasoning and non-reasoning models. While it is expected that their primary difference lies in the stylistic "thinking cues", LRMs uniquely exhibit two pivotal, previously under-explored phenomena: a Global Misalignment Rebound, where their divergence from non-reasoning models persists or even grows as response length increases, and more critically, a Local Misalignment Diminish, where the misalignment concentrates at the "thinking cues" each sentence starts with but rapidly declines in the remaining of the sentence. Motivated by the Local Misalignment Diminish, we propose FoReaL-Decoding, a collaborative fast-slow thinking decoding method for cost-quality trade-off. In FoReaL-Decoding, a Leading model leads the first few tokens for each sentence, and then a weaker draft model completes the following tokens to the end of each sentence. FoReaL-Decoding adopts a stochastic gate to smoothly interpolate between the small and the large model. On four popular math-reasoning benchmarks (AIME24, GPQA-Diamond, MATH500, AMC23), FoReaL-Decoding reduces theoretical FLOPs by 30 to 50% and trims CoT length by up to 40%, while preserving 86 to 100% of model performance. These results establish FoReaL-Decoding as a simple, plug-and-play route to controllable cost-quality trade-offs in reasoning-centric tasks.Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- From System 1 to System 2: A Survey of Reasoning Large Language ModelsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2025
- Towards Coarse-to-Fine Evaluation of Inference Efficiency for Large Language Models2025
- Evaluating Mathematical Reasoning Beyond AccuracyProceedings of the AAAI Conference on Artificial Intelligence, 2025
- TokenSkip: Controllable Chain-of-Thought Compression in LLMs2025
- LightThinker: Thinking Step-by-Step Compression2025
- CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation2025
- What Happened in LLMs Layers when Trained for Fast vs. Slow Thinking: A Gradient Perspective2025
- Data Diversity Matters for Robust Instruction Tuning2024
- Learning to Decode Collaboratively with Multiple Language Models2024
- Mementos: A Comprehensive Benchmark for Multimodal Large Language Model Reasoning over Image Sequences2024