ENTRA: Entropy-Based Redundancy Avoidance in Large Language Model Reasoning
Preprint
Abstract
Large Reasoning Models (LRMs) often suffer from overthinking, generating unnecessarily long reasoning chains even for simple tasks. This leads to substantial computational overhead with limited performance gain, primarily due to redundant verification and repetitive generation. While prior work typically constrains output length or optimizes correctness, such coarse supervision fails to guide models toward concise yet accurate inference. In this paper, we propose ENTRA, an entropy-based training framework that suppresses redundant reasoning while preserving performance. ENTRA first estimates the token-level importance using a lightweight Bidirectional Importance Estimation (BIE) method, which accounts for both prediction confidence and forward influence. It then computes a redundancy reward based on the entropy of low-importance tokens, normalized by its theoretical upper bound, and optimizes this reward via reinforcement learning. Experiments on mathematical reasoning benchmarks demonstrate that ENTRA reduces output length by 37% to 53% with no loss-and in some cases, gains-in accuracy. Our approach offers a principled and efficient solution to reduce overthinking in LRMs, and provides a generalizable path toward redundancy-aware reasoning optimization.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- HAPO: Training Language Models to Reason Concisely via History-Aware Policy OptimizationProceedings of the AAAI Conference on Artificial Intelligence, 2026
- Toward large reasoning models: A survey of reinforced reasoning with large language modelsPatterns, 2025
- DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Models2025
- LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression2024
- Generalizing Math Word Problem Solvers via Solution DiversificationProceedings of the AAAI Conference on Artificial Intelligence, 2023
- Tree of Thoughts: Deliberate Problem Solving with Large Language Models2023
- Compressing Context to Enhance Inference Efficiency of Large Language Models2023
- Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models2022
- Deep Neural Solver for Math Word Problems2017
- MAWPS: A Math Word Problem Repository2016