Yang Xu

Mainstream Large Language Models (LLMs) work under the paradigm of Next-Token Prediction (NTP). Multi-Token Prediction (MTP) is motivated by higher decoding efficiency, extending NTP to enable LLMs to draft multiple tokens during each forward pass. However, existing MTP approaches pretrain MTP along with the target LLM, making it difficult to unlock MTP for LLMs without official support. In this work, we propose a post-hoc approach to training an MTP module for a target LLM, providing an efficient way to evolve the LLM from NTP to MTP. The proposed approach features two main characteristics. (1) No changes to the target LLM, since it is frozen during MTP training. (2) Efficient MTP training via self-distillation from the target LLM’s native NTP capability. Results show that our approach can post-hoc train a performant MTP module via lightweight pretraining.

Crisis events generate massive information flows from diverse sources, which need to be consolidated into public opinion reports to enable timely response by governments and enterprises. Current LLMs, despite strong generation capabilities, fail to achieve perspective diversity, maintain factual consistency, and perform coherent high-level planning. To address these gaps, we propose MPSR: a multi-perspective self-reflection framework. Our framework first assigns diverse stakeholder personas to agents who independently generate initial writing plans from complementary viewpoints. Subsequently, a three-stage debate mechanism refines these plans by identifying conflicts, formulates resolution strategies, and produces a consensus plan, thereby enhancing factual consistency. Finally, we introduce a Report Fusion mechanism to synthesize reports across temporal batches, ensuring comprehensive event coverage. Extensive experiments demonstrate that MPSR significantly outperforms baselines, achieving Date F1 of 0.67, G-Eval of 4.54, and MiniCheck score of 79.43, which represent improvements of 17.5%, 70.0%, and 25.8% over the strongest baseline, respectively.

Public Opinion Reports are essential tools for crisis management, yet their evaluation remains a critical bottleneck that often delays response actions. Recently, dominant Large Language Model (LLM)-based evaluators often overlook a critical challenge: highly open-ended dimensions such as “innovation” and “feasibility” require synthesizing diverse stakeholder perspectives, as different groups judge these qualities from fundamentally different perspectives. Motivated by this, we propose the Role-based Adaptive Evaluation (RAE) framework. This framework employs an adaptive mechanism leveraging multi-perspective evaluation insights through role-based analysis, and further introduces dynamically generated roles tailored to specific contexts for these dimensions. RAE further incorporates multi-role reasoning aggregation to minimize individual biases and enhance evaluation robustness. Extensive experiments demonstrate that RAE significantly improves alignment with human expert judgments, especially on challenging highly open-ended dimensions.