Wen Wang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Wen Wang has a total of 49 co-authors. Here are the top co-authors by number of shared publications:
- Qian Chen
Token Foundry, Alibaba Group, Hangzhou, China
6 shared publications - Guanrou Yang
Shanghai Jiao Tong University, Shanghai, China
2 shared publications - Wenrui Liu
Zhejiang University, Hangzhou, China
2 shared publications - Xie Chen
Shanghai Jiao Tong University, Shanghai Innovation Institute, Shanghai, China
2 shared publications - Baotian Hu
Harbin Institute of Technology (Shenzhen) Shenzhen China
1 shared publication Bin Ma
Speech Lab, Institute for Intelligent Computing, Alibaba Group, Singapore
1 shared publication
Latest Publications
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 7969-7978
Enabling Proactive Spoken Turns via a Generalized Style-Aware Full-Duplex Framework
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 7969-7978
Open AccessEnabling Proactive Spoken Turns via a Generalized Style-Aware Full-Duplex Framework
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 1979-1988
Efficient Chain-of-Modality Reasoning via Progressive Compression for Spoken Language Models
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 1979-1988
Open AccessEfficient Chain-of-Modality Reasoning via Progressive Compression for Spoken Language Models
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 10083-10092
MelodyEdit: Zero-shot Music Editing with Disentangled Inversion Control
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 10083-10092
Open AccessMelodyEdit: Zero-shot Music Editing with Disentangled Inversion Control
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 10748-10757
EmoVoice: LLM-based Emotional Text-To-Speech Model with Freestyle Text Prompting
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 10748-10757
Open AccessEmoVoice: LLM-based Emotional Text-To-Speech Model with Freestyle Text Prompting
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 10632-10641
Speech Token Prediction via Compressed-to-fine Language Modeling for Speech Generation
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 10632-10641
Open AccessSpeech Token Prediction via Compressed-to-fine Language Modeling for Speech Generation
- 27 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Signal Processing Letters
- Vol. 31 (10709908) , 1740-1744
Tuning Large Language Model for Speech Recognition With Mixed-Scale Re-Tokenization
Large Language Models (LLMs) have proven successful across a spectrum of speech-related tasks, such as speech recognition, text-to-speech, and spoken language understanding. Recently, the use of discretized speech features has gained attention as an efficient and compatible alternative to continuous features for LLMs. This is mainly due to their reduced storage requirements and better alignment of these features with LLM's input space. However, the typical practice of freezing the speech encoder during training poses challenges in bridging the modality gap between speech and text. To address this, we propose to use a mixed-scale re-tokenization layer, integrating multiple granularities in discretized speech features directly within the LLM's input module. Our experimental results demonstrated that the proposed method can effectively enhance the performance of ASR in the setting of continuous learning of an LLM, highlighting the importance of a meticulously designed input module for the integration of discretized speech features with an LLM.
- 27 June 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Signal Processing Letters
- Vol. 31 (10709908) , 1740-1744
Tuning Large Language Model for Speech Recognition With Mixed-Scale Re-Tokenization
Large Language Models (LLMs) have proven successful across a spectrum of speech-related tasks, such as speech recognition, text-to-speech, and spoken language understanding. Recently, the use of discretized speech features has gained attention as an efficient and compatible alternative to continuous features for LLMs. This is mainly due to their reduced storage requirements and better alignment of these features with LLM's input space. However, the typical practice of freezing the speech encoder during training poses challenges in bridging the modality gap between speech and text. To address this, we propose to use a mixed-scale re-tokenization layer, integrating multiple granularities in discretized speech features directly within the LLM's input module. Our experimental results demonstrated that the proposed method can effectively enhance the performance of ASR in the setting of continuous learning of an LLM, highlighting the importance of a meticulously designed input module for the integration of discretized speech features with an LLM.
- Open Access
- 26 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 5223-5234
Enhancing Multi-modal Multi-hop Question Answering via Structured Knowledge and Unified Retrieval-Generation
Multi-modal multi-hop question answering involves answering a question by reasoning over multiple input sources from different modalities. Existing methods often retrieve evidences separately and then use a language model to generate an answer based on the retrieved evidences, and thus do not adequately connect candidates and are unable to model the interdependent relations during retrieval. Moreover, the pipelined approaches of retrieval and generation might result in poor generation performance when retrieval performance is low. To address these issues, we propose a Structured Knowledge and Unified Retrieval-Generation (SKURG) approach. SKURG employs an Entity-centered Fusion Encoder to align sources from different modalities using shared entities. It then uses a unified Retrieval-Generation Decoder to integrate intermediate retrieval results for answer generation and also adaptively determine the number of retrieval steps. Extensive experiments on two representative multi-modal multi-hop QA datasets MultimodalQA and WebQA demonstrate that SKURG outperforms the state-of-the-art models in both source retrieval and answer generation performance with fewer parameters1.
- 26 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 5223-5234
Open AccessEnhancing Multi-modal Multi-hop Question Answering via Structured Knowledge and Unified Retrieval-Generation
Multi-modal multi-hop question answering involves answering a question by reasoning over multiple input sources from different modalities. Existing methods often retrieve evidences separately and then use a language model to generate an answer based on the retrieved evidences, and thus do not adequately connect candidates and are unable to model the interdependent relations during retrieval. Moreover, the pipelined approaches of retrieval and generation might result in poor generation performance when retrieval performance is low. To address these issues, we propose a Structured Knowledge and Unified Retrieval-Generation (SKURG) approach. SKURG employs an Entity-centered Fusion Encoder to align sources from different modalities using shared entities. It then uses a unified Retrieval-Generation Decoder to integrate intermediate retrieval results for answer generation and also adaptively determine the number of retrieval steps. Extensive experiments on two representative multi-modal multi-hop QA datasets MultimodalQA and WebQA demonstrate that SKURG outperforms the state-of-the-art models in both source retrieval and answer generation performance with fewer parameters1.