JFJiazhan Feng

Jiazhan Feng

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Jiazhan Feng has a total of 24 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    RetriEVAL: Evaluating Text Generation with Contextualized Lexical Match

    by Zhen Li, Xinchi Li, Chongyang Tao, Jiazhan Feng, Tao Shen, Can Xu, Hao Wang, Dongyan Zhao, Shuai Ma

    Pre-trained language models have made significant advancements in text generation tasks. Nevertheless, evaluating the generated text with automatic metrics is still challenging. Compared with supervised metrics, unsupervised metrics which are known for generality and robustness, are frequently employed to assess the quality of generated text efficiently. The representative unsupervised metric BERTScore uses pretrained embedding to calculate the word-to-word similarity across all tokens as evaluation scores, which can introduce potential noise due to the inclusion of tokens that do not contribute significantly to the semantics of the text. Furthermore, its heavy reliance on dense embeddings may lead to lower accuracy when evaluating text outside the common contexts represented in the training data, making it less effective in handling uncommon linguistic patterns Additionally, BERTScore treats all tokens with equal importance and lacks the ability to perform meaningful contextual expansion, which can result in less accurate similarity measurements, particularly when dealing with paraphrased or semantically rich text. To address this problem, we propose an unsupervised automatic evaluation metric inspired by the concept of lexical match in information retrieval. Our method leverages contextualized lexical matching to measure exact matches between identical tokens and dynamically matches different tokens based on their contextualized representations. Experiments on SummEval and Topical-Chat demonstrate our proposed RetriEVAL can correlate better with human judgments than previous unsupervised metrics.


  • Open Access

    A Step Closer to Comprehensive Answers: Constrained Multi-Stage Question Decomposition with Large Language Models

    by Hejing Cao, Zhenwei An, Jiazhan Feng, Kun Xu, Liwei Chen, Dongyan Zhao

    While large language models exhibit remarkable performance in the Question Answering task, they are susceptible to hallucinations. Challenges arise when these models grapple with understanding multi-hop relations in complex questions or lack the necessary knowledge for a comprehensive response. To address this issue, we introduce the "Decompose-and-Query" framework (D&Q). This framework guides the model to think and utilize external knowledge similar to ReAct, while also restricting its thinking to reliable information, effectively mitigating the risk of hallucinations. Experiments confirm the effectiveness of D&Q: On our ChitChatQA dataset, D&Q does not lose to ChatGPT in 67% of cases; on the HotPotQA question-only setting, D&Q achieved an F1 score of 59.6%. Our code is available at https://github.com/alkaidpku/DQ-ToolQA.


  • Open Access

    Dimension-Prompts Boost Commonsense Consolidation

    by Jiazhan Feng, Chongyang Tao, Tao Shen, Chang Liu, Dongyan Zhao

  • Open Access

    Learning Multi-turn Response Selection in Grounded Dialogues with Reinforced Knowledge and Context Distillation

    by Jiazhan Feng, Chongyang Tao, Xueliang Zhao, Dongyan Zhao

    Recently, knowledge-grounded dialogue systems have gained increasing attention. Great efforts have been made to build response matching models where all dialogue content and knowledge sentences are leveraged. However, knowledge redundancy and distraction of irrelevant dialogue content often exist in knowledge-grounded conversations, which may affect the matching process and lead to inferior performance. In addition, irrelevant dialogue history and excessive knowledge also hinder the exploitation of popular pre-trained language models (PLMs) due to the limitation of input length. To address these challenges, we propose a new knowledge-grounded dialogue model based on PLMs, where a knowledge selector and a context selector are designed for filtering out irrelevant knowledge sentences and redundant dialogue history, respectively. Considering the lack of labeled data for the learning of two selectors, we pre-train them with weakly-supervised tasks and then jointly conduct the optimization of knowledge and context selection and fine-tuning of PLMs for response ranking with reinforcement learning (RL). By this means, the dialogue model can distill more accurate and concise knowledge and dialogue content for subsequent response ranking module, and the overall model can converge and perform better. We conduct experiments on two benchmarks and evaluation results indicate that our model can significantly outperform the state-of-the-art methods.


  • Open Access

    Length-Adaptive Distillation: Customizing Small Language Model for Dynamic Token Pruning

    by Chang Liu, Chongyang Tao, Jianxin Liang, Jiazhan Feng, Tao Shen, Quzhe Huang, Dongyan Zhao

    Chang Liu, Chongyang Tao, Jianxin Liang, Jiazhan Feng, Tao Shen, Quzhe Huang, Dongyan Zhao. Findings of the Association for Computational Linguistics: EMNLP 2023. 2023.

  • Open Access

    FAA: Fine-grained Attention Alignment for Cascade Document Ranking

    by Zhen Li, Chongyang Tao, Jiazhan Feng, Tao Shen, Dongyan Zhao, Xiubo Geng, Daxin Jiang

    Zhen Li, Chongyang Tao, Jiazhan Feng, Tao Shen, Dongyan Zhao, Xiubo Geng, Daxin Jiang. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.


  • Training Two-Stage Knowledge-Grounded Dialogues with Attention Feedback

    by Zhen Li, Jiazhan Feng, Chongyang Tao, Dongyan Zhao

  • Open Access

    Building an Efficient and Effective Retrieval-based Dialogue System via Mutual Learning

    by Chongyang Tao, Jiazhan Feng, Chang Liu, Juntao Li, Xiubo Geng, Daxin Jiang

    Establishing retrieval-based dialogue systems that can select appropriate responses from the pre-built index has gained increasing attention from researchers. For this task, the adoption of pre-trained language models (such as BERT) has led to remarkable progress in a number of benchmarks. There exist two common approaches, including cross-encoders which perform full attention over the inputs, and bi-encoders that encode the context and response separately. The former gives considerable improvements in accuracy but is often inapplicable in practice for large-scale retrieval given the cost of the full attention required for each sample at test time. The latter is efficient for billions of indexes but suffers from sub-optimal performance. In this work, we propose to combine the best of both worlds to build a retrieval system. Specifically, we employ a fast bi-encoder to replace the traditional feature-based pre-retrieval model (such as BM25) and set the response re-ranking model as a more complicated architecture (such as cross-encoder). To further improve the effectiveness of our framework, we train the pre-retrieval model and the re-ranking model at the same time via mutual learning, which enables two models to learn from each other throughout the training process. We conduct experiments on two benchmarks and evaluation results demonstrate the efficiency and effectiveness of our proposed framework.