Jingping Liu
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Research Interests
Jingping Liu has a total of 136 co-authors. Here are the top co-authors by number of shared publications:
- Tong Ruan
East China University of Science and Technology Shanghai China
13 shared publications - Yanghua Xiao
Fudan University
13 shared publications - Jiaqing Liang
Fudan University, Shanghai, China
7 shared publications - Chao Wang
Shanghai University, Shanghai, China
5 shared publications - Haiyun Jiang
Shanghai Key Laboratory of Data Science, School of Computer Science, Fudan University, Shanghai, China
4 shared publications - Lihan Chen
School of Computer Science, Fudan University, Shanghai, China
4 shared publications
Latest Publications
- 13 October 2025
- Published by Elsevier in Neural Networks
- Vol. 195, 108203
VLExpan: A visual-enhanced LLM framework with inductive and deductive policies for entity set expansion
- 13 October 2025
- Published by Elsevier in Neural Networks
- Vol. 195, 108203
VLExpan: A visual-enhanced LLM framework with inductive and deductive policies for entity set expansion
- Open Access
DeepSeek: Paradigm Shifts and Technical Evolution in Large AI Models
DeepSeek, a Chinese Artificial Intelligence (AI) startup, has released their V3 and R1 series models, which attracted global attention due to their low cost, high performance, and open-source advantages. This paper begins by reviewing the evolution of large AI models focusing on paradigm shifts, the mainstream Large Language Model (LLM) paradigm, and the DeepSeek paradigm. Subsequently, the paper highlights novel algorithms introduced by DeepSeek, including Multi-head Latent Attention (MLA), Mixture-of-Experts (MoE), Multi-Token Prediction (MTP), and Group Relative Policy Optimization (GRPO). The paper then explores DeepSeek engineering breakthroughs in LLM scaling, training, inference, and system-level optimization architecture. Moreover, the impact of DeepSeek models on the competitive AI landscape is analyzed, comparing them to mainstream LLMs across various fields. Finally, the paper reflects on the insights gained from DeepSeek innovations and discusses future trends in the technical and engineering development of large AI models, particularly in data, training, and reasoning.
Open AccessDeepSeek: Paradigm Shifts and Technical Evolution in Large AI Models
DeepSeek, a Chinese Artificial Intelligence (AI) startup, has released their V3 and R1 series models, which attracted global attention due to their low cost, high performance, and open-source advantages. This paper begins by reviewing the evolution of large AI models focusing on paradigm shifts, the mainstream Large Language Model (LLM) paradigm, and the DeepSeek paradigm. Subsequently, the paper highlights novel algorithms introduced by DeepSeek, including Multi-head Latent Attention (MLA), Mixture-of-Experts (MoE), Multi-Token Prediction (MTP), and Group Relative Policy Optimization (GRPO). The paper then explores DeepSeek engineering breakthroughs in LLM scaling, training, inference, and system-level optimization architecture. Moreover, the impact of DeepSeek models on the competitive AI landscape is analyzed, comparing them to mainstream LLMs across various fields. Finally, the paper reflects on the insights gained from DeepSeek innovations and discusses future trends in the technical and engineering development of large AI models, particularly in data, training, and reasoning.
- Open Access
- 11 April 2025
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39 (28) , 28810-28818
IMQC: A Large Language Model Platform for Medical Quality Control
Medical quality control (MQC) indicators are essential for evaluating the performance of healthcare institutions to ensure high-quality patient care. In this paper, we report the design, implementation, and deployment of the Intelligent EMR-LLM platform for Medical Quality Control (IMQC), a large language model (LLM)-empowered system for automatically computing MQC indicators for enhancing the quality of medical services in Shanghai. It consists of an LLM (i.e., EMR-LLM) for processing electronic medical records (EMRs). With EMR-LLM, IMQC translates existing MQC indicators into a standardized representation language and automatically computes them based on EMRs. Since its deployment in February 2024, IMQC has been adopted by the Shanghai Medical Quality Management Center and associated hospitals. So far, it has processed 1,245 medical quality indicators for secondary- and tertiary-level hospitals, achieving an MQC evaluation accuracy of 93.31%, which is comparable to human experts. It has significantly improved efficiency, increasing from 10 EMRs per hour per human expert to over 1,000 EMRs per hour on average using one single H800 GPU. Over the first round of deployment in Shanghai, it is estimated that IMQC saves around 3.42 million RMB per month in manpower costs compared to traditional reporting methods. The successful deployment of IMQC sets a precedence for other regions to adopt similar AI-driven solutions to enhance medical quality control.
- 11 April 2025
- Published by Association for the Advancement of Artificial Intelligence (AAAI) in Proceedings of the AAAI Conference on Artificial Intelligence
- Vol. 39 (28) , 28810-28818
Open AccessIMQC: A Large Language Model Platform for Medical Quality Control
Medical quality control (MQC) indicators are essential for evaluating the performance of healthcare institutions to ensure high-quality patient care. In this paper, we report the design, implementation, and deployment of the Intelligent EMR-LLM platform for Medical Quality Control (IMQC), a large language model (LLM)-empowered system for automatically computing MQC indicators for enhancing the quality of medical services in Shanghai. It consists of an LLM (i.e., EMR-LLM) for processing electronic medical records (EMRs). With EMR-LLM, IMQC translates existing MQC indicators into a standardized representation language and automatically computes them based on EMRs. Since its deployment in February 2024, IMQC has been adopted by the Shanghai Medical Quality Management Center and associated hospitals. So far, it has processed 1,245 medical quality indicators for secondary- and tertiary-level hospitals, achieving an MQC evaluation accuracy of 93.31%, which is comparable to human experts. It has significantly improved efficiency, increasing from 10 EMRs per hour per human expert to over 1,000 EMRs per hour on average using one single H800 GPU. Over the first round of deployment in Shanghai, it is estimated that IMQC saves around 3.42 million RMB per month in manpower costs compared to traditional reporting methods. The successful deployment of IMQC sets a precedence for other regions to adopt similar AI-driven solutions to enhance medical quality control.
- 15 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 29 (5) , 3642-3653
Decision Tree Extraction for Clinical Decision Support System With If-Else Pseudocode and PlanSelect Strategy
Decision trees, as a structured representation of medical knowledge, are critical resources for building clinical decision support systems. Their structured decision pathways can be used for retrieval to enhance clinical decision making. Currently, mainstream methods mainly utilize large language models and in-context learning for decision tree extraction. However, these methods often face challenges in understanding the structure of decision trees and accurately extracting the complete content of tree nodes, leading to noise in the extracted trees and ultimately impacting their effectiveness in clinical decision support system. To this end, in this paper, we propose a novel decision tree extraction framework, including two stages. In the first stage, we propose to use the If-Else pseudocode to represent the decision tree structure and design specific constraints on format and content to guide the LLM in generating outputs. In the second stage, we introduce a novel node-filling strategy called PlanSelect to match the extracted triplets with sub-sentences in the generated pseudocode, including four reasoning steps: observation, plan, action, and answer. To evaluate the effectiveness of our proposed method, we construct an English decision tree extraction dataset (EMDT) and conduct extensive experiments on the built and public datasets. Experiments on the Text2DT and EMDT datasets demonstrate that our method outperforms the current stateof- the-art approaches, achieving improvements of 1.37% and 1.54% on the $ER$ metric (which is lower is better), respectively. Furthermore, we use the medical decision trees extracted using our framework to improve the model's performance on clinical decision making tasks, i.e., CMBClin and MedQA. Our code and data are publicly available at https://github.com/nlper-hou/TSDTE.
- 15 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 29 (5) , 3642-3653
Decision Tree Extraction for Clinical Decision Support System With If-Else Pseudocode and PlanSelect Strategy
Decision trees, as a structured representation of medical knowledge, are critical resources for building clinical decision support systems. Their structured decision pathways can be used for retrieval to enhance clinical decision making. Currently, mainstream methods mainly utilize large language models and in-context learning for decision tree extraction. However, these methods often face challenges in understanding the structure of decision trees and accurately extracting the complete content of tree nodes, leading to noise in the extracted trees and ultimately impacting their effectiveness in clinical decision support system. To this end, in this paper, we propose a novel decision tree extraction framework, including two stages. In the first stage, we propose to use the If-Else pseudocode to represent the decision tree structure and design specific constraints on format and content to guide the LLM in generating outputs. In the second stage, we introduce a novel node-filling strategy called PlanSelect to match the extracted triplets with sub-sentences in the generated pseudocode, including four reasoning steps: observation, plan, action, and answer. To evaluate the effectiveness of our proposed method, we construct an English decision tree extraction dataset (EMDT) and conduct extensive experiments on the built and public datasets. Experiments on the Text2DT and EMDT datasets demonstrate that our method outperforms the current stateof- the-art approaches, achieving improvements of 1.37% and 1.54% on the $ER$ metric (which is lower is better), respectively. Furthermore, we use the medical decision trees extracted using our framework to improve the model's performance on clinical decision making tasks, i.e., CMBClin and MedQA. Our code and data are publicly available at https://github.com/nlper-hou/TSDTE.
- Open Access
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 19767-19790
Text-to-ES Bench: A Comprehensive Benchmark for Converting Natural Language to Elasticsearch Query
Dongge Xue, Zhili Pu, Zhentao Xia, Hongli Sun, Ruihui Hou, Guangya Yu, Yupian Lin, Yongqi Fan, Jingping Liu, Tong Ruan. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 19767-19790
Open AccessText-to-ES Bench: A Comprehensive Benchmark for Converting Natural Language to Elasticsearch Query
Dongge Xue, Zhili Pu, Zhentao Xia, Hongli Sun, Ruihui Hou, Guangya Yu, Yupian Lin, Yongqi Fan, Jingping Liu, Tong Ruan. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.
- Open Access
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 620-632
Can Multimodal Large Language Models Understand Spatial Relations?
Jingping Liu, Ziyan Liu, Zhedong Cen, Yan Zhou, Yinan Zou, Weiyan Zhang, Haiyun Jiang, Tong Ruan. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 620-632
Open AccessCan Multimodal Large Language Models Understand Spatial Relations?
Jingping Liu, Ziyan Liu, Zhedong Cen, Yan Zhou, Yinan Zou, Weiyan Zhang, Haiyun Jiang, Tong Ruan. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.
- Open Access
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 2825-2851
MedEureka: A Medical Domain Benchmark for Multi-Granularity and Multi-Data-Type Embedding-Based Retrieval
Yongqi Fan, Nan Wang, Kui Xue, Jingping Liu, Tong Ruan. Findings of the Association for Computational Linguistics: NAACL 2025. 2025.
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 2825-2851
Open AccessMedEureka: A Medical Domain Benchmark for Multi-Granularity and Multi-Data-Type Embedding-Based Retrieval
Yongqi Fan, Nan Wang, Kui Xue, Jingping Liu, Tong Ruan. Findings of the Association for Computational Linguistics: NAACL 2025. 2025.
- 27 October 2024
- Published by Springer Nature
- p. 324-341
A Hierarchy-aware Entity Alignment Method for Educational Knowledge Graphs
- 27 October 2024
- Published by Springer Nature
- p. 324-341
A Hierarchy-aware Entity Alignment Method for Educational Knowledge Graphs
- 15 August 2024
- Published by Springer Nature in SN Computer Science
- Vol. 5 (6) , 1-14
Don’t Ignore the Drive of Curiosity: Rethinking Subtleties Between Universality of Commonsense Knowledge and Excellence of Large Language Models
- 15 August 2024
- Published by Springer Nature in SN Computer Science
- Vol. 5 (6) , 1-14
Don’t Ignore the Drive of Curiosity: Rethinking Subtleties Between Universality of Commonsense Knowledge and Excellence of Large Language Models
- 30 July 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Knowledge and Data Engineering
- Vol. 36 (12) , 7442-7454
A Bidirectional Extraction-Then-Evaluation Framework for Complex Relation Extraction
Relation extraction is an important task in the field of natural language processing. Previous works mainly focus on adopting pipeline methods or joint methods to model relation extraction in general scenarios. However, these existing methods face challenges when adapting to complex relation extraction scenarios, such as handling overlapped triplets, multiple triplets, and cross-sentence triplets. In this paper, we revisit the advantages and disadvantages of the aforementioned methods in complex relation extraction. Based on the in-depth analysis, we propose a novel two-stage bidirectional extract-then-evaluate framework named BEERE. In the extraction stage, we first obtain the subject set, relation set, and object set. Then, we design subject- and object-oriented triplet extractors to iteratively recurrent obtain candidate triplets, ensuring high recall. In the evaluation stage, we adopt a relation-oriented triplet filter to determine subject-object pairs based on relations in triplets obtained in the first stage, ensuring high precision. We conduct extensive experiments on three public datasets to show that BEERE achieves state-of-the-art performance in both complex and general relation extraction scenarios. Even when compared to large language models like closed-source/open-source LLMs, BEERE still has significant performance gains
- 30 July 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Knowledge and Data Engineering
- Vol. 36 (12) , 7442-7454
A Bidirectional Extraction-Then-Evaluation Framework for Complex Relation Extraction
Relation extraction is an important task in the field of natural language processing. Previous works mainly focus on adopting pipeline methods or joint methods to model relation extraction in general scenarios. However, these existing methods face challenges when adapting to complex relation extraction scenarios, such as handling overlapped triplets, multiple triplets, and cross-sentence triplets. In this paper, we revisit the advantages and disadvantages of the aforementioned methods in complex relation extraction. Based on the in-depth analysis, we propose a novel two-stage bidirectional extract-then-evaluate framework named BEERE. In the extraction stage, we first obtain the subject set, relation set, and object set. Then, we design subject- and object-oriented triplet extractors to iteratively recurrent obtain candidate triplets, ensuring high recall. In the evaluation stage, we adopt a relation-oriented triplet filter to determine subject-object pairs based on relations in triplets obtained in the first stage, ensuring high precision. We conduct extensive experiments on three public datasets to show that BEERE achieves state-of-the-art performance in both complex and general relation extraction scenarios. Even when compared to large language models like closed-source/open-source LLMs, BEERE still has significant performance gains