ZWZheng Wang

Zheng Wang

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Zheng Wang has a total of 53 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Large Generative Models Meet Multimodal Applications (LGM 3 A)

    by Zheng Wang, Qianqian Chen, Yiyang Luo, Zhiqiu Ye, Shi Wei, Hanwang Zhang, Tat-Seng Chua

  • Open Access

    PromptBoost: Annotation-Free Prompt Expansion for Chinese Text-to-Image Generation

    by Hovan Hoe Man Leong, Qianqian Chen, Zhiqiu Ye, Zheng Wang, Yangkai Ding

  • TrajSV: A Trajectory-Based Model for Sports Video Representations and Applications

    by Zheng Wang, Shihao Xu, Wei Shi

    Sports analytics has received significant attention from both academia and industry in recent years. Despite the growing interest and efforts in this field, several issues remain unresolved, including (1) data unavailability, (2) lack of an effective trajectory-based framework, and (3) requirement for sufficient supervision labels. In this paper, we present TrajSV, a trajectory-based framework that addresses various issues in existing studies. TrajSV comprises three components: data preprocessing, Clip Representation Network (CRNet), and Video Representation Network (VRNet). The data preprocessing module extracts player and ball trajectories from sports broadcast videos. CRNet utilizes a trajectory-enhanced Transformer module to learn clip representations based on these trajectories. Additionally, VRNet learns video representations by aggregating clip representations and visual features with an encoder-decoder architecture. Finally, a triple contrastive loss is introduced to optimize both video and clip representations in an unsupervised manner. The experiments are conducted on three broadcast video datasets to verify the effectiveness of TrajSV for three types of sports (i.e., soccer, basketball, and volleyball) with three downstream applications (i.e., sports video retrieval, action spotting, and video captioning). The results demonstrate that TrajSV achieves state-of-the-art performance in sports video retrieval, showcasing a nearly 70% improvement. It outperforms baselines in action spotting, achieving state-of-the-art results in 9 out of 17 action categories, and demonstrates a nearly 20% improvement in video captioning. Additionally, we introduce a deployed system along with the three applications based on TrajSV.

  • Open Access

    A Method for Efficient Structured Data Generation with Large Language Models

    by Zongzhi Hou, Ruohan Zhao, Zhongyang Li, Zheng Wang, Yizhen Wu, Junwei Gou, Zhifeng Zhu

    With the rapid development of large language model technology, we find ourselves at an interesting juncture regarding the importance of data. The textual data samples from these large unsupervised models are often of poor quality, which in turn produces substandard results. Implicitly, this means that the model struggles to learn the exact underlying structure of the data distribution without supervision, which can manifest as output lacking fidelity and relevance to real data distributions. In order to overcome some of these limitations in data-driven text generation tasks, this paper presents a Efficient Data Generation System (EDGS) for multimodal structured data generation. EDGS achieves this by using clustering abstraction to process various data input types through templates, thereby enabling quick data generation and reducing resource consumption. EDGS is robust, ensuring stable and reliable performance across different conditions and is more cost-effective in terms of token usage. Traditional methods handle data on a per-instance basis without using templates, but for EDGS, by abstracting and clustering different input types, it efficiently generates data from large models. Its adaptability makes it ideal for multi-modal data generation tasks since it relies on the fundamental functions of general large-scale language models. A query-answer bidirectional generation mechanism is used to rapidly amplify data with the system's help.


  • Open Access

    LGM 3 A '24: the 2nd Workshop on Large Generative Models Meet Multimodal Applications

    by Shihao Xu, Yiyang Luo, Justin Dauwels, Andy Khong, Zheng Wang, Qianqian Chen, Chen Cai, Wei Shi, Tat-Seng Chua

    This workshop aims to explore the potential of large generative models to revolutionize how we interact with multimodal information. A Large Language Model (LLM) represents a sophisticated form of artificial intelligence engineered to comprehend and produce natural language text, exemplified by technologies such as GPT, LLaMA, Flan-T5, ChatGLM, Qwen, etc. These models undergo training on extensive text datasets, exhibiting commendable attributes including robust language generation, zero-shot transfer capabilities, and In-Context Learning (ICL). With the surge in multimodal content-encompassing images, videos, audio, and 3D models-over the recent period, Large MultiModal Models (LMMs) have seen significant enhancements. These improvements enable the augmentation of conventional LLMs to accommodate multimodal inputs or outputs, as seen in BLIP, Flamingo, KOSMOS, LLaVA, Gemini, GPT-4, etc. Concurrently, certain research initiatives have developed specific modalities, with Kosmos2 and MiniGPT-5 focusing on image generation, and SpeechGPT on speech production. There are also endeavors to integrate LLMs with external tools to achieve a near "any-to-any" multimodal comprehension and generation capacity, illustrated by projects like Visual-ChatGPT, ViperGPT, MMREACT, HuggingGPT, and AudioGPT. Collectively,these models, spanning not only text and image generation but also other modalities, are referred to as large generative models. This workshop will allow researchers, practitioners, and industry professionals to explore the latest trends and best practices in the multimodal applications of large generative models.


  • Open Access

    Billiards Sports Analytics: Datasets and Tasks

    by Qianru Zhang, Zheng Wang, Cheng Long, Siu-Ming Yiu

    Nowadays, it becomes a common practice to capture some data of sports games with devices such as GPS sensors and cameras and then use the data to perform various analyses on sports games, including tactics discovery, similar game retrieval, performance study, and so forth. While this practice has been conducted to many sports such as basketball and soccer, it remains largely unexplored on the billiards sports, which is mainly due to the lack of publicly available datasets. Motivated by this, we collect a dataset of billiards sports, which includes the layouts (i.e., locations) of billiards balls after performing break shots, called break shot layouts, the traces of the balls as a result of strikes (in the form of trajectories), and detailed statistics and performance indicators. We then study and develop techniques for three tasks on the collected dataset, including (1) prediction and (2) generation on the layouts data, and (3) similar billiards layout retrieval on the layouts data, which can serve different users such as coaches, players and fans. We conduct extensive experiments on the collected dataset and the results show that our methods perform effectively and efficiently.


  • Open Access

    Multimodal Query Suggestion with Multi-Agent Reinforcement Learning from Human Feedback

    by Zheng Wang, Bingzheng Gan, Wei Shi

    In the rapidly evolving landscape of information retrieval, search engines strive to provide more personalized and relevant results to users. Query suggestion systems play a crucial role in achieving this goal by assisting users in formulating effective queries. However, existing query suggestion systems mainly rely on textual inputs, potentially limiting user search experiences for querying images. In this paper, we introduce a novel Multimodal Query Suggestion (MMQS) task, which aims to generate query suggestions based on user query images to improve the intentionality and diversity of search results. We present the RL4Sugg framework, leveraging the power of Large Language Models (LLMs) with Multi-Agent Reinforcement Learning from Human Feedback to optimize the generation process. Through comprehensive experiments, we validate the effectiveness of RL4Sugg, demonstrating a 18% improvement compared to the best existing approach. Moreover, the MMQS has been transferred into real-world search engine products, which yield enhanced user engagement. Our research advances query suggestion systems and provides a new perspective on multimodal information retrieval.


  • Open Access

    NeurSEG: A Segment Driven Deep Neural Model for Nested Named Entity Recognition

    by Zheng Wang, Fei Li, Cheng Long

    Named Entity Recognition (NER) is a fundamental problem in natural language processing (NLP). Apart from flat entities, nested entities are also commonly existed in real-life textual data. However, the current methods are not capable of handling nested structures for NER effectively. In this paper, we propose a novel segment driven modeling method (NeurSEG) for the nested NER problem, which can effectively extract entities from the nested structures in complex nesting scenarios. The proposed NeurSEG model first finds the nested label of each word in a sentence and determines the positional relationships between neighbouring words, and then extracts the entities and predicts the corresponding entity types. In addition, we also propose an augmented training method for further improving the performance. We have conducted extensive experiments based on both flat and nested NER benchmark datasets. The performance results have shown that our proposed NeurSEG model has achieved promising performance while retaining its runtime efficiency for the nested NER task. Moreover, the proposed model has also achieved very competitive results when compared with the existing models for the flat NER task, demonstrating its capability for tackling both nested and flat NER tasks.