Bingzheng Gan
Bingzheng Gan has a total of 6 co-authors. Here are the top co-authors by number of shared publications:
- Wei Shi
Huawei Singapore Research Center, Singapore, Singapore
2 shared publications - Zheng Wang
Nanyang technological university, Singapore, Singapore
2 shared publications - Cheng Long
Nanyang technological university, Singapore, Singapore
1 shared publication - Shihao Xu
Huawei Singapore Research Center, Singapore, Singapore
1 shared publication - Tat-Seng Chua
National University of Singapore, Singapore, Singapore
1 shared publication - Zhao Cao
Huawei Poisson Lab
1 shared publication
Latest Publications
- Open Access
- 13 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1374-1385
Multimodal Query Suggestion with Multi-Agent Reinforcement Learning from Human Feedback
In the rapidly evolving landscape of information retrieval, search engines strive to provide more personalized and relevant results to users. Query suggestion systems play a crucial role in achieving this goal by assisting users in formulating effective queries. However, existing query suggestion systems mainly rely on textual inputs, potentially limiting user search experiences for querying images. In this paper, we introduce a novel Multimodal Query Suggestion (MMQS) task, which aims to generate query suggestions based on user query images to improve the intentionality and diversity of search results. We present the RL4Sugg framework, leveraging the power of Large Language Models (LLMs) with Multi-Agent Reinforcement Learning from Human Feedback to optimize the generation process. Through comprehensive experiments, we validate the effectiveness of RL4Sugg, demonstrating a 18% improvement compared to the best existing approach. Moreover, the MMQS has been transferred into real-world search engine products, which yield enhanced user engagement. Our research advances query suggestion systems and provides a new perspective on multimodal information retrieval.
- 13 May 2024
- Published by Association for Computing Machinery (ACM)
- p. 1374-1385
Open AccessMultimodal Query Suggestion with Multi-Agent Reinforcement Learning from Human Feedback
In the rapidly evolving landscape of information retrieval, search engines strive to provide more personalized and relevant results to users. Query suggestion systems play a crucial role in achieving this goal by assisting users in formulating effective queries. However, existing query suggestion systems mainly rely on textual inputs, potentially limiting user search experiences for querying images. In this paper, we introduce a novel Multimodal Query Suggestion (MMQS) task, which aims to generate query suggestions based on user query images to improve the intentionality and diversity of search results. We present the RL4Sugg framework, leveraging the power of Large Language Models (LLMs) with Multi-Agent Reinforcement Learning from Human Feedback to optimize the generation process. Through comprehensive experiments, we validate the effectiveness of RL4Sugg, demonstrating a 18% improvement compared to the best existing approach. Moreover, the MMQS has been transferred into real-world search engine products, which yield enhanced user engagement. Our research advances query suggestion systems and provides a new perspective on multimodal information retrieval.
- Open Access
- 26 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 9744-9745
LGM3A '23: 1st Workshop on Large Generative Models Meet Multimodal Applications
A large language model is a type of artificial intelligence model designed to understand and generate natural language text, such as GPT, T5, RoBERTa, BERT, etc. These models are trained on vast amounts of text data, allowing them to learn the patterns and structures of human language. With the increasing amount of multimodal information such as audio, visual, and text data generated, there is a growing need of leveraging large generative language model for multimodal applications. Recently, a few notable multimodal models (e.g., BLIP, Flamingo, KOSMOS, PaLM-E, LLaVA, Visual ChatGPT, GPT-4, etc.) with a combination of large language models significantly enhanced their understanding and generate more accurate and nuanced responses. The workshop will provide an opportunity for researchers, practitioners, and industry professionals to explore the latest trends and best practices in the field of multimodal applications of large generative models. The workshop will also focus on exploring the challenges and opportunities of integrating large language models with other AI technologies such as computer vision and speech recognition.
- 26 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 9744-9745
Open AccessLGM3A '23: 1st Workshop on Large Generative Models Meet Multimodal Applications
A large language model is a type of artificial intelligence model designed to understand and generate natural language text, such as GPT, T5, RoBERTa, BERT, etc. These models are trained on vast amounts of text data, allowing them to learn the patterns and structures of human language. With the increasing amount of multimodal information such as audio, visual, and text data generated, there is a growing need of leveraging large generative language model for multimodal applications. Recently, a few notable multimodal models (e.g., BLIP, Flamingo, KOSMOS, PaLM-E, LLaVA, Visual ChatGPT, GPT-4, etc.) with a combination of large language models significantly enhanced their understanding and generate more accurate and nuanced responses. The workshop will provide an opportunity for researchers, practitioners, and industry professionals to explore the latest trends and best practices in the field of multimodal applications of large generative models. The workshop will also focus on exploring the challenges and opportunities of integrating large language models with other AI technologies such as computer vision and speech recognition.