JHJia-Hong Huang

Jia-Hong Huang

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Jia-Hong Huang has a total of 47 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Keeping the Evidence Chain: Semantic Evidence Allocation for Training-Free Token Pruning in Video Temporal Grounding

    by Jiaqi Li, Shuntian Zheng, Yixian Shen, Jia-Hong Huang, Xiaoman Lu, Minzhe Ni, Yu Guan

  • Open Access

    A-MAR: Agent-based Multimodal Art Retrieval for Fine-Grained Artwork Understanding

    by Shuai Wang, Hongyi Zhu, Jiahong Huang, Yixian Shen, Chengxi Zeng, Stevan Rudinac, Monika Kackovic, Nachoem Wijnberg, Marcel Worring

  • Open Access

    Agent-Based Query Reformulation: Simulating Feedback and Mitigating Negation Blindness in Interactive Image Retrieval

    by Hongyi Zhu, Shuai Wang, Jia-Hong Huang, Yixian Shen, Stevan Rudinac, Evangelos Kanoulas

  • Top Cited Papers
    Open Access

    Conditional Modeling-Based Automatic Video Summarization

    by Jia-Hong Huang, Chao-Han Huck Yang, Pin-Yu Chen, Min-Hung Chen, Marcel Worring

    The aim of video summarization is to shorten videos automatically while retaining the key information necessary to convey the overall story. Video summarization methods mainly rely on visual factors, such as visual consecutiveness and diversity, which may not be sufficient to fully understand the content of the video. There are other non-visual factors, such as interestingness, representativeness, and storyline consistency that should also be considered for generating high-quality video summaries. Current methods do not adequately take into account these non-visual factors, resulting in suboptimal performance. In this work, a new approach to video summarization is proposed based on insights gained from how humans create ground truth video summaries. The method utilizes a conditional modeling perspective and introduces multiple meaningful random variables and joint distributions to characterize the key components of video summarization. Helper distributions are employed to improve the training of the model. A conditional attention module is designed to mitigate potential performance degradation in the presence of multi-modal input. The proposed video summarization method incorporates the above innovative design choices that aim to narrow the gap between human-generated and machine-generated video summaries. Extensive experiments show that the proposed approach outperforms existing methods and achieves state-of-the-art performance on commonly used video summarization datasets.


  • Open Access

    Interactive Image Retrieval Meets Query Rewriting with Large Language and Vision Language Models

    by Hongyi Zhu, Jia-Hong Huang, Yixian Shen, Stevan Rudinac, Evangelos Kanoulas

    Image search stands as a pivotal task in multimedia and computer vision, finding applications across diverse domains, ranging from internet search to medical diagnostics. Conventional image search systems operate by accepting textual or visual queries, retrieving the top-relevant candidate results from the database. However, prevalent methods often rely on single-turn procedures, introducing potential inaccuracies and limited recall. These methods also face the challenges, such as vocabulary mismatch and the semantic gap, constraining their overall effectiveness. To address these issues, we propose an interactive image retrieval system capable of refining queries based on user relevance feedback in a multi-turn setting. This system incorporates an image captioner based on vision language model (VLM) to enhance the quality of text-based queries, resulting in more informative queries with each iteration. Moreover, we introduce a denoiser based on large language model (LLM) to refine text-based query expansions, mitigating inaccuracies in image descriptions generated by captioning models. To evaluate our system, we curate a new dataset by adapting the MSR-VTT and MSVD video retrieval datasets to the image retrieval task, offering multiple relevant ground truth images for each query. Through comprehensive experiments, we validate the effectiveness of our proposed system against baseline methods, achieving state-of-the-art performance with a notable 10% improvement in terms of recall. Our contributions encompass the development of an innovative interactive image retrieval system, the integration of an LLM-based denoiser, the curation of a meticulously designed evaluation dataset, and thorough experimental validation.


  • Open Access

    Optimizing Numerical Estimation and Operational Efficiency in the Legal Domain through Large Language Models

    by Jia-Hong Huang, Chao-Chun Yang, Yixian Shen, Alessio M. Pacces, Evangelos Kanoulas

    The legal landscape encompasses a wide array of lawsuit types, presenting lawyers with challenges in delivering timely and accurate information to clients, particularly concerning critical aspects like potential imprisonment duration or financial repercussions. Compounded by the scarcity of legal experts, there's an urgent need to enhance the efficiency of traditional legal workflows. Recent advances in deep learning, especially Large Language Models (LLMs), offer promising solutions to this challenge. Leveraging LLMs' mathematical reasoning capabilities, we propose a novel approach integrating LLM-based methodologies with specially designed prompts to address precision requirements in legal Artificial Intelligence (LegalAI) applications. The proposed work seeks to bridge the gap between traditional legal practices and modern technological advancements, paving the way for a more accessible, efficient, and equitable legal system. To validate this method, we introduce a curated dataset tailored to precision-oriented LegalAI tasks, serving as a benchmark for evaluating LLM-based approaches. Extensive experimentation confirms the efficacy of our methodology in generating accurate numerical estimates within the legal domain, emphasizing the role of LLMs in streamlining legal processes and meeting the evolving demands of LegalAI. Github: https://github.com/Jhhuangkay/Optimizing-Numerical-Estimation-and-Operational-Efficiency-in-the-Legal-Domain.


  • Open Access

    Multi-modal Video Summarization

    by Jia-Hong Huang

    The increasing prevalence of video content on platforms like YouTube and Vimeo has led to a growing demand for effective video summarization techniques. These methods aim to extract relevant content from videos and present it in a condensed form, addressing the challenge of information overload faced by users. However, the quality of the generated summaries is crucial, especially in fields like forensics, journalism, and sports analysis, where accurate and concise summaries are essential for decision-making and analysis. Traditional video summarization methods often rely solely on visual information, limiting their ability to capture textual cues and emotional content present in videos. To overcome this limitation, this thesis proposes leveraging text-based queries as context to enhance the effectiveness of video summarization. By incorporating textual information alongside visual data, the proposed approach aims to generate query-dependent video summaries that better align with users' requirements. Furthermore, the thesis leverages a novel conditional modeling perspective to impart a more human-like quality to the video summarization process. These methods hold promise for various applications, including documentary filmmaking and educational content creation. Additionally, the thesis addresses the challenge of data scarcity in video summarization by proposing a self-supervised learning approach that leverages pretext tasks to generate pseudo-labels for model training. In 2020, we proposed a query-controllable video summarization technology, which is elaborated upon in the second chapter of this thesis. Notably, Google has adopted a similar feature, integrating it into their large language model (LLM) Gemini Pro-1.5 version, in 2024.


  • Open Access

    Enhancing Interactive Image Retrieval With Query Rewriting Using Large Language Models and Vision Language Models

    by Hongyi Zhu, Jia-Hong Huang, Stevan Rudinac, Evangelos Kanoulas

    Image search stands as a pivotal task in multimedia and computer vision, finding applications across diverse domains, ranging from internet search to medical diagnostics. Conventional image search systems operate by accepting textual or visual queries, retrieving the top-relevant candidate results from the database. However, prevalent methods often rely on single-turn procedures, introducing potential inaccuracies and limited recall. These methods also face the challenges, such as vocabulary mismatch and the semantic gap, constraining their overall effectiveness. To address these issues, we propose an interactive image retrieval system capable of refining queries based on user relevance feedback in a multi-turn setting. This system incorporates a vision language model (VLM) based image captioner to enhance the quality of text-based queries, resulting in more informative queries with each iteration. Moreover, we introduce a large language model (LLM) based denoiser to refine text-based query expansions, mitigating inaccuracies in image descriptions generated by captioning models. To evaluate our system, we curate a new dataset by adapting the MSR-VTT video retrieval dataset to the image retrieval task, offering multiple relevant ground truth images for each query. Through comprehensive experiments, we validate the effectiveness of our proposed system against baseline methods, achieving state-of-the-art performance with a notable 10% improvement in terms of recall. Our contributions encompass the development of an innovative interactive image retrieval system, the integration of an LLM-based denoiser, the curation of a meticulously designed evaluation dataset, and thorough experimental validation.