Application of Large Language Model-Driven Pairwise Constrained Clustering Fusion Algorithm in New Intent Discovery

Authors:
  • FZFeiyang Zhang
    Feiyang Zhang
  • XRXingge Ren
    Xingge Ren
  • JSJinmiao Song
    Jinmiao Song
  • XPXinjun Pei
    Xinjun Pei

Abstract
In open-world dialogue systems, new intent discovery (NID) aims to identify previously unseen user intents from raw utterances and expand the supported intent set. This process is traditionally time-consuming and often requires domain experts to explore data and define intent labels, which substantially increases the cost. To address this challenge, we propose SCNID, a semi-supervised framework that integrates contrastive representation learning with LLM-driven pairwise-constraint-guided clustering fusion. Specifically, SCNID learns discriminative intent representations via semi-supervised contrastive learning, and then performs clustering fusion by combining a Gaussian Mixture Model (GMM) with K-means++ to improve robustness and stability. Moreover, we leverage a large language model (LLM) to generate Must-Link and Cannot-Link constraints, providing structured semantic guidance for clustering. Extensive experiments on four benchmarks---three intent discovery datasets (Clinc150, Banking77, and StackOverflow20) and an additional cross-domain topic dataset (DBPedia14)---demonstrate that SCNID achieves strong and stable performance under diverse data distributions, validating the effectiveness of the proposed approach.

This publication has 14 references indexed in Scilit: