Sequence clustering and labeling for unsupervised query intent discovery
Authors:
Open Access
- 8 February 2012
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 383-392
Abstract
One popular form of semantic search observed in several modern search engines is to recognize query patterns that trigger instant answers or domain-specific search, producing semantically enriched search results. This often requires understanding the query intent in addition to the meaning of the query terms in order to access structured data sources. A major challenge in intent understanding is to construct a domain-dependent schema and to annotate search queries based on such a schema, a process that to date has required much manual annotation effort. We present an unsupervised method for clustering queries with similar intent and for producing a pattern consisting of a sequence of semantic concepts and/or lexical items for each intent. Furthermore, we leverage the discovered intent patterns to automatically annotate a large number of queries beyond those used in clustering. We evaluated our method on 10 selected domains, discovering over 1400 intent patterns and automatically annotating 125K (and potentially many more) queries. We found that over 90% of patterns and 80% of instance annotations tested are judged to be correct by a majority of annotators.Keywords
This publication has 15 references indexed in Scilit:
- Sequential Labeling Using Deep-Structured Conditional Random FieldsIEEE Journal of Selected Topics in Signal Processing, 2010
- Structured annotations of web queries2010
- Extracting structured information from user queries with semi-supervised conditional random fields2009
- Ontologizing semantic relations2006
- Enriching Web taxonomies through subject categorization of query terms from search engine logsDecision Support Systems, 2003
- Concept discovery from text2002
- Clustering user queries of a search engine2001
- Agglomerative clustering of a search engine query log2000
- Data clusteringACM Computing Surveys, 1999
- Automatic acquisition of hyponyms from large text corpora1992