Partial Sequence Labeling With Structured Gaussian Processes
- 25 July 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 35 (2) , 2783-2792
Abstract
Existing partial sequence labeling models mainly focus on a max-margin framework that fails to provide an uncertainty estimation of the prediction. Furthermore, the unique ground-truth disambiguation strategy employed by these models may include wrong label information for parameter learning. In this article, we propose structured Gaussian processes for partial sequence labeling (SGPPSL), which encodes uncertainty in the prediction and does not need extra effort for model selection and hyperparameter learning. The model employs factor-as-piece approximation that divides the linear-chain graph structure into the set of pieces, which preserves the basic Markov random field structure and effectively avoids handling a large number of candidate output sequences generated by partially annotated data. Then, confidence measure is introduced in the model to address different contributions of candidate labels, which enables the ground-truth label information to be utilized in parameter learning. Based on the derived lower bound of the variational lower bound of the proposed model, variational parameters and confidence measures are estimated in the framework of alternating optimization. Moreover, a weighted Viterbi algorithm is proposed to incorporate confidence measures to sequence prediction, which considers label ambiguity arose from multiple annotations in the training data and thus helps improve the performance. SGPPSL is evaluated on several sequence labeling tasks and the experimental results show the effectiveness of the proposed model.Keywords
All Related Versions
Funding Information
- Hong Kong Government GRF CityU (11216620)
This publication has 27 references indexed in Scilit:
- GPstruct: Bayesian Structured Prediction Using Gaussian ProcessesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2014
- Structured Output Learning with Candidate Labels for Local PartsLecture Notes in Computer Science, 2013
- Gaussian Variational Approximate Inference for Generalized Linear Mixed ModelsJournal of Computational and Graphical Statistics, 2012
- Piecewise training for structured predictionMachine Learning, 2009
- Semi-supervised learning for structured output variables2006
- Semi-supervised conditional random fields for improved sequence segmentation and labeling2006
- Support vector machine learning for interdependent and structured output spaces2004
- Introduction to the CoNLL-2000 shared task2000
- Text Chunking Using Transformation-Based Learning1999
- Hidden Markov Models for Speech RecognitionTechnometrics, 1991