A Joint BERT-based Approach with Explicitly Incorporated Slot into Intent
Authors:
- 3 November 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 109-112
Abstract
Intent detection and slot filling are two prominent tasks to capture the conversational utterance. Extant studies have shown that joint learning models can promote the two tasks mutually. However, the previous joint models mainly think over the guideline of intent detection for slot filling and show little improvement about the opposite way. This study fully considers the guidance of slot filling applying on intent detection and proposes a joint approach on the basis of BERT (Bidirectional Encoder Representation from Transformers) with max pooling layer and CRF (Conditional Random Field) for intent detection and slot filling. The joint approach fuses the max-pooled slot embedding representation with the beginning of the sentence token vector for intent detection and replaces the softmax layer with CRF for slot filling. The comparative experiments by using two common public datasets demonstrate that the proposed joint model achieve competitive performance for both tasks.Keywords
This publication has 13 references indexed in Scilit:
- Joint agricultural intent detection and slot filling based on enhanced heterogeneous attention mechanismComputers and Electronics in Agriculture, 2023
- Anti-Drugs Chatbot: Chinese BERT-Based Cognitive Intent AnalysisIEEE Transactions on Computational Social Systems, 2023
- Incorporating BERT With Probability-Aware Gate for Spoken Language UnderstandingIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2023
- Unsupervised cross-domain rolling bearing fault diagnosis based on time-frequency information fusionJournal of the Franklin Institute, 2022
- Machine learning based iterative learning control for non‐repetitive time‐varying systemsInternational Journal of Robust and Nonlinear Control, 2022
- Evaluation of BERT and ALBERT Sentence Embedding Performance on Downstream NLP Tasks2021
- Collaborative Production Planning with Unknown Parameters using Model Predictive Control and Machine Learning2020
- Multi-turn intent determination and slot filling with neural networks and regular expressionsKnowledge-Based Systems, 2020
- An unsupervised fault diagnosis method for rolling bearing using STFT and generative neural networksJournal of the Franklin Institute, 2020
- Machine learning based decision making for time varying systems: Parameter estimation and performance optimizationKnowledge-Based Systems, 2020