Proton: Probing Schema Linking Information from Pre-trained Language Models for Text-to-SQL Parsing
Authors:
- Lihan WangLWLihan Wang
- Bowen QinBQBowen Qin
- Binyuan HuiBHBinyuan Hui
- Bowen LiBLBowen Li
- Min YangMYMin Yang
- Bailin WangBWBailin Wang
- Binhua LiBLBinhua Li
- Fei HuangFHFei Huang
- Luo SiLSLuo Si
- Yongbin LiYLYongbin Li
Preprint
Abstract
The importance of building text-to-SQL parsers which can be applied to new databases has long been acknowledged, and a critical step to achieve this goal is schema linking, i.e., properly recognizing mentions of unseen columns or tables when generating SQLs. In this work, we propose a novel framework to elicit relational structures from large-scale pre-trained language models (PLMs) via a probing procedure based on Poincar\'e distance metric, and use the induced relations to augment current graph-based parsers for better schema linking. Compared with commonly-used rule-based methods for schema linking, we found that probing relations can robustly capture semantic correspondences, even when surface forms of mentions and entities differ. Moreover, our probing procedure is entirely unsupervised and requires no additional parameters. Extensive experiments show that our framework sets new state-of-the-art performance on three benchmarks. We empirically verify that our probing procedure can indeed find desired relational structures through qualitative analysis. Our code can be found at https://github.com/AlibabaResearch/DAMO-ConvAI.Keywords
All Related Versions
This publication has 34 references indexed in Scilit:
- Unified Dialog Model Pre-training for Task-Oriented Dialog Understanding and Generation2022
- GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-supervised Learning and Explicit Policy InjectionProceedings of the AAAI Conference on Artificial Intelligence, 2022
- RYANSQL: Recursively Applying Sketch-based Slot Fillings for Complex Text-to-SQL in Cross-Domain DatabasesComputational Linguistics, 2021
- Dynamic Hybrid Relation Exploration Network for Cross-Domain Context-Dependent Semantic ParsingProceedings of the AAAI Conference on Artificial Intelligence, 2021
- Beyond I.I.D.: Three Levels of Generalization for Question Answering on Knowledge Bases2021
- A Primer in BERTology: What We Know About How BERT WorksTransactions of the Association for Computational Linguistics, 2020
- Heads-up! Unsupervised Constituency Parsing via Self-Attention Heads2020
- Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT2020
- Learning Low-Resource End-To-End Goal-Oriented Dialog for Fast and Reliable System Deployment2020
- Hyperbolic geometry of complex networksPhysical Review E, 2010