Proton: Probing Schema Linking Information from Pre-trained Language Models for Text-to-SQL Parsing
Authors:
Open Access
- 14 August 2022
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1889-1898
Abstract
The importance of building text-to-SQL parsers which can be applied to new databases has long been acknowledged, and a critical step to achieve this goal is schema linking, i.e., properly recognizing mentions of unseen columns or tables when generating SQLs. In this work, we propose a novel framework to elicit relational structures from large-scale pre-trained language models (PLMs) via a probing procedure based on Poincaré distance metric, and use the induced relations to augment current graph-based parsers for better schema linking. Compared with commonly-used rule-based methods for schema linking, we found that probing relations can robustly capture semantic correspondences, even when surface forms of mentions and entities differ. Moreover, our probing procedure is entirely unsupervised and requires no additional parameters. Extensive experiments show that our framework sets new state-of-the-art performance on three benchmarks. We empirically verify that our probing procedure can indeed find desired relational structures through qualitative analysis.Keywords
All Related Versions
Funding Information
- Shenzhen Basic Research Foundation (JCYJ20210324115614039, JCYJ20200109113441941)
- Shenzhen Science and Technology Innovation Program (KQTD20190929172835662)
- Youth Innovation Promotion Association of CAS China (2020357)
- National Natural Science Foundation of China (61906185)
This publication has 14 references indexed in Scilit:
- Beyond I.I.D.: Three Levels of Generalization for Question Answering on Knowledge Bases2021
- A Primer in BERTology: What We Know About How BERT WorksTransactions of the Association for Computational Linguistics, 2020
- RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers2020
- Language Models as Knowledge Bases?2019
- Designing and Interpreting Probes with Control Tasks2019
- Revealing the Dark Secrets of BERT2019
- Representing Schema Structure with Graph Neural Networks for Text-to-SQL Parsing2019
- Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representation2019
- What do Neural Machine Translation Models Learn about Morphology?2017
- Hyperbolic geometry of complex networksPhysical Review E, 2010