SPARQL-LLM: Real-Time SPARQL Query Generation from Natural Language Questions
Open Access
- 15 October 2026
- journal article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on the Web
- Vol. 20 (4) , 1-21
Abstract
The advent of large language models is contributing to the emergence of novel approaches that promise to better tackle the challenge of generating structured queries, such as SPARQL queries, from natural language. However, these new approaches mostly focus on response accuracy while ignoring other evaluation criteria, such as runtime and cost to generate SPARQL queries. Consequently, they are often not production-ready or easy to deploy over real-world knowledge graphs with good accuracy. To mitigate these issues, in this article, we describe and systematically evaluate SPARQL-LLM, an open-source and triplestore-agnostic approach, powered by lightweight metadata, that generates SPARQL queries from natural language text. First, we describe its architecture, which consists of dedicated components for metadata indexing, prompt building, and query generation and execution. Then, we evaluate it based on a state-of-the-art challenge with multilingual questions, and a collection of questions from three of the most prevalent knowledge graphs within the field of bioinformatics. Our results demonstrate a substantial improvement of up to 59% in F1 score over the second-best system participating in the challenge, adaptability to high-resource languages such as English, Spanish, and German, as well as ability to form complex bioinformatics queries. Furthermore, our results show that our system is up to 27× faster than the second-best system participating in the challenge, while costing a maximum of $0.01 per question, making it suitable for real-time, low-cost text-to-SPARQL applications. SPARQL-LLM is publicly released as an open-source project at https://github.com/sib-swiss/sparql-llm and is currently deployed over real-world decentralized knowledge graphs at https://www.expasy.org/chat .Keywords
All Related Versions
Funding Information
- HORIZON EUROPE Research Infrastructures (101188179)
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (10.002.786)
This publication has 21 references indexed in Scilit:
- PYTHIA: Universal Question Answering over Knowledge Graphs2026
- 12 Shades of RDF: Impact of Syntaxes on Data Extraction with Language Models2025
- A large collection of bioinformatics question–query pairs over federated knowledge graphs: methodology and applicationsGigaScience, 2025
- UniProt: the Universal Protein Knowledgebase in 2025Nucleic Acids Research, 2024
- Lost in the Middle: How Language Models Use Long ContextsTransactions of the Association for Computational Linguistics, 2024
- QALD-9-plus: A Multilingual Dataset for Question Answering over DBpedia and Wikidata Translated by Native Speakers2022
- The Bgee suite: integrated curated expression atlas and comparative transcriptomics in animalsNucleic Acids Research, 2020
- LC-QuAD: A Corpus for Complex Question Answering over Knowledge Graphs2017
- Core techniques of question answering systems over knowledge bases: a surveyKnowledge and Information Systems, 2017
- SPARQL Assist language-neutral query composerBMC Bioinformatics, 2012