Data-Driven Function Calling Improvements in Large Language Model for Online Financial QA
Open Access
- 12 April 2026
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 7821-7832
Abstract
Large language models (LLMs) have been incorporated into numerous industrial applications. Meanwhile, a vast array of API assets is scattered across various functions in the financial domain. An online financial question-answering system can leverage both LLMs and private APIs to provide timely financial analysis and information. The key is equipping the LLM model with function calling capability tailored to a financial scenario. However, a generic LLM requires customized financial APIs to call and struggles to adapt to the financial domain. Additionally, online user queries are diverse and contain out-of-distribution parameters compared with the required function input parameters, which makes it more difficult for a generic LLM to serve online users. In this paper, we propose a data-driven pipeline to enhance function calling in LLM for our online, deployed financial QA, comprising dataset construction, data augmentation, and model training. Specifically, we construct a dataset based on a previous study and update it periodically, incorporating queries and an augmentation method named AugFC. The addition of user query-related samples will exploit our financial toolset in a data-driven manner, and AugFC explores the possible parameter values to enhance the diversity of our updated dataset. Then, we train an LLM with a two-step method, which enables the use of our financial functions. Extensive experiments on existing offline datasets, as well as the deployment of an online scenario, illustrate the superiority of our pipeline. The related pipeline has been adopted in the financial QA of YuanBao. https://yuanbao.tencent.com/chat/, one of the largest chat platforms in China.Keywords
All Related Versions
Funding Information
- Scientific Research Capacity Enhancement Program for Key Construction Disciplines in Guangdong Province (2024ZDJS063)
- Shenzhen Technology University School-level Research Project (20251061020002)
- The National Natural Science Foundation of China (NSFC) (62506238)
This publication has 13 references indexed in Scilit:
- DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learningNature, 2025
- Tool Learning in the Wild: Empowering Language Models as Automatic Tool Agents2025
- The rise and potential of large language model based agents: a surveyScience China Information Sciences, 2025
- ToolHop: A Query-Driven Benchmark for Evaluating Large Language Models in Multi-Hop Tool Use2025
- FinanceReasoning: Benchmarking Financial Numerical Reasoning More Credible, Comprehensive and Challenging2025
- xLAM: A Family of Large Action Models to Empower AI Agent Systems2025
- Enhancing Financial Question Answering with a Multi-Agent Reflection Framework2024
- Fine-tuning Smaller Language Models for Question Answering over Financial Documents2024
- FinTextQA: A Dataset for Long-form Financial Question Answering2024
- API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs2023