IsaMini: Redesigned Isabelle Proof Lanugage for Machine Learning
Authors:
Preprint
Abstract
Neural Theorem Proving (NTP) employs deep learning methods, particularly Large Language Models (LLMs), to automate formal proofs in proof assistants. This approach holds promise for reducing the dramatic labor costs or computation costs required in proof engineering, which is fundamental to formal verification and other software engineering methods. The paper explores the potential of improving NTP by redesigning the proof language, given that LLMs' capabilities depend highly on representations. We introduce \emph{MiniLang}, a redesigned proof language for Isabelle/HOL incorporating an improved version of Sledgehammer. Experiments show MiniLang benefits two fine-tuned LLMs by improving the success rate on the PISA benchmark by up to 29\% in comparison to generation of Isar proof script. The success rate under one attempt (so-called \emph{pass@1}) reaches 69.1\%, exceeding the previous Baldur's pass@64 (65.7\%); The pass@8 reaches 79.2\%, exceeding the state-of-the-art on PISA (71.0\%) achieved by Magnushammer.Keywords
All Related Versions
This publication has 14 references indexed in Scilit:
- Lean-Auto: An Interface Between Lean 4 and Automated Theorem Provers2025
- lean-smt: An SMT Tactic for Discharging Proof Goals in Lean2025
- Towards Neural Synthesis for SMT-Assisted Proof-Oriented Programming2025
- Passport: Improving Automated Formal Verification Using IdentifiersACM Transactions on Programming Languages and Systems, 2023
- TacticToe: Learning to Prove with TacticsJournal of Automated Reasoning, 2020
- First Neural Conjecturing Datasets and Experiments2020
- Monte Carlo Tableau Proof SearchLecture Notes in Computer Science, 2017
- Extending Sledgehammer with SMT SolversJournal of Automated Reasoning, 2013
- MaSh: Machine Learning for SledgehammerLecture Notes in Computer Science, 2013
- Translating Higher-Order Clauses to First-Order ClausesJournal of Automated Reasoning, 2007