Abstract
Ensuring the accuracy of biomedical information is critical. Traditional validation methods are slow and error-prone, making automated solutions increasingly necessary. This study introduces LLM-FactCheck, a tool that is used to improve data quality in SemMedDB, a large biomedical database. Using LLMs such as Llama, GPT-4.0, GPT-3.5 Turbo, and Mistral, the system validates biomedical statements of subject-predicate-object triples. This approach increases the reliability of biomedical research by removing erroneous data input, ensuring that only accurate information is retained for further analysis. Comparisons of different LLMs show significant improvements in accuracy and efficiency. This research represents a significant step forward in automated fact-checking to improve the integrity of biomedical databases and support informed clinical and research decisions. The tool and additional resources related to this study can be found in the GitHub repository at https://github.com/KoslickiLab/LLMFactCheck.