Cross-Provider Consistency of LLM-Based Semantic Enrichment for e-Science Workflows

Author:
Publisher website
Google Scholar
Abstract
Integrating large language models (LLMs) into data-intensive scientific workflows requires understanding their robustness, efficiency, and cross-provider consistency. This paper evaluates five LLMs (ChatGPT, Gemini, DeepSeek, Mistral, and Claude) as interchangeable components of a semantic enrichment pipeline, using 2,387 Billboard Year-End Hot 100 songs (2000–2023) as a case study. All models had near-perfect success rates (99.7–100%) but over 20-fold cost differences ($0.46–$10.70). Thematic agreement was substantial (Cohen’s Kappa = 0.70–0.81), and temporal trajectories were consistent across providers (Pearson r = 0.65–0.82). Thus, provider choice minimally affects aggregate analytical conclusions, allowing workflow designs that prioritize cost or latency without sacrificing reproducibility.

This publication has 9 references indexed in Scilit: