Michał Młodawski

Clinical decision support systems face a fundamental challenge in the form of incomplete medical data. This study presents a methodology for designing an expert system for classifying disease entities according to ICD-10. The system operates under conditions of incomplete clinical data and diagnostic uncertainty. A total of 1,655 clinical cases from internal medicine (n = 616) and pediatrics (n = 1,039) were analyzed. Nineteen classification algorithms were compared. A two-stage diagnostic model (TOP-1 and TOP-3), reflecting clinical practice, was applied. The main hypothesis assumed that machine learning algorithms can effectively compensate for diagnostic uncertainty through mechanisms of differential diagnosis. In internal medicine, Gradient Boosting achieved the highest performance (84.74% accuracy), whereas in pediatrics the Multi-Layer Perceptron performed best (87.20% accuracy). Pediatrics demonstrated higher overall performance (mean TOP-3: 81.99%) than internal medicine (77.43%). The TOP-3 mechanism ensured a probability of identifying the correct diagnosis exceeding 94% in pediatrics and 91% in internal medicine, which substantially enhances the clinical safety of the system. The identification of different optimal algorithms for each specialty confirms the need to tailor modeling strategies to the clinical context. On external validation on 120 cases from a separate facility, treated as the primary benchmark of real-world performance, TOP-1 accuracy was 73.33% (internal medicine) and 78.33% (pediatrics), with TOP-3 accuracy of 86.67% and 90.00% respectively, and SHAP and LIME confirming clinically relevant predictions. The TOP-3 methodology developed constitutes an important contribution to the design of decision support systems under limited data availability.

Background/Aim: Although large language models (LLMs) achieve >90% accuracy on medical multiple-choice questions, their actual diagnostic utility remains unproven. This study compared four approaches to general medical diagnostics using real clinical data and LLMs: standalone LLM, LLM with Retrieval-Augmented Generation (RAG), LLM with expert system, and full integration of LLM, RAG, and expert system. Methods: Twenty LLMs were tested on 1655 unpublished clinical cases (paediatrics and internal medicine) in Polish and English, yielding 264,800 diagnostic evaluations. TOP-1, TOP-3, and grouped International Classification of Diseases, 10th Revision (ICD-10) accuracy were measured. Results: Standalone LLMs achieved only 16–20% TOP-1 accuracy. RAG added minimal benefit (+2–5 percentage points). The expert system improved performance 2.5-fold (55% paediatrics, 39% internal medicine). Unexpectedly, combining all components reduced results compared to the expert system alone, revealing an “integration paradox”. Conclusions: LLMs alone are insufficient for clinical diagnostics in practical applications with incomplete data. Developers of clinical decision support systems should not treat LLMs as standalone diagnostic engines. Expert systems based on machine learning algorithms provide better support and should serve as the primary component in hybrid architectures. Combining LLM, RAG, and expert systems without deliberate output weighting paradoxically reduces performance. Hybrid systems should implement dynamic source selection or prediction weighting mechanisms rather than simple integration.

The article presents the Parrot AI smart system, which supports physicians specialising in internal medicine and pediatrics. The solution is designed to automate time-consuming administrative processes and assist physicians in clinical decision-making. The main purpose of the system is to relieve medical staff of administrative tasks, such as manually filling out medical records, which translates into increased work efficiency and more time to focus on direct contact with patients. Parrot AI uses advanced natural language processing methods and generative language models to enable automated semantic analysis of medical interviews. Based on this, the system generates the content necessary to complete the electronic medical records and assists the physician by suggesting preliminary diagnoses and treatment options. Interaction with the patient can take place both at the initial medical interview stage (via chatbot or voicebot) and during a personal appointment at the clinic. Functional tests have demonstrated that the solution is highly effective. All tested components of the system achieved results well above the 80% effectiveness threshold, confirming the validity of further implementing the system into everyday clinical practice. Parrot AI has the potential not only to streamline physicians’ work, but also to improve the quality and consistency of medical records.