BSBurcu Sayin

Burcu Sayin

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Burcu Sayin has a total of 38 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Multimodal large language models and dental students in radiographic landmark identification: a novel grid-based assessment

    by Mehmet Egemen Aydemir, Burcu Sayin, Burcu Yeliz Kollayan, Can Günel, Kadir Cem, Mehmet Ali Gül

    Objectives: To introduce a novel grid-based coordinate framework for evaluating spatial landmark localization competence in multimodal large language models (MLLMs) applied to dental radiography, and to benchmark 2 MLLMs against dental student consensus. Methods: GPT-5.4 and Gemini 3.1 Pro were evaluated on 900 queries spanning 200 dental radiographs (100 panoramic, 50 periapical, 50 cephalometric) and 12 landmarks (9 point, 3 area) under zero-shot and guided prompting across 3 repetitions (10 800 API calls). Performance was scored against a 2-rater adjudicated ground truth and a team-adjudicated fourth-year dental student consensus (n = 40). Metrics included Euclidean distance, the successful detection rate, and the Jaccard index. Results: GPT-5.4 matched student-level performance on cephalometric landmarks but was substantially outperformed on periapical and panoramic tasks. Gemini 3.1 Pro outperformed GPT-5.4 on panoramic and periapical landmarks, while GPT-5.4 retained an advantage on cephalometric points. Guided prompting produced heterogeneous effects, improving localization of some landmarks while causing severe regression on others; most notably, GPT-5.4 directed the majority of lower-left canine apex predictions to the incorrect lower-right molar region, a prompt-resistant mislocalization pattern absent in Gemini 3.1 Pro. Both models remained inferior to the student consensus on all periapical and panoramic comparisons, with Gemini 3.1 Pro approaching parity only on cephalometric points under guided prompting. Conclusion: Current MLLMs demonstrate modality-specific spatial competence insufficient for autonomous clinical deployment. The grid-based framework provides a reproducible, modality-agnostic benchmark for tracking MLLM spatial reasoning across future model generations.

  • Zoom In Disparities in Healthcare LLM Q&A

    by Ipek Baris Schlicht, Burcu Sayin, Zhixue Zhao, Frederik M. Labonté, Cesare Barbera, Marco Viviani, Paolo Rosso, Lucie Flek

  • Open Access

    Human–LLM Dialogue Improves Diagnostic Accuracy in Emergency Care: Controlled Crossover Study (Preprint)

    by Burcu Sayin, Ngoc Vo Hong, Ipek Baris Schlicht, Jacopo Staiano, Pasquale Minervini, Sara Allievi, Nicola Susca, Nicola Osti, Alberto Maino, Vito Racanelli, et al.

    BACKGROUND Clinical decision-making in emergency medicine requires rapid, accurate diagnoses under uncertainty. Large language models (LLMs) have shown diagnostic potential, but evidence for their benefit in real physician workflows remains limited. OBJECTIVE This study evaluated whether MedSyn, a physician–LLM dialogue system, improves diagnostic accuracy in live emergency care workflows, and examined how benefit varies with case difficulty and physician expertise. METHODS Seven physicians (three senior internists, four residents) completed unassisted (baseline) and AI-assisted (interactive) sessions across 52 MIMIC-IV cases stratified by difficulty (Easy, Medium, Hard). Diagnostic performance was measured using automated fuzzy-matching metrics (any-match accuracy, exact-match accuracy, diagnosis-set F1) and blinded expert evaluation. Between-condition comparisons used paired bootstrap resampling (20,000 replicates, seed 42). RESULTS Interactive support significantly improved overall diagnostic accuracy: difficulty-standardised any-match accuracy improved by 0.156 (P < .001; d = 1.25). Residents showed the largest gains, with F1 improving by 0.138 (P < .001; d = 0.94). Blinded evaluation revealed residents’ Hard-case mean correctness rose from 0.589 to 0.734 (+0.145); difficulty-standardised completely-correct rates confirmed a medium effect (Δ = 0.092; P = .07; d = 0.47). Automated and blinded manual assessment showed only modest agreement (Cohen's κ = 0.148), with 45% of clinically correct diagnoses receiving no automated credit; reported F1 improvements are therefore conservative estimates. Cross-expertise concordance increased significantly (Δ = 0.142; P < .001; d = 1.48). Dialogue analysis revealed expertise-dependent strategies: seniors employed targeted, hypothesis-driven queries while residents used broader exploratory questioning. CONCLUSIONS Interactive LLM assistance meaningfully enhances diagnostic accuracy, particularly for resident physicians on complex cases. The expertise-stratified benefit profile, large effect sizes, and significant convergence in cross-expertise concordance suggest that adaptive LLM deployment, targeting high-complexity cases and less experienced clinicians, is a promising and evidence-grounded strategy for emergency diagnostic support.

  • Open Access

    Bringing Machine Learning Classifiers Into Critical Cyber-Physical Systems: A Matter of Design

    by Burcu Sayin, Tommaso Zoppi, Nicolò Marchini, Fahad Ahmed Khokhar, Andrea Passerini

    Domain experts are striving to solve classification problems through the application of Machine Learning (ML) models, aiming for the highest accuracy. However, classifiers are inherently prone to misclassifications, especially when encountering unknown input data. This complicates their deployment in critical Cyber-Physical Systems (CPSs), where misclassifications can have detrimental effects on people, the environment, or infrastructure. This paper argues that ML classifiers should not be designed or evaluated in isolation but rather conceptualized as Critical System Classifiers (CSCs), which can reject uncertain predictions and trigger mitigation strategies in accordance to the encompassing CPS. We present an high-level architecture for CSCs that supports black-box integration, preprocessing for unknown detection, post-hoc calibration, and cost-sensitive thresholding, emphasizing the importance of evaluating classifiers using cost-aware metrics that account for rejections. We validate our proposal through experiments on tabular datasets related to failure prediction, intrusion and error detection, which are common applications of classifiers within CPSs. Key findings are: i) cost-sensitive evaluation promotes the usage of different classifiers compared to standard performance metrics, ii) tree-based models outperform statistical ones, classification-wise, iii) prediction calibration and rejection strategies provide a robust notion of confidence, and iv) combining multiple uncertainty-based rejectors achieves a favorable balance between high accuracy, low rejection rate, and cost. The analyses above were conducted using our publicly available CINNABAR GitHub repository for experimentations with CSCs. Overall, our study offers a system-level perspective, software architecture and implementation on classifier deployment in critical CPSs domains, paving the way for a trustworthy ML integration into real-world infrastructures.


  • Open Access

    Rethinking and recomputing the value of machine learning models

    by Burcu Sayin, Jie Yang, Xinyue Chen, Andrea Passerini, Fabio Casati

    In this paper, we argue that the prevailing approach to training and evaluating machine learning models often fails to consider their real-world application within organizational or societal contexts, where they are intended to create beneficial value for people. We propose a shift in perspective, redefining model assessment and selection to emphasize integration into workflows that combine machine predictions with human expertise, particularly in scenarios requiring human intervention for low-confidence predictions. Traditional metrics like accuracy and f-score fail to capture the beneficial value of models in such hybrid settings. To address this, we introduce a simple yet theoretically sound “value” metric that incorporates task-specific costs for correct predictions, errors, and rejections, offering a practical framework for real-world evaluation. Through extensive experiments, we show that existing metrics fail to capture real-world needs, often leading to suboptimal choices in terms of value when used to rank classifiers. Furthermore, we emphasize the critical role of calibration in determining model value, showing that simple, well-calibrated models can often outperform more complex models that are challenging to calibrate.


  • Do LLMs Provide Consistent Answers to Health-Related Questions Across Languages?

    by Ipek Baris Schlicht, Zhixue Zhao, Burcu Sayin, Lucie Flek, Paolo Rosso


  • Open Access

    Do LLMs Provide Consistent Answers to Health-Related Questions across Languages?

    by Ipek Baris Schlicht, Zhixue Zhao, Burcu Sayin, Lucie Flek, Paolo Rosso

    Equitable access to reliable health information is vital for public health, but the quality of online health resources varies by language, raising concerns about inconsistencies in Large Language Models (LLMs) for healthcare. In this study, we examine the consistency of responses provided by LLMs to health-related questions across English, German, Turkish, and Chinese. We largely expand the HealthFC dataset by categorizing health-related questions by disease type and broadening its multilingual scope with Turkish and Chinese translations. We reveal significant inconsistencies in responses that could spread healthcare misinformation. Our main contributions are 1) a multilingual health-related inquiry dataset with meta-information on disease categories, and 2) a novel prompt-based evaluation workflow that enables sub-dimensional comparisons between two languages through parsing. Our findings highlight key challenges in deploying LLM-based tools in multilingual contexts and emphasize the need for improved cross-lingual alignment to ensure accurate and equitable healthcare information.


  • Open Access

    Rethinking and Recomputing the Value of Machine Learning Models

    by Burcu Sayin, Jie Yang, Xinyue Chen, Andrea Passerini, Fabio Casati

    In this paper, we contend that the conventional approach to training and evaluating machine learning models frequently overlooks their application within the real-world organizational or societal contexts they are meant to serve. By shifting to this perspective, we redefine how we assess and choose machine learning models. Our focus is particularly on integrating these models into practical workflows that involve both machines and human experts, with human intervention occurring when machines lack sufficient confidence in their predictions. We demonstrate that traditional metrics such as accuracy and f-score fall short in capturing the true value of machine learning models in such hybrid settings. To address this issue, we introduce a simple but theoretically sound strategy to adapt existing machine learning models so as to maximize value. An extensive experimental evaluation highlights the importance of the value-based perspective in evaluating models, and the impact of calibration and out-of-distribution settings on model value.

  • Open Access

    Value-Based Hybrid Intelligence

    by Burcu Sayin, Jie Yang, Andrea Passerini, Fabio Casati

    In this paper, we argue that the way we have been training and evaluating ML models has largely forgotten the fact that they are applied in an organization or societal context as they provide value to people. We show that with this perspective we fundamentally change how we evaluate and select machine learning models.


  • Open Access

    Perspective: Leveraging Human Understanding for Identifying and Characterizing Image Atypicality

    by Shahin Sharifi Noorian, Sihang Qiu, Burcu Sayin, Agathe Balayn, Ujwal Gadiraju, Jie Yang, Alessandro Bozzon

    High-quality data plays a vital role in developing reliable image classification models. Despite that, what makes an image difficult to classify remains an unstudied topic. This paper provides a first-of-its-kind, model-agnostic characterization of image atypicality based on human understanding. We consider the setting of image classification “in the wild”, where a large number of unlabeled images are accessible, and introduce a scalable and effective human computation approach for proactive identification and characterization of atypical images. Our approach consists of i) an image atypicality identification and characterization task that presents to the human worker both a local view of visually similar images and a global view of images from the class of interest and ii) an automatic image sampling method that selects a diverse set of atypical images based on both visual and semantic features. We demonstrate the effectiveness and cost-efficiency of our approach through controlled crowdsourcing experiments and provide a characterization of image atypicality based on human annotations of 10K images. We showcase the utility of the identified atypical images by testing state-of-the-art image classification services against such images and provide an in-depth comparative analysis of the alignment between human- and machine-perceived image atypicality. Our findings have important implications for developing and deploying reliable image classification systems.