Tobi Olatunji
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Tobi Olatunji has a total of 135 co-authors. Here are the top co-authors by number of shared publications:
Mardhiyah Sanni
Intron Health
6 shared publicationsChibuzor Okocha
BioRAMP
5 shared publicationsEmmanuel Ayodele
Intron Health
5 shared publicationsTassallah Abdullahi
Brown University
5 shared publicationsFolafunmi Omofoye
BioRAMP
4 shared publications- Adeline Adwoa Boatin
Department of Obstetrics and Gynecology Massachusetts General Hospital, Harvard Medical School Boston Massachusetts USA
3 shared publications
Latest Publications
- Open Access
- 24 September 2026
- Published by JMIR Publications Inc. in JMIR Preprints
From idea to impact: A roadmap to bring digital health interventions to scale in low- and middle-income countries (Preprint)
UNSTRUCTURED Digital health innnovations have potential to improve health care, particularly in low-and middle income countries, yet few progress from concepts to implementation at scale. We present the Digital Intervetions Idea to Impact (DI3) roadmap, as a practical roadmap that maps the interconnected steps require to move digital health tools and interventions from initial concept through development, evaluation, health system integration and scale. Developed by a multi-disciplianry, cross-sector group with experience advancing digital health innovations across Africa, DI3, is intended to help diverse stakeholders anticipate requirements beyond their individual areas of expertise, identify gaps that may impede progress and coordinate more effectively toward sustainable and equitable health impact.
- 24 September 2026
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessFrom idea to impact: A roadmap to bring digital health interventions to scale in low- and middle-income countries (Preprint)
UNSTRUCTURED Digital health innnovations have potential to improve health care, particularly in low-and middle income countries, yet few progress from concepts to implementation at scale. We present the Digital Intervetions Idea to Impact (DI3) roadmap, as a practical roadmap that maps the interconnected steps require to move digital health tools and interventions from initial concept through development, evaluation, health system integration and scale. Developed by a multi-disciplianry, cross-sector group with experience advancing digital health innovations across Africa, DI3, is intended to help diverse stakeholders anticipate requirements beyond their individual areas of expertise, identify gaps that may impede progress and coordinate more effectively toward sustainable and equitable health impact.
- Open Access
NigBench: A multilingual point-of-care medical query benchmarking study of large language models in Nigeria
In this study, we introduce a novel benchmark comprising over 9,000 real-world, point-of-care, multilingual, and multimodal clinical question-answer pairs sourced from frontline health workers in Nigeria. Using the dataset, we compare local general practitioners to multiple leading open and closed LLMs. Our results reveal several critical insights into the suitability of LLMs as clinical decision support systems in low-resource contexts. The results confirm that performance varies widely by language and input modality (e.g., text vs speech): while models perform best on English text inputs, their accuracy drops significantly for local-language speech. Critically, it is possible to achieve substantial performance gains by transcribing and translating other languages into English before prompting an LLM – an important insight for non-anglophone product developers. Finally, this benchmark highlights key limitations of SLMs in supporting frontline healthcare in low-resource settings and provides a clear opportunity to track improvements as novel solutions are developed.
Open AccessNigBench: A multilingual point-of-care medical query benchmarking study of large language models in Nigeria
In this study, we introduce a novel benchmark comprising over 9,000 real-world, point-of-care, multilingual, and multimodal clinical question-answer pairs sourced from frontline health workers in Nigeria. Using the dataset, we compare local general practitioners to multiple leading open and closed LLMs. Our results reveal several critical insights into the suitability of LLMs as clinical decision support systems in low-resource contexts. The results confirm that performance varies widely by language and input modality (e.g., text vs speech): while models perform best on English text inputs, their accuracy drops significantly for local-language speech. Critically, it is possible to achieve substantial performance gains by transcribing and translating other languages into English before prompting an LLM – an important insight for non-anglophone product developers. Finally, this benchmark highlights key limitations of SLMs in supporting frontline healthcare in low-resource settings and provides a clear opportunity to track improvements as novel solutions are developed.
- Top Cited PapersOpen Access
- 14 January 2026
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 28, e63495
Africa’s Digital Health Revolution: The Digital Fit-Viability Model to Move From Innovation to Scaled Implementation
Digital innovations hold immense potential to transform health care delivery, particularly in sub-Saharan Africa, where financial, geographical, and infrastructural constraints continue to hinder progress toward universal health care delivery. Although a growing health tech sector offers creative solutions, few digital health interventions reach scaled implementation. In this paper, we present the digital fit/viability model—an adapted determinant framework to describe facilitators and barriers to moving from digital tools to integrated digital health implementation. We then use this model to describe the specific challenges and recommended solutions when developing digital health tools for health systems in sub-Saharan Africa.
- 14 January 2026
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 28, e63495
Top Cited PapersOpen AccessAfrica’s Digital Health Revolution: The Digital Fit-Viability Model to Move From Innovation to Scaled Implementation
Digital innovations hold immense potential to transform health care delivery, particularly in sub-Saharan Africa, where financial, geographical, and infrastructural constraints continue to hinder progress toward universal health care delivery. Although a growing health tech sector offers creative solutions, few digital health interventions reach scaled implementation. In this paper, we present the digital fit/viability model—an adapted determinant framework to describe facilitators and barriers to moving from digital tools to integrated digital health implementation. We then use this model to describe the specific challenges and recommended solutions when developing digital health tools for health systems in sub-Saharan Africa.
- Open Access
AfriSpeech-MultiBench: A Verticalized Multidomain Multicountry Benchmark Suite for African Accented English ASR
Recent advances in speech-enabled AI, including Google's NotebookLM and OpenAI's speech-to-speech API, are driving widespread interest in voice interfaces globally. Despite this momentum, there exists no publicly available application-specific model evaluation that caters to Africa's linguistic diversity. We present AfriSpeech-MultiBench, the first domain-specific evaluation suite for over 100 African English accents across 10+ countries and seven application domains: Finance, Legal, Medical, General dialogue, Call Center, Named Entities and Hallucination Robustness. We benchmark a diverse range of open, closed, unimodal ASR and multimodal LLM-based speech recognition systems using both spontaneous and non-spontaneous speech conversation drawn from various open African accented English speech datasets. Our empirical analysis reveals systematic variation: open-source ASR models excels in spontaneous speech contexts but degrades on noisy, non-native dialogue; multimodal LLMs are more accent-robust yet struggle with domain-specific named entities; proprietary models deliver high accuracy on clean speech but vary significantly by country and domain. Models fine-tuned on African English achieve competitive accuracy with lower latency, a practical advantage for deployment, hallucinations still remain a big problem for most SOTA models. By releasing this comprehensive benchmark, we empower practitioners and researchers to select voice technologies suited to African use-cases, fostering inclusive voice applications for underserved communities.
Open AccessAfriSpeech-MultiBench: A Verticalized Multidomain Multicountry Benchmark Suite for African Accented English ASR
Recent advances in speech-enabled AI, including Google's NotebookLM and OpenAI's speech-to-speech API, are driving widespread interest in voice interfaces globally. Despite this momentum, there exists no publicly available application-specific model evaluation that caters to Africa's linguistic diversity. We present AfriSpeech-MultiBench, the first domain-specific evaluation suite for over 100 African English accents across 10+ countries and seven application domains: Finance, Legal, Medical, General dialogue, Call Center, Named Entities and Hallucination Robustness. We benchmark a diverse range of open, closed, unimodal ASR and multimodal LLM-based speech recognition systems using both spontaneous and non-spontaneous speech conversation drawn from various open African accented English speech datasets. Our empirical analysis reveals systematic variation: open-source ASR models excels in spontaneous speech contexts but degrades on noisy, non-native dialogue; multimodal LLMs are more accent-robust yet struggle with domain-specific named entities; proprietary models deliver high accuracy on clean speech but vary significantly by country and domain. Models fine-tuned on African English achieve competitive accuracy with lower latency, a practical advantage for deployment, hallucinations still remain a big problem for most SOTA models. By releasing this comprehensive benchmark, we empower practitioners and researchers to select voice technologies suited to African use-cases, fostering inclusive voice applications for underserved communities.
- Open Access
Probing the Surgical Competence of LLMs: A global health study leveraging AfriMedQA benchmarks
Global surgical care faces a severe workforce shortage, with more than 1.2 million additional specialists needed by 2030, particularly in low- and middle-income countries (LMICs). Large language models (LLMs) have demonstrated impressive medical reasoning on standardized exams, but their safety, reliability, and specialty-specific performance—especially in procedural fields such as surgery—remain uncertain. Here we evaluate over 40 state-of-the-art LLMs on 3,900 expert-authored multiple-choice questions across 32 medical specialties from the AfriMed-QA benchmark, developed by 20 African medical professors. Top models (o1, GPT-4o, Claude 3.5) achieved mean accuracies exceeding 82%, showing strong diagnostic reasoning, yet consistently underperformed in surgery, pathology, and obstetrics compared with medical disciplines. Error analyses revealed frequent procedural reasoning failures, omission of local clinical guidelines, and overconfident but incorrect answers. Smaller or biomedical models exhibited higher hallucination and formatting error rates, while prompting strategies had inconsistent benefits. These results highlight the uneven readiness of LLMs for specialty-specific decision support and underscore the need for locally grounded evaluation frameworks, improved instruction tuning, and rigorous real-world validation to ensure the safe and equitable deployment of AI-assisted clinical tools in LMICs.
Open AccessProbing the Surgical Competence of LLMs: A global health study leveraging AfriMedQA benchmarks
Global surgical care faces a severe workforce shortage, with more than 1.2 million additional specialists needed by 2030, particularly in low- and middle-income countries (LMICs). Large language models (LLMs) have demonstrated impressive medical reasoning on standardized exams, but their safety, reliability, and specialty-specific performance—especially in procedural fields such as surgery—remain uncertain. Here we evaluate over 40 state-of-the-art LLMs on 3,900 expert-authored multiple-choice questions across 32 medical specialties from the AfriMed-QA benchmark, developed by 20 African medical professors. Top models (o1, GPT-4o, Claude 3.5) achieved mean accuracies exceeding 82%, showing strong diagnostic reasoning, yet consistently underperformed in surgery, pathology, and obstetrics compared with medical disciplines. Error analyses revealed frequent procedural reasoning failures, omission of local clinical guidelines, and overconfident but incorrect answers. Smaller or biomedical models exhibited higher hallucination and formatting error rates, while prompting strategies had inconsistent benefits. These results highlight the uneven readiness of LLMs for specialty-specific decision support and underscore the need for locally grounded evaluation frameworks, improved instruction tuning, and rigorous real-world validation to ensure the safe and equitable deployment of AI-assisted clinical tools in LMICs.
- Open Access
A multi-country study comparing typed to automatic speech recognition-based medical documentation speeds among Low- and Middle-Income Country Trained Clinicians
For decades, medical voice dictation and scribe services have boosted productivity in high-resource settings. Yet, they remain virtually absent in low- and middle-income countries (LMICs), where healthcare systems face physician shortages and heavier patient loads, but rely on outdated, paper-based workflows. Digital transformation efforts in these settings often overlook a critical barrier: the limited computer proficiency of overworked clinicians. While voice input is typically considered a suitable alternative that alleviates the additional cognitive burden from keyboard-based data entry, studies in high-resource settings report mixed findings on its efficiency. This study evaluates whether those findings hold in LMIC contexts. We assessed typing and dictation speeds among over 1,000 clinicians and health workers across 60+ hospitals in 15+ LMICs. Results reveal a median keyboard speed of just 21.4 words per minute (wpm), compared to dictation speeds of 4–5x faster on average (median 93 wpm). This significant speed improvement underscores the potential of speech recognition to reduce documentation burdens, improve workflow efficiency, and transform clinician experiences, evoking feelings of regret at the time lost to inefficient systems, and reinforcing the urgency of integrating voice solutions into LMIC digital health strategies.
Open AccessA multi-country study comparing typed to automatic speech recognition-based medical documentation speeds among Low- and Middle-Income Country Trained Clinicians
For decades, medical voice dictation and scribe services have boosted productivity in high-resource settings. Yet, they remain virtually absent in low- and middle-income countries (LMICs), where healthcare systems face physician shortages and heavier patient loads, but rely on outdated, paper-based workflows. Digital transformation efforts in these settings often overlook a critical barrier: the limited computer proficiency of overworked clinicians. While voice input is typically considered a suitable alternative that alleviates the additional cognitive burden from keyboard-based data entry, studies in high-resource settings report mixed findings on its efficiency. This study evaluates whether those findings hold in LMIC contexts. We assessed typing and dictation speeds among over 1,000 clinicians and health workers across 60+ hospitals in 15+ LMICs. Results reveal a median keyboard speed of just 21.4 words per minute (wpm), compared to dictation speeds of 4–5x faster on average (median 93 wpm). This significant speed improvement underscores the potential of speech recognition to reduce documentation burdens, improve workflow efficiency, and transform clinician experiences, evoking feelings of regret at the time lost to inefficient systems, and reinforcing the urgency of integrating voice solutions into LMIC digital health strategies.
- Top Cited PapersOpen Access
- 23 April 2025
- Published by Springer Nature in npj Digital Medicine
- Vol. 8 (1) , 1-4
Global Initiative on AI for Health (GI-AI4H): strategic priorities advancing governance across the United Nations
The Global Initiative on Artificial Intelligence for Health (GI-AI4H), established by the World Health Organization, serves to harmonize governance standards for artificial intelligence (AI). The GI-AI4H spearheads novel on-the-ground efforts, especially in low- and middle-income countries, to advance ethical, regulatory, implementation, and operational dimensions of global governance for health AI. The GI-AI4H’s efforts across the United Nations drives safe, ethical, equitable, and sustainable health AI use for the global community.
- 23 April 2025
- Published by Springer Nature in npj Digital Medicine
- Vol. 8 (1) , 1-4
Top Cited PapersOpen AccessGlobal Initiative on AI for Health (GI-AI4H): strategic priorities advancing governance across the United Nations
The Global Initiative on Artificial Intelligence for Health (GI-AI4H), established by the World Health Organization, serves to harmonize governance standards for artificial intelligence (AI). The GI-AI4H spearheads novel on-the-ground efforts, especially in low- and middle-income countries, to advance ethical, regulatory, implementation, and operational dimensions of global governance for health AI. The GI-AI4H’s efforts across the United Nations drives safe, ethical, equitable, and sustainable health AI use for the global community.
- Open Access
Afrispeech-Dialog: A Benchmark Dataset for Spontaneous English Conversations in Healthcare and Beyond
Speech technologies are transforming interactions across various sectors, from healthcare to call centers and robots, yet their performance on African-accented conversations remains underexplored. We introduce Afrispeech-Dialog, a benchmark dataset of 50 simulated medical and non-medical African-accented English conversations, designed to evaluate automatic speech recognition (ASR) and related technologies. We assess state-of-the-art (SOTA) speaker diarization and ASR systems on long-form, accented speech, comparing their performance with native accents and discover a 10%+ performance degradation. Additionally, we explore medical conversation summarization capabilities of large language models (LLMs) to demonstrate the impact of ASR errors on downstream medical summaries, providing insights into the challenges and opportunities for speech technologies in the Global South. Our work highlights the need for more inclusive datasets to advance conversational AI in low-resource settings.
Open AccessAfrispeech-Dialog: A Benchmark Dataset for Spontaneous English Conversations in Healthcare and Beyond
Speech technologies are transforming interactions across various sectors, from healthcare to call centers and robots, yet their performance on African-accented conversations remains underexplored. We introduce Afrispeech-Dialog, a benchmark dataset of 50 simulated medical and non-medical African-accented English conversations, designed to evaluate automatic speech recognition (ASR) and related technologies. We assess state-of-the-art (SOTA) speaker diarization and ASR systems on long-form, accented speech, comparing their performance with native accents and discover a 10%+ performance degradation. Additionally, we explore medical conversation summarization capabilities of large language models (LLMs) to demonstrate the impact of ASR errors on downstream medical summaries, providing insights into the challenges and opportunities for speech technologies in the Global South. Our work highlights the need for more inclusive datasets to advance conversational AI in low-resource settings.
- Top Cited PapersOpen Access
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 1948-1973
AfriMed-QA: A Pan-African, Multi-Specialty, Medical Question-Answering Benchmark Dataset
Charles Nimo, Tobi Olatunji, Abraham Toluwase Owodunni, Tassallah Abdullahi, Emmanuel Ayodele, Mardhiyah Sanni, Ezinwanne C. Aka, Folafunmi Omofoye, Foutse Yuehgoh, Timothy Faniran, Bonaventure F. P. Dossou, Moshood O. Yekini, Jonas Kemp, Katherine A Heller, Jude Chidubem Omeke, Chidi Asuzu Md, Naome A Etori, Aïmérou Ndiaye, Ifeoma Okoh, Evans Doe Ocansey, Wendy Kinara, Michael L. Best, Irfan Essa, Stephen Edward Moore, Chris Fourie, Mercy Nyamewaa Asiedu. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 1948-1973
Top Cited PapersOpen AccessAfriMed-QA: A Pan-African, Multi-Specialty, Medical Question-Answering Benchmark Dataset
Charles Nimo, Tobi Olatunji, Abraham Toluwase Owodunni, Tassallah Abdullahi, Emmanuel Ayodele, Mardhiyah Sanni, Ezinwanne C. Aka, Folafunmi Omofoye, Foutse Yuehgoh, Timothy Faniran, Bonaventure F. P. Dossou, Moshood O. Yekini, Jonas Kemp, Katherine A Heller, Jude Chidubem Omeke, Chidi Asuzu Md, Naome A Etori, Aïmérou Ndiaye, Ifeoma Okoh, Evans Doe Ocansey, Wendy Kinara, Michael L. Best, Irfan Essa, Stephen Edward Moore, Chris Fourie, Mercy Nyamewaa Asiedu. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.
- Open Access
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 8399-8417
Afrispeech-Dialog: A Benchmark Dataset for Spontaneous English Conversations in Healthcare and Beyond
Mardhiyah Sanni, Tassallah Abdullahi, Devendra Deepak Kayande, Emmanuel Ayodele, Naome A Etori, Michael Samwel Mollel, Moshood O. Yekini, Chibuzor Okocha, Lukman Enegi Ismaila, Folafunmi Omofoye, Boluwatife A. Adewale, Tobi Olatunji. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.
- 1 January 2025
- Published by Association for Computational Linguistics (ACL)
- p. 8399-8417
Open AccessAfrispeech-Dialog: A Benchmark Dataset for Spontaneous English Conversations in Healthcare and Beyond
Mardhiyah Sanni, Tassallah Abdullahi, Devendra Deepak Kayande, Emmanuel Ayodele, Naome A Etori, Michael Samwel Mollel, Moshood O. Yekini, Chibuzor Okocha, Lukman Enegi Ismaila, Folafunmi Omofoye, Boluwatife A. Adewale, Tobi Olatunji. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025.