Henry Bergman
Research Subjects
Research Interests
Henry Bergman has a total of 30 co-authors. Here are the top co-authors by number of shared publications:
Vivian Liu
Clinical AI Research, Heidi Health, 140 Goswell Road, London EC1V 7DY, United Kingdom
4 shared publicationsBen Austin
Clinical AI Research, Heidi Health, 140 Goswell Road, London EC1V 7DY, United Kingdom
3 shared publicationsAlun H. Davies
Academic Section of Vascular Surgery, Department of Surgery and Cancer, Imperial College London, London, UK
2 shared publications- Ankur Thapar
Mid and South Essex Vascular Unit, Mid and South Essex NHS Foundation Trust, Basildon, UK; Medical Technology Research Centre and School of Medicine, Anglia Ruskin University, Chelmsford, UK
2 shared publications - Benjamin Clay
Department of Trauma and Orthopaedic Surgery, East Suffolk and North Essex NHS Foundation Trust, United Kingdom; Department of Public Health and Primary Care, University of Cambridge, United Kingdom
2 shared publications Rohan Sanghera
Clinical AI Research, Heidi Health, 140 Goswell Road, London EC1V 7DY, United Kingdom
2 shared publications
Latest Publications
- Open Access
- 3 November 2026
- Published by Springer Nature
Comparing the accuracy of hospital clinic notes written by an AI medical scribe with notes written by doctors
- 3 November 2026
- Published by Springer Nature
Open AccessComparing the accuracy of hospital clinic notes written by an AI medical scribe with notes written by doctors
- Open Access
Quality and transcript-grounded clinical-risk error burden of AI-generated vs allied health professional clinical notes: a paired simulation study across four disciplines
Background: Ambient artificial-intelligence (AI) scribes are being deployed beyond medicine into allied-health practice, but comparative evidence mainly concerns physician documentation. Objective: To compare quality and transcript-grounded clinical-risk error burden of AI-generated and allied health professional notes from identical simulated consultations. Methods: Paired simulation across dietetics, physiotherapy, occupational therapy and speech and language therapy (five consultations each; 20 pairs; 40 notes). Heidi generated an unedited first-draft note and a professional documented each encounter. Professionals rated PDQI-9 quality. Professional and automated review identified unsupported statements and omissions; two language models graded clinical risk. Co-primary outcomes were PDQI-9 score and Critical+High error density. Results: Mean PDQI-9 scores were 38.45 and 35.05 for AI and professional notes (paired difference +3.40, 95% CI +0.01 to +6.79; Wilcoxon p=0.042; dz=0.47). The Bayesian estimate was +3.47 (94% equal-tailed credible interval +0.66 to +6.30; P[AI>professional]=0.989). Automated-reviewer all-error density was 3.35 vs 8.55 per note (professional-minus-AI difference 5.20, 95% CI 3.85 to 6.55); omission density was 1.45 vs 5.20 (difference 3.75, 2.80 to 4.65). Critical+High density was 0.35 vs 0.75 (difference 0.40, −0.10 to 1.00). The pooled quality result was driven principally by dietetics. Conclusions: AI-generated notes achieved modestly higher pooled quality and fewer transcript-grounded errors, chiefly omissions; serious-error evidence was inconclusive. Small discipline samples and incomplete ground truth preclude safety or discipline-wide superiority claims.
Open AccessQuality and transcript-grounded clinical-risk error burden of AI-generated vs allied health professional clinical notes: a paired simulation study across four disciplines
Background: Ambient artificial-intelligence (AI) scribes are being deployed beyond medicine into allied-health practice, but comparative evidence mainly concerns physician documentation. Objective: To compare quality and transcript-grounded clinical-risk error burden of AI-generated and allied health professional notes from identical simulated consultations. Methods: Paired simulation across dietetics, physiotherapy, occupational therapy and speech and language therapy (five consultations each; 20 pairs; 40 notes). Heidi generated an unedited first-draft note and a professional documented each encounter. Professionals rated PDQI-9 quality. Professional and automated review identified unsupported statements and omissions; two language models graded clinical risk. Co-primary outcomes were PDQI-9 score and Critical+High error density. Results: Mean PDQI-9 scores were 38.45 and 35.05 for AI and professional notes (paired difference +3.40, 95% CI +0.01 to +6.79; Wilcoxon p=0.042; dz=0.47). The Bayesian estimate was +3.47 (94% equal-tailed credible interval +0.66 to +6.30; P[AI>professional]=0.989). Automated-reviewer all-error density was 3.35 vs 8.55 per note (professional-minus-AI difference 5.20, 95% CI 3.85 to 6.55); omission density was 1.45 vs 5.20 (difference 3.75, 2.80 to 4.65). Critical+High density was 0.35 vs 0.75 (difference 0.40, −0.10 to 1.00). The pooled quality result was driven principally by dietetics. Conclusions: AI-generated notes achieved modestly higher pooled quality and fewer transcript-grounded errors, chiefly omissions; serious-error evidence was inconclusive. Small discipline samples and incomplete ground truth preclude safety or discipline-wide superiority claims.
- Open Access
Mind the gap: emergent clinical risk at the interface of two individually safe AI systems in a multilingual ambient scribe
Objectives: To determine whether single-layer evaluation characterises clinical risk in the final note of a multilingual ambient AI scribe, and where serious errors arise. Design: Two-arm evaluation on one common clinical-risk scale, using a frozen, reference-aligned synthetic corpus. Setting: Scripted consultations spanning 77 languages and five complexity levels, from simple general practice to expert multidisciplinary-team handover. Main outcome measures: Per-session incidence of at least one serious (HIGH or CRITICAL) note-layer error in intrinsic (reference script to note, n=385) and end-to-end (automatic speech recognition (ASR) transcript to note, n=2,302) generation; three LLM raters classified discrepancies using a four-mode taxonomy. Results: Intrinsic note generation had a 3.4% serious-error rate, with no detectable gradient across complexity (1% to 6%; Cochran-Armitage p=0.42) or language resource (high 2%, medium 5%, low 4%). End-to-end serious errors rose steeply with complexity (L1 1% to L4/L5 24%) and language scarcity (high 7%, medium 10%, low 15%; OR 1.54 per tier, 95% CI 1.15-2.07; p=0.004, GEE clustered on language). Of serious in-note errors, 87% were ASR-derived and 7% note-originated; amplification exceeded correction threefold (fate entropy 1.19 bits). Repeated generation showed moderate reproducibility (Fleiss kappa 0.42); word error rate explained 24% of cross-language variance versus ∼1% for transcription risk density. Conclusions: Low serious-error rates in individual layers did not preclude higher end-to-end note risk. Evaluation should therefore include the clinician-facing note, because component-level metrics cannot fully capture risk created or transformed at the transcription-to-generation interface. What is already known on this topic: Ambient AI scribes are being deployed globally, and safety evaluation has characterised the transcription layer in isolation, most commonly with word error rate Frequency-based transcription metrics such as word error rate do not directly encode clinical consequence, and component-level performance may not represent the risk of the final generated note Large language models produce clinically relevant output stochastically, so identical prompts can yield materially different content What this study adds: Note generation from a perfect transcript had a low observed serious-error rate and no detectable complexity or language-resource gradient, whereas the end-to-end system showed steep gradients on both axes. Serious note-layer errors were predominantly ASR-derived, but note generation transformed upstream errors variably, amplifying serious errors about three times more often than it corrected them Transcription-layer metrics had limited predictive value for note-layer risk after accounting for complexity and language-level dependence; word error rate explained approximately one-quarter of cross-language variance How this study might affect research, practice or policy: Evaluation and procurement of ambient scribes should include end-to-end assessment of the clinician-facing note, with results stratified by consultation complexity and language where relevant A three-surface root-cause alignment using a four-mode root-cause taxonomy (note-originated, propagated-ASR, amplified-ASR and silent-correction) can attribute documentation errors to their point of origin or transformation Language-related performance should remain a surveillance target because disparities can emerge after components are composed, even when corresponding gradients are not detectable in an individual layer
Open AccessMind the gap: emergent clinical risk at the interface of two individually safe AI systems in a multilingual ambient scribe
Objectives: To determine whether single-layer evaluation characterises clinical risk in the final note of a multilingual ambient AI scribe, and where serious errors arise. Design: Two-arm evaluation on one common clinical-risk scale, using a frozen, reference-aligned synthetic corpus. Setting: Scripted consultations spanning 77 languages and five complexity levels, from simple general practice to expert multidisciplinary-team handover. Main outcome measures: Per-session incidence of at least one serious (HIGH or CRITICAL) note-layer error in intrinsic (reference script to note, n=385) and end-to-end (automatic speech recognition (ASR) transcript to note, n=2,302) generation; three LLM raters classified discrepancies using a four-mode taxonomy. Results: Intrinsic note generation had a 3.4% serious-error rate, with no detectable gradient across complexity (1% to 6%; Cochran-Armitage p=0.42) or language resource (high 2%, medium 5%, low 4%). End-to-end serious errors rose steeply with complexity (L1 1% to L4/L5 24%) and language scarcity (high 7%, medium 10%, low 15%; OR 1.54 per tier, 95% CI 1.15-2.07; p=0.004, GEE clustered on language). Of serious in-note errors, 87% were ASR-derived and 7% note-originated; amplification exceeded correction threefold (fate entropy 1.19 bits). Repeated generation showed moderate reproducibility (Fleiss kappa 0.42); word error rate explained 24% of cross-language variance versus ∼1% for transcription risk density. Conclusions: Low serious-error rates in individual layers did not preclude higher end-to-end note risk. Evaluation should therefore include the clinician-facing note, because component-level metrics cannot fully capture risk created or transformed at the transcription-to-generation interface. What is already known on this topic: Ambient AI scribes are being deployed globally, and safety evaluation has characterised the transcription layer in isolation, most commonly with word error rate Frequency-based transcription metrics such as word error rate do not directly encode clinical consequence, and component-level performance may not represent the risk of the final generated note Large language models produce clinically relevant output stochastically, so identical prompts can yield materially different content What this study adds: Note generation from a perfect transcript had a low observed serious-error rate and no detectable complexity or language-resource gradient, whereas the end-to-end system showed steep gradients on both axes. Serious note-layer errors were predominantly ASR-derived, but note generation transformed upstream errors variably, amplifying serious errors about three times more often than it corrected them Transcription-layer metrics had limited predictive value for note-layer risk after accounting for complexity and language-level dependence; word error rate explained approximately one-quarter of cross-language variance How this study might affect research, practice or policy: Evaluation and procurement of ambient scribes should include end-to-end assessment of the clinician-facing note, with results stratified by consultation complexity and language where relevant A three-surface root-cause alignment using a four-mode root-cause taxonomy (note-originated, propagated-ASR, amplified-ASR and silent-correction) can attribute documentation errors to their point of origin or transformation Language-related performance should remain a surveillance target because disparities can emerge after components are composed, even when corresponding gradients are not detectable in an individual layer
- Open Access
- 17 September 2026
- Published by JMIR Publications Inc. in JMIR Preprints
Reliability and construct validity of a multi-model LLM-as-judge evaluator battery for clinician-facing clinical question answering: a controlled adversarial benchmark study (Preprint)
BACKGROUND Open-ended clinical question systems generate fluent free-text for which conventional accuracy metrics are insufficient. Complex questions may admit more than one defensible answer depending on clinical context, jurisdiction, guideline version and the intent of the person asking. Human expert review remains important but is costly, hard to scale and variable. Large language models (LLMs) are increasingly used as automated evaluators, but an LLM-as-judge should be characterised as a measurement instrument before its outputs are treated as evidence. OBJECTIVE To characterise the inter-judge reliability, within-judge stability, severity-rating heterogeneity, difficulty discrimination and construct validity of an LLM-as-judge battery used to evaluate clinician-facing clinical question answering. METHODS We evaluated a three-model FACTFULNESS ensemble (Claude Opus 4.7, Gemini 3.1 Pro, GPT-5.4, each with native web search) and a single-model COMPLETENESS evaluator (GPT-5.4) on an author-constructed adversarial bank of 1,424 clinical questions across five difficulty tiers and 16 construct families. Of 1,421 scored items, 1,173 (82.5%) carried complete three-judge verdicts. Agreement on binary factuality used Gwet AC1, Fleiss kappa and unanimity, overall and within 22 strata. Sensitivity analyses addressed the complete-case restriction, claim-set non-identity, repeated scoring and retrieved-evidence overlap. RESULTS Gwet AC1 was 0.344 (95% CI 0.302-0.386), Fleiss kappa 0.212, and 46.3% of items unanimous. AC1 fell from 0.60 at difficulty tier D1 to 0.27 at D5, ranged from 0.79 to 0.13 across construct families, and ranged from 0.183 to 0.525 between judge pairs with non-overlapping intervals. The serious-or-worse harm-severity rate differed 17.1-fold across judges (0.23%, 2.34% and 3.94%). Requiring two of three judges reduced HIGH+ classification from 13.79% to 2.89%, and no answer met CRITICAL by consensus (0/1,421). FACTFULNESS-target families were more often consensus-inaccurate than non-target families (36.8% vs 27.9%, P=0.002), whereas the COMPLETENESS check was null once refusal-driven security items were excluded (1.7% vs 4.3%, P=0.27). On rescoring byte-identical answers, mean within-judge AC1 was 0.703 against 0.265 between judges, and the consensus risk band was reproduced across all five replicates for 52.0% of items. CONCLUSIONS Treated as a measurement instrument, the battery's reliability and severity grading depended strongly on judge identity and on the task being judged, and one of two planned construct checks behaved as intended. Consensus reduced extreme classifications, and aggregate estimates were reproducible enough to support period-on-period monitoring, whereas single-run item-level verdicts were not. Multi-model batteries are therefore usable for monitoring when consensus is reported with judge-specific dispersion, missingness and stratified reliability. Human-anchored calibration remains necessary before severity scale or risk labels can be interpreted as absolute clinical risk. CLINICALTRIAL NA
- 17 September 2026
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessReliability and construct validity of a multi-model LLM-as-judge evaluator battery for clinician-facing clinical question answering: a controlled adversarial benchmark study (Preprint)
BACKGROUND Open-ended clinical question systems generate fluent free-text for which conventional accuracy metrics are insufficient. Complex questions may admit more than one defensible answer depending on clinical context, jurisdiction, guideline version and the intent of the person asking. Human expert review remains important but is costly, hard to scale and variable. Large language models (LLMs) are increasingly used as automated evaluators, but an LLM-as-judge should be characterised as a measurement instrument before its outputs are treated as evidence. OBJECTIVE To characterise the inter-judge reliability, within-judge stability, severity-rating heterogeneity, difficulty discrimination and construct validity of an LLM-as-judge battery used to evaluate clinician-facing clinical question answering. METHODS We evaluated a three-model FACTFULNESS ensemble (Claude Opus 4.7, Gemini 3.1 Pro, GPT-5.4, each with native web search) and a single-model COMPLETENESS evaluator (GPT-5.4) on an author-constructed adversarial bank of 1,424 clinical questions across five difficulty tiers and 16 construct families. Of 1,421 scored items, 1,173 (82.5%) carried complete three-judge verdicts. Agreement on binary factuality used Gwet AC1, Fleiss kappa and unanimity, overall and within 22 strata. Sensitivity analyses addressed the complete-case restriction, claim-set non-identity, repeated scoring and retrieved-evidence overlap. RESULTS Gwet AC1 was 0.344 (95% CI 0.302-0.386), Fleiss kappa 0.212, and 46.3% of items unanimous. AC1 fell from 0.60 at difficulty tier D1 to 0.27 at D5, ranged from 0.79 to 0.13 across construct families, and ranged from 0.183 to 0.525 between judge pairs with non-overlapping intervals. The serious-or-worse harm-severity rate differed 17.1-fold across judges (0.23%, 2.34% and 3.94%). Requiring two of three judges reduced HIGH+ classification from 13.79% to 2.89%, and no answer met CRITICAL by consensus (0/1,421). FACTFULNESS-target families were more often consensus-inaccurate than non-target families (36.8% vs 27.9%, P=0.002), whereas the COMPLETENESS check was null once refusal-driven security items were excluded (1.7% vs 4.3%, P=0.27). On rescoring byte-identical answers, mean within-judge AC1 was 0.703 against 0.265 between judges, and the consensus risk band was reproduced across all five replicates for 52.0% of items. CONCLUSIONS Treated as a measurement instrument, the battery's reliability and severity grading depended strongly on judge identity and on the task being judged, and one of two planned construct checks behaved as intended. Consensus reduced extreme classifications, and aggregate estimates were reproducible enough to support period-on-period monitoring, whereas single-run item-level verdicts were not. Multi-model batteries are therefore usable for monitoring when consensus is reported with judge-specific dispersion, missingness and stratified reliability. Human-anchored calibration remains necessary before severity scale or risk labels can be interpreted as absolute clinical risk. CLINICALTRIAL NA
- Open Access
Beyond word error rate: clinical risk as the necessary standard for ambient AI scribe evaluation: evidence from 77 global languages
Objective Ambient AI scribes evaluated using frequency-based metrics such as word error rate (WER), which do not represent clinical consequence. We tested whether variation in these metrics tracks consequential transcription errors. Methods We constructed a multilingual corpus from five clinical dictation scripts spanning a complexity gradient, translated into 99 languages, rendered to synthetic speech under three acoustic conditions, and transcribed by a production ambient scribe. Six frequency metrics were computed. Three independent large language model raters from external providers assessed clinically meaningful error patterns in context using a Severity x Likelihood framework informed by UK digital clinical-safety-risk-management principles. Results Across 59,819 genuine transcription-error occurrences, 58,329 (97.5%) were LOW risk and 251 (0.42%) CRITICAL or HIGH. None of six frequency metrics showed a statistically detectable association with serious clinical risk across languages; correlations were small (absolute Spearman ρ<0.16). A Severity x Likelihood sum remained strongly correlated with WER (ρ=0.80), showing that the aggregate remained dominated by benign errors. At complexity level 3, low-resource languages had worse WER than high-resource languages (β=+0.078, 95% CI +0.045 to +0.111; p<0.0001), without a detectable difference in CRITICAL/HIGH risk (OR 1.21, 95% CI 0.43 to 3.43; p=0.72). Consultation complexity was the principal predictor of serious risk (OR 3.06 per level, p<0.0001). Conclusion Across this controlled multilingual corpus, aggregate transcription-frequency metrics did not reliably track the sparse severe tail of clinically consequential errors. WER remains appropriate for transcription quality, but these data do not support its use alone as a proxy for clinical safety. Context-aware assessment of error consequence provides complementary information that frequency measures can dilute.
Open AccessBeyond word error rate: clinical risk as the necessary standard for ambient AI scribe evaluation: evidence from 77 global languages
Objective Ambient AI scribes evaluated using frequency-based metrics such as word error rate (WER), which do not represent clinical consequence. We tested whether variation in these metrics tracks consequential transcription errors. Methods We constructed a multilingual corpus from five clinical dictation scripts spanning a complexity gradient, translated into 99 languages, rendered to synthetic speech under three acoustic conditions, and transcribed by a production ambient scribe. Six frequency metrics were computed. Three independent large language model raters from external providers assessed clinically meaningful error patterns in context using a Severity x Likelihood framework informed by UK digital clinical-safety-risk-management principles. Results Across 59,819 genuine transcription-error occurrences, 58,329 (97.5%) were LOW risk and 251 (0.42%) CRITICAL or HIGH. None of six frequency metrics showed a statistically detectable association with serious clinical risk across languages; correlations were small (absolute Spearman ρ<0.16). A Severity x Likelihood sum remained strongly correlated with WER (ρ=0.80), showing that the aggregate remained dominated by benign errors. At complexity level 3, low-resource languages had worse WER than high-resource languages (β=+0.078, 95% CI +0.045 to +0.111; p<0.0001), without a detectable difference in CRITICAL/HIGH risk (OR 1.21, 95% CI 0.43 to 3.43; p=0.72). Consultation complexity was the principal predictor of serious risk (OR 3.06 per level, p<0.0001). Conclusion Across this controlled multilingual corpus, aggregate transcription-frequency metrics did not reliably track the sparse severe tail of clinically consequential errors. WER remains appropriate for transcription quality, but these data do not support its use alone as a proxy for clinical safety. Context-aware assessment of error consequence provides complementary information that frequency measures can dilute.
- Open Access
Quality, consistency, and clinical safety of AI-generated versus clinician-written clinical notes: a multi-country paired simulation study
Background: Ambient AI documentation tools (“scribes”) are entering routine clinical practice at scale, but the evidence comparing the notes they produce against clinician-written notes is dominated by single-site, single-language studies that rely on human review to find errors — a method known to miss most documentation errors. Methods: We conducted a paired simulation across five countries and languages (Cambridge/English, Barcelona/Spanish, Milan/Italian, Paris/French, Cologne/German; 385 paired consultations, 770 notes). From each actor-performed consultation, an AI scribe (Heidi) and a junior-to-middle-grade clinician independently produced a note. Notes were scored on the PDQI-9 by evaluators blinded to authorship. Documentation errors were identified by two methods of deliberately different sensitivity — clinician adjudication, and a calibrated automated reviewer externally validated against a blinded ten-clinician panel — then graded for clinical risk by a three-model panel. The co-primary outcomes were PDQI-9 total and Critical+High error burden, the latter reported under both detection arms. The analysis plan was registered before any pooling across sites. Results: AI notes scored higher than clinician notes on the PDQI-9 (40.6 vs 35.6; difference +5.08, 95% CI 4.6–5.6; Cohen’s dz=0.55), consistently across all five sites (dz 0.41–0.75), and were less dispersed (5.7% of AI vs 27.8% of clinician notes fell below the study pre-specified low-score threshold (<32)). On the principal safety outcome — the paired probability that a note carried ≥1 Critical+High error — clinician notes were affected more often under both detection arms: 61.0% versus 24.4% by the calibrated reviewer (relative risk 2.50, 95% CI 2.09–3.00) and 21.8% versus 6.2% by clinician adjudication (relative risk 3.50, 95% CI 2.32–5.27). The difference was largest for omissions. Unaided clinician review identified roughly 12% of the errors the calibrated reviewer retained, and a smaller fraction in AI notes than in clinician notes. Conclusions: In this simulation, AI-generated notes scored higher on documentation quality, varied less, and carried fewer clinically significant errors than notes written on the same consultations by junior-to-middle-grade clinicians. The magnitude of the safety difference depends on the sensitivity of error detection, so we report both detection regimes and bound rather than point-estimate the absolute error rate. Extension to live practice, consultant-authored documentation, and notes as filed after clinician editing remains to be established. One sentence description: In a five-country, five-language paired simulation, AI-generated clinical notes scored higher on documentation quality and carried fewer clinically significant errors than notes written on the same consultations by junior-to-middle-grade clinicians.
Open AccessQuality, consistency, and clinical safety of AI-generated versus clinician-written clinical notes: a multi-country paired simulation study
Background: Ambient AI documentation tools (“scribes”) are entering routine clinical practice at scale, but the evidence comparing the notes they produce against clinician-written notes is dominated by single-site, single-language studies that rely on human review to find errors — a method known to miss most documentation errors. Methods: We conducted a paired simulation across five countries and languages (Cambridge/English, Barcelona/Spanish, Milan/Italian, Paris/French, Cologne/German; 385 paired consultations, 770 notes). From each actor-performed consultation, an AI scribe (Heidi) and a junior-to-middle-grade clinician independently produced a note. Notes were scored on the PDQI-9 by evaluators blinded to authorship. Documentation errors were identified by two methods of deliberately different sensitivity — clinician adjudication, and a calibrated automated reviewer externally validated against a blinded ten-clinician panel — then graded for clinical risk by a three-model panel. The co-primary outcomes were PDQI-9 total and Critical+High error burden, the latter reported under both detection arms. The analysis plan was registered before any pooling across sites. Results: AI notes scored higher than clinician notes on the PDQI-9 (40.6 vs 35.6; difference +5.08, 95% CI 4.6–5.6; Cohen’s dz=0.55), consistently across all five sites (dz 0.41–0.75), and were less dispersed (5.7% of AI vs 27.8% of clinician notes fell below the study pre-specified low-score threshold (<32)). On the principal safety outcome — the paired probability that a note carried ≥1 Critical+High error — clinician notes were affected more often under both detection arms: 61.0% versus 24.4% by the calibrated reviewer (relative risk 2.50, 95% CI 2.09–3.00) and 21.8% versus 6.2% by clinician adjudication (relative risk 3.50, 95% CI 2.32–5.27). The difference was largest for omissions. Unaided clinician review identified roughly 12% of the errors the calibrated reviewer retained, and a smaller fraction in AI notes than in clinician notes. Conclusions: In this simulation, AI-generated notes scored higher on documentation quality, varied less, and carried fewer clinically significant errors than notes written on the same consultations by junior-to-middle-grade clinicians. The magnitude of the safety difference depends on the sensitivity of error detection, so we report both detection regimes and bound rather than point-estimate the absolute error rate. Extension to live practice, consultant-authored documentation, and notes as filed after clinician editing remains to be established. One sentence description: In a five-country, five-language paired simulation, AI-generated clinical notes scored higher on documentation quality and carried fewer clinically significant errors than notes written on the same consultations by junior-to-middle-grade clinicians.
- 28 February 2025
- Published by The Vascular Society of Great Britain and Ireland in Journal of Vascular Societies Great Britain & Ireland
- Vol. 4 (2) , 74-80
What’s the denominator? An 8-year audit of ruptured abdominal aortic aneurysm outcomes, including rates of conservative and palliative management
Background: A ruptured aortic aneurysm (rAAA) is a surgical emergency with rapid onset and poor survival. Expedited diagnosis and surgical repair are required to prevent exsanguination. Extensive literature compares outcomes of endovascular and open repair but there is little discussion of the denominator – the patients undergoing non-operative palliative management. Without this, all reported outcomes are confounded by case selection. We carried out this review to learn more about those operated on and to include the rate of palliation to learn more about overall outcomes. Methods: A retrospective note review of the electronic patient record identified all rAAAs treated at Addenbrooke’s hospital between January 2015 and November 2022. Demographics, treatment strategy, mortality, pre-operative imaging, transfusion requirement and inpatient stay were interrogated. Results: There were 209 rAAAs identified. Management was with open surgery (44%), endovascular surgery (36%) and palliation (20%). Open repair was associated with higher mortality, longer inpatient and ITU stay, and larger transfusion requirements than endovascular repair. Open repair was associated with a higher rate of return to theatre in the short term, while endovascular repair (EVAR) had a higher rate in the long term. 7% of palliative rAAAs remained alive at one year. Conclusions: An EVAR-first approach results in lower mortality rates and better short-term outcomes. Although interventional outcomes are widely reported, approximately 20% of the caseload is palliated. This denominator should be reported alongside conventional outcomes to address the bias of case selection and resource allocation and to provide a more comprehensive picture of overall outcomes.
- 28 February 2025
- Published by The Vascular Society of Great Britain and Ireland in Journal of Vascular Societies Great Britain & Ireland
- Vol. 4 (2) , 74-80
What’s the denominator? An 8-year audit of ruptured abdominal aortic aneurysm outcomes, including rates of conservative and palliative management
Background: A ruptured aortic aneurysm (rAAA) is a surgical emergency with rapid onset and poor survival. Expedited diagnosis and surgical repair are required to prevent exsanguination. Extensive literature compares outcomes of endovascular and open repair but there is little discussion of the denominator – the patients undergoing non-operative palliative management. Without this, all reported outcomes are confounded by case selection. We carried out this review to learn more about those operated on and to include the rate of palliation to learn more about overall outcomes. Methods: A retrospective note review of the electronic patient record identified all rAAAs treated at Addenbrooke’s hospital between January 2015 and November 2022. Demographics, treatment strategy, mortality, pre-operative imaging, transfusion requirement and inpatient stay were interrogated. Results: There were 209 rAAAs identified. Management was with open surgery (44%), endovascular surgery (36%) and palliation (20%). Open repair was associated with higher mortality, longer inpatient and ITU stay, and larger transfusion requirements than endovascular repair. Open repair was associated with a higher rate of return to theatre in the short term, while endovascular repair (EVAR) had a higher rate in the long term. 7% of palliative rAAAs remained alive at one year. Conclusions: An EVAR-first approach results in lower mortality rates and better short-term outcomes. Although interventional outcomes are widely reported, approximately 20% of the caseload is palliated. This denominator should be reported alongside conventional outcomes to address the bias of case selection and resource allocation and to provide a more comprehensive picture of overall outcomes.
- Top Cited PapersOpen Access
- 12 February 2025
- Published by Elsevier in Computers in Biology and Medicine
- Vol. 188, 109808
Natural language processing techniques applied to the electronic health record in clinical research and practice - an introduction to methodologies
- 12 February 2025
- Published by Elsevier in Computers in Biology and Medicine
- Vol. 188, 109808
Top Cited PapersOpen AccessNatural language processing techniques applied to the electronic health record in clinical research and practice - an introduction to methodologies
- Open Access
- 4 December 2023
- Published by SAGE Publications in Journal of Endovascular Therapy
- Vol. 32 (5) , 1351-1359
Meta-Analysis of Duplex Surveillance Following Lower Limb Endovascular Intervention
Introduction: The aim of this systematic review was to identify the evidence in the literature for limb salvage with the introduction of duplex surveillance. Methods: A systematic review and meta-analysis was performed using the Preferred Reporting Items for Systematic Reviews and Meta-Analysis guidelines (PRISMA) methodology for all studies which compared a group undergoing clinical surveillance with a group undergoing combined clinical and duplex surveillance after endovascular therapy for peripheral arterial disease. MEDLINE, EMBASE, the Cochrane Database for Systematic Reviews, and ClinicalTrials.gov were searched for relevant studies by 2 reviewers. Studies were quality assessed using the ROBINS-I tool. An individual patient data survival analysis and meta-analysis for 1- and 2-year amputation outcomes using a random-effects model were performed. Results: Two low-quality nonrandomized studies met the inclusion criteria. There was a statistically and clinically significant reduction in major amputation in patients undergoing combined clinical and duplex surveillance (log-rank p<0.001). The number needed to treat to prevent 1 amputation at 2 years was 5 patients. At 1 year, the odds ratio (OR) for amputation was 0.22, 95% confidence interval (CI)=0.10-0.48, with no statistical heterogeneity. At 2 years, the numbers of patients were low and the effect on amputation was less certain OR=0.25, 95% CI=0.04-1.58. Conclusions: Preliminary, low-quality data suggests that there may be a clinically significant reduction in major amputation with the introduction of duplex surveillance. It is recommended that a randomized controlled trial is performed to confirm these findings and identify the anatomical subgroups that benefit the most from surveillance. Clinical Impact: “Two low-quality studies reveal a significant clinical impact: combined clinical and duplex surveillance markedly reduces major amputations (log-rank p<0.001). At 1-year, the odds ratio for amputation is 0.22 (95% CI=0.10-0.48), emphasizing limb salvage benefits. Despite less certainty at 2-years, a notable absolute risk reduction of 19% is seen, with a number needed to treat of 5. This underscores the urgent need for a randomized controlled trial to validate findings and identify key subgroups. The meta-analysis strongly advocates implementing duplex surveillance for a year post-endovascular interventions, especially in patients fit for reintervention, with important considerations for cost-effectiveness and focused clinical trials.”
- 4 December 2023
- Published by SAGE Publications in Journal of Endovascular Therapy
- Vol. 32 (5) , 1351-1359
Open AccessMeta-Analysis of Duplex Surveillance Following Lower Limb Endovascular Intervention
Introduction: The aim of this systematic review was to identify the evidence in the literature for limb salvage with the introduction of duplex surveillance. Methods: A systematic review and meta-analysis was performed using the Preferred Reporting Items for Systematic Reviews and Meta-Analysis guidelines (PRISMA) methodology for all studies which compared a group undergoing clinical surveillance with a group undergoing combined clinical and duplex surveillance after endovascular therapy for peripheral arterial disease. MEDLINE, EMBASE, the Cochrane Database for Systematic Reviews, and ClinicalTrials.gov were searched for relevant studies by 2 reviewers. Studies were quality assessed using the ROBINS-I tool. An individual patient data survival analysis and meta-analysis for 1- and 2-year amputation outcomes using a random-effects model were performed. Results: Two low-quality nonrandomized studies met the inclusion criteria. There was a statistically and clinically significant reduction in major amputation in patients undergoing combined clinical and duplex surveillance (log-rank p<0.001). The number needed to treat to prevent 1 amputation at 2 years was 5 patients. At 1 year, the odds ratio (OR) for amputation was 0.22, 95% confidence interval (CI)=0.10-0.48, with no statistical heterogeneity. At 2 years, the numbers of patients were low and the effect on amputation was less certain OR=0.25, 95% CI=0.04-1.58. Conclusions: Preliminary, low-quality data suggests that there may be a clinically significant reduction in major amputation with the introduction of duplex surveillance. It is recommended that a randomized controlled trial is performed to confirm these findings and identify the anatomical subgroups that benefit the most from surveillance. Clinical Impact: “Two low-quality studies reveal a significant clinical impact: combined clinical and duplex surveillance markedly reduces major amputations (log-rank p<0.001). At 1-year, the odds ratio for amputation is 0.22 (95% CI=0.10-0.48), emphasizing limb salvage benefits. Despite less certainty at 2-years, a notable absolute risk reduction of 19% is seen, with a number needed to treat of 5. This underscores the urgent need for a randomized controlled trial to validate findings and identify key subgroups. The meta-analysis strongly advocates implementing duplex surveillance for a year post-endovascular interventions, especially in patients fit for reintervention, with important considerations for cost-effectiveness and focused clinical trials.”