James Creeden

Background. Comprehensive genomic reports in oncology contain complex molecular information that must be translated into concise summaries for treating clinicians. Manual summarisation is time-intensive, and robust evidence supporting AI-assisted summarisation systems remains limited. We evaluated whether AI-generated summaries were non-inferior to manually produced summaries in faithfully representing information from tertiary paediatric cancer genomics reports. Methods. QNOMX-VHIR-CPSP-001 Phase 1 was a single-site, non-interventional clinical performance study conducted at Vall d'Hebron Institut de Recerca under ISO 20916:2019. De-identified tertiary paediatric cancer genomics reports were summarised using both the AI-assisted system under evaluation and the standard manual workflow. Three qualified raters assessed both summary types against the source reports using the six-domain, five-point Quality Summary Index (QSI) in a blinded, counterbalanced crossover design with a minimum fourteen-day washout. Co-primary endpoints were composite content and presentation scores. Non-inferiority was assessed using a pre-specified Bayesian hierarchical model with a margin of 0.5 points and confirmed using a frequentist linear mixed model. Results. Thirty-seven of 38 planned cases contributed 74 paired evaluations. Mean content scores were similar between AI-generated and manual summaries (4.01 vs 3.97), whereas AI-generated summaries achieved higher presentation scores (4.39 vs 4.12). Non-inferiority was demonstrated for both co-primary endpoints: content +0.05 (95% CrI −0.18 to +0.28), presentation +0.28 (+0.10 to +0.47). Frequentist analyses were concordant. Accuracy was the only QSI dimension for which Bayesian non-inferiority was not concluded. Inter-rater reliability was low across all QSI domains. Conclusions. AI-generated summaries were non-inferior to manually produced summaries for both content and presentation endpoints in tertiary paediatric oncology genomics reports. Presentation scores were higher for AI-generated summaries, whereas content scores were comparable between approaches. Low inter-rater reliability limits confidence in the dimension-level estimates and is the main constraint on future validation studies. Clinical validity and workflow effect were not assessed.

Background: Tertiary clinical genomics reports condense layered molecular findings into documents that treating oncologists must read, translate, and act upon; manual summarisation of these reports is time-consuming and variable. Tools that assist summarisation and translation into local languages are emerging, yet the field lacks an agreed methodology for evaluating such tools before any downstream clinical use. The appropriate first endpoint is fidelity of the generated summary to its source report, assessed by qualified human raters under blinded scoring, not downstream variant classification. Methods: QNOMX-VHIR-CPSP-001 Phase 1 is a single-site, non-interventional clinical performance study conducted at Vall d’Hebron Institut de Recerca (VHIR) under ISO 20916:2019 as a Clinical Performance Study Protocol. De-identified tertiary cancer genomics reports from pediatric oncology cases are summarised by the AI-assisted summarisation system under evaluation and, in parallel, by the standard manual workflow. Qualified raters score both summary types against the source genomics report using the Quality Summary Index (QSI), a six-dimension, five-point rubric adapted from the Provider Documentation Summarization Quality Instrument, under a blinded, counterbalanced, two-period crossover with a minimum fourteen-day washout. Two co-primary composite endpoints, content and presentation, are analysed for non-inferiority under a Bayesian hierarchical model, with a frequentist linear mixed model as the convergence check. Inter-rater reliability is reported as Krippendorff’s α; a Monte-Carlo power analysis of the fixed clustered design is pre-specified. Discussion: The design isolates summarisation quality from clinical decision-making by scoring both summary types against the same source report under blinding, counterbalancing, and a fourteen-day washout. Conclusion: The QSI rubric, the counterbalanced crossover, and the pre-specified Bayesian primary with frequentist convergence check define a replicable protocol for early-stage evaluation of AI-assisted summarisation in tertiary genomics reporting; observed variance components will inform sample-size determination for Phase 2.

FoundationOneCDx is approved in the US and Japan as a companion diagnostic test to identify patients with cancer who may benefit from treatment with 30 drug therapies in the US and 23 in Japan. Tumor profiling with FoundationOneCDx also detects genomic findings with evidence of clinical significance that may inform clinical care decisions beyond companion diagnostic claims. This observational study reports the breadth and impact of clinical decision insights from FoundationOneCDx solid tumor profiles. Consecutive test result reports for patients with solid tumor diagnoses (n = 109 695) were retrospectively analyzed for clinically significant predictive, prognostic, and diagnostic genomic alterations and signatures, determined in accordance with professional guidelines. Interventional clinical trials with targeted therapies or immune checkpoint inhibitors were matched to tumor profiles based on evidence that the genomic finding may be an actionable, investigational, or hypothetical target in the patient’s tumor type. In 14 predefined cancer types (80.7% of analyzed solid tumors), predictive, prognostic, and diagnostic markers were reported in 47.6%, 13.2%, and 4.5% of samples, respectively, accounting for a total of 51.2% of tumor profiles. Pan-cancer predictive markers of tumor mutational burden (TMB) of 10 or more mutations per megabase, high microsatellite instability (MSI), or NTRK1/2/3 fusions were observed in 15.6%, 2.0%, and 0.1% of solid tumors, respectively. Most solid tumor profiles (89.2%) had genomic results that could theoretically inform decisions on the selection of immunotherapy and targeted therapy clinical trials. For this real-world population of patients with FoundationOneCDx solid tumor profiles in the routine course of clinical care, clinically significant findings were reported for approximately half of patients with genomic results.

Unadjusted for known treatment assignment imbalances, patients receiving 1st line ICPI vs. chemotherapy have more favorable outcomes when TMB ≥ 10 but not TMB < 10. Kaplan-Meier curves are unadjusted for imbalances (propensity weights applied). TTNT is shown by drug class for (A) TMB < 10, and (B) TMB ≥ 10. OS is shown by drug class for (C) TMB < 10, and (D) TMB ≥ 10. X-axis is truncated at 36 months. Overall survival estimates are left truncated (see Methods) with at-risk tables adjusted accordingly. Interaction models can be found in Supplemental Figure 9.

1st line treatment-TMB interaction models from Figure 5. The (A) TTNT and (B) OS interaction models are shown for for propensity adjusted analyses in Figure 5.

Unadjusted for known treatment assignment imbalances, patients receiving 1st line ICPI vs. chemotherapy have more favorable outcomes when TMB ≥ 10 but not TMB < 10. Kaplan-Meier curves are unadjusted for imbalances (propensity weights applied). TTNT is shown by drug class for (A) TMB < 10, and (B) TMB ≥ 10. OS is shown by drug class for (C) TMB < 10, and (D) TMB ≥ 10. X-axis is truncated at 36 months. Overall survival estimates are left truncated (see Methods) with at-risk tables adjusted accordingly. Interaction models can be found in Supplemental Figure 9.

Patients with advanced gastroesophageal cancer (mEG) and tumor mutational burden ≥10 mut/Mb (TMB ≥ 10) have more favorable outcomes on immune checkpoint inhibitor (ICPI) monotherapy compared with chemotherapy in subgroup analyses of randomized controlled trials. We sought to evaluate the robustness of these associations in real-world settings where patients and practices are more diverse. A total of 362 2 L and 692 1 L patients, respectively received ICPI (n = 99, 33) or chemotherapy (n = 263, 659) across approximately 280 U.S. academic or community-based cancer clinics March 2014–July 2021. Deidentified data were captured into a real-world clinico-genomic database. All patients underwent Foundation Medicine testing. Time to next treatment (TTNT) and overall survival (OS) comparing ICPI versus chemotherapy were adjusted for treatment assignment imbalances using propensity scores. 2L: TMB ≥ 10 had more favorable TTNT [median 24 vs. 4.1 months; HR: 0.19; 95% confidence interval (CI): 0.09–0.44; P = 0.0001] and OS (median 43.1 vs. 6.2 months; HR: 0.24; 95% CI: 0.011–0.54; P = 0.0005), TMB < 10 did not (P > 0.05). 1L: TMB ≥ 10 had more favorable TTNT (not reached vs. median 4.1 months; HR: 0.13; 95% CI: 0.03–0.48; P = 0.0024) and OS (not reached vs. median 17.1 months; HR: 0.30; 95% CI: 0.08–1.14; P = 0.078), TMB < 10 had less favorable TTNT (median 2.8 vs. 6.5 months; HR: 2.36; 95% CI: 1.25–4.45; P = 0.008) and OS (median 4.5 vs. 13.1 months; HR: 1.82, 95% CI: 0.87–3.81; P = 0.11). TMB ≥ 10 robustly identifies patients with mEG with more favorable outcomes on 2 L ICPI monotherapy versus chemotherapy. 1 L data are more limited, but effects are consistent with 2L.Significance:Using real-world data, we sought to evaluate robustness of these clinical associations using the same assay platform and biomarker cut-off point used in both clinical trials and pan-tumor CDx approvals for later treatment lines. TMB ≥ 10 robustly identified patients with mEG with more favorable outcomes on ICPI monotherapy versus chemotherapy and suggests this subset of patients could be targeted for further trial development.

Patient demographics in the second line patient cohort.

Propensity Weighting and Balance Adjustment: TMB. The pre- and post-adjustment balance for (A) 2nd line cohort with TMB < 10, (B) 2nd line cohort with TMB 10 or greater, (C) 1st line cohort with TMB < 10, (D) 1st line cohort with TMB 10 or greater.

Patient demographics in Sequential Cohort