Reproducibility of subtle MRI findings suggestive of minimal endometriosis: agreement, Kappa paradox, and interpretive variability

Authors:
Publisher website
Google Scholar
Abstract
Purpose To evaluate intraobserver and interobserver agreement in the interpretation of subtle MRI findings classified as minimal endometriosis, to assess the impact of the Kappa paradox in low-prevalence compartments, and to compare blinded rereads with the original prospective reports. Materials and methods In this retrospective study, 100 pelvic MRI examinations (50 originally reported as minimal or incipient endometriosis and 50 originally reported as normal) were independently reviewed by three radiologists on two blinded occasions 4–8 weeks apart. Seven anatomical compartments were assessed. Agreement was measured with Cohen’s Kappa, Fleiss’ Kappa, PABAK, Gwet’s AC1, and positive/negative agreement. Original reports were used as a clinical reference for descriptive comparison, and exact McNemar tests assessed directional asymmetry of discordance. Results Intraobserver agreement for any finding was fair (mean Kappa = 0.36, 95%CI 0.16–0.54; AC1 = 0.49), and interobserver agreement was also fair (Fleiss Kappa = 0.28; AC1 = 0.40). Agreement was consistently stronger for negative than for positive classifications. Ovarian findings showed the highest reproducibility, whereas very low-prevalence compartments showed a marked Kappa paradox. Multicategory analysis of uterosacral ligament laterality reduced agreement, mainly because unilateral-versus-bilateral disagreement remained frequent among positive cases. Comparison with original reports showed slight-to-moderate agreement (Kappa 0.18–0.46), with specificity (76–86%) consistently exceeding sensitivity (32–66%), indicating a more restricted threshold on blinded rereads, particularly for uterosacral ligament involvement. Conclusions Agreement for subtle MRI findings interpreted as minimal endometriosis was only fair and clearly lower than that usually reported for deep endometriosis. Low-prevalence compartments were strongly affected by the Kappa paradox, and the low-to-moderate discordance with original reports, driven primarily by lower sensitivity on blinded rereads, supports the need for clearer operational criteria and more standardized interpretation.

This publication has 27 references indexed in Scilit: