JCJulien Cobert

Julien Cobert

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Julien Cobert has a total of 138 co-authors. Here are the top co-authors by number of shared publications:

  • John Boscardin

    8 shared publications
  • Karthik Raghunathan

    Department of Anesthesia, Duke University, Durham, North Carolina

    8 shared publications
  • Edie Espejo

    6 shared publications
  • Hunter Mills

    Bakar Computational Health Sciences Institute, University of California, San Francisco, San Francisco, CA, United States of America

    6 shared publications
  • Vijay Krishnamoorthy

    Duke Univ, Med Ctr, Dept Anesthesiol, Crit Care & Perioperat Populat Hlth Res CAPER Unit, Durham, NC USA

    5 shared publications
  • Alexander K. Smith

    Department of Geriatrics, Palliative, and Extended Care, Veterans Affairs Medical Center, University of California, San Francisco, San Francisco, CA, United States of America

    4 shared publications

Latest Publications

  • Maintaining Rigor in Language Model Studies: Reply

    by Nicole E. Dundas, Tyler Law, Teva Brender, Hunter Mills, Edie Espejo, Timothy Heintz, Arthur W. Wallace, Julien Cobert

    Was not provided for this article.

  • Open Access

    Clinician Self-Referentiality

    by Kristen Vossler, Zoë Kopp, Natalie Stumpf, Teva Brender, Samuel K. McGowan, Shannen Kim, Edie Espejo, John Boscardin, Hunter Mills, Allyson C. Chapman, et al.

  • Open Access

    How ICU clinicians document ‘Futility’: A 10-year analysis of critical care notes using natural language processing notes

    by Teva D. Brender, Natalie Stumpf, Kristen Vossler, Shannen Kim, Hunter Mills, Albert Lee, Hao Yuan Yang, Timothy Heintz, Edie Espejo, John Boscardin, et al.

  • Open Access

    Time and Motion Analysis of Controlled Substance Disposals: A Study of Workflows at a Single Center using Automated Dispensing Cabinets

    by Cassandra W. M.S. Cu, Timothy Heintz, Nicole B.S. Dundas, Hassan Farhan, Hunter M.S. Mills, Jennifer W. Pharm.D. Lee, Christian M.S. Williams, Arthur W. M.D. Wallace, Julien Cobert

  • Patient Race and Preferred Language Influence the Use of Physical Restraints on Nonintubated Intensive Care Unit Patients

    by Samuel K. McGowan, Hayley B. Gershengorn, Andrew Sudler, Edie Espejo, John Boscardin, Lingsheng Li, Alexander K. Smith, Deepshikha C. Ashana, Karthik Raghunathan, Shannen Kim, et al.

    Rationale: Physical limb restraints are commonly used in intensive care units (ICUs) to protect patients and staff but are associated with increased morbidity and disparities in care, particularly in intubated patients. Whether disparities in restraint use persist for nonintubated patients remains less clear. Objectives: We sought to identify whether patient race, ethnicity, and preferred language are associated with restraint use in nonintubated patients across multiple ICUs in a large U.S. hospital system. Methods: We performed a retrospective cohort study using electronic health record data across five ICUs within the University of California, San Francisco, from 2013 to 2022. We included adults who were 18 years of age and older. We excluded patients who received mechanical ventilation during their ICU stay. Our primary independent variables were primary language, race, and ethnicity. The outcome of interest was restraint use, defined as at least one restraint order placed during the patient’s ICU stay. We modeled any restraint use using a multivariable logistic regression adjusted for sociodemographic and clinical covariates and explored interactions of our primary exposures using sensitivity analyses and Wald testing. Results: Across 22,259 unique ICU admissions, we identified 11,676 nonintubated patients. Of these, 2,411 (20%) received an order for physical restraints. In a multivariable regression model, compared with English, Chinese (all dialects) (odds ratio [OR], 1.57; 95% confidence interval [CI], 1.31–1.87) and a language other than Chinese, English, or Spanish (OR, 1.60; 95% CI, 1.36–1.89) were associated with increased use of restraints. Patients identifying as Black or African American were also more likely to be restrained at least once during the encounter (OR, 1.51; 95% CI, 1.27–1.79) compared with non-Hispanic White patients. Conclusions: Patients preferring Chinese or any language other than English or Spanish and those who identify as Black are more likely to be restrained in the ICU when not intubated. Interventions to minimize the use of unnecessary physical restraints could improve an inequity known to be associated with downstream harms.


  • Open Access

    Validity of two subjective skin tone scales and its implications on healthcare model fairness

    by Cassandra W. Cu, Nicole E. Dundas, Timothy Heintz, Zahida A. Sheikh, Bianca Alonso-Bermudez, Jasmine Walker, Avery Wooten, Anusha Badathala, Allyson Chapman, Odinakachukwu Ehie, et al.

    Skin tone assessments are critical for fairness evaluation in healthcare algorithms (e.g., pulse oximetry) but lack validation. Using prospectively collected facial images from 90 hospitalized adults at the San Francisco VA, three independent annotators rated facial regions in triplicate using Fitzpatrick (I–VI) and Monk (1–10) skin tone scales. Patients also self-identified their skin tone. Annotator confidence was recorded using 5-point Likert scales. Across 810 images in 90 patients (9 images each), within-rater agreement was high, but inter-annotator agreement was moderate to low. Annotators frequently rated patients as darker when patients self-identified as lighter, and lighter when patients self-identified as darker. In linear mixed-effects models controlling for facial region and annotator confidence, darker self-reported skin tones were associated with lighter annotator scores. These findings highlight challenges in consistent skin tone labeling and suggest that current methods for assessing representation in biosensor-based algorithm studies may be influenced by labeling bias.


  • Open Access

    Lexical associations can characterize clinical documentation trends related to palliative care and metastatic cancer

    by Hao Yuan Yang, Karthik Raghunathan, Eric Widera, Steven Z. Pantilat, Teva Brender, Timothy A. Heintz, Edie Espejo, John Boscardin, Hunter Mills, Albert Lee, et al.

    Palliative care is known to improve quality of life in advanced cancer. Natural language processing offers insights to how documentation around palliative care in relation to metastatic cancer has changed. We analyzed inpatient clinical notes using unsupervised language models that learn how words related to metastatic cancer (e.g. “mets”, “metastases”) and palliative care (e.g. “palliative care”, “pal care”) appear relationally and change over time. We included any note from adults hospitalized at the University of California, San Francisco system. The primary outcome was how similarly terms related to metastatic cancer and palliative care appeared in notes using a mathematical approach (cosine similarity). We used word2vec to model language numerically as vectors. Relational data between vectors was captured using cosine similarity. We performed linear regression to identify changes in these relationships of terms over time. As a sensitivity analysis, we performed the same analysis per year restricted only to patients with an ICD-9/10 diagnosis code for metastatic cancer. Metastatic cancer and palliative care terms appeared in similar contexts in clinical notes each year, suggesting a close relationship in documentation. However, over time, this relationship weakened, with these terms becoming less commonly used together as measured by cosine similarities. We found similar trends when we retrained models just on patients with a diagnosis code for metastatic cancer. Text in clinical notes offers unique insights into how medical providers document palliative care in patients with advanced malignancies and how these documentation practices evolve over time.

  • Preliminary Development and Validation of Automated Nociception Recognition Using Computer Vision in Perioperative Patients

    by Timothy A. Heintz, Anusha Badathala, Avery Wooten, Cassandra W. Cu, Alfred Wallace, Benjamin Pham, Arthur W. Wallace, Julien Cobert

    Background: Effective pain recognition and treatment in perioperative environments reduce length of stay and decrease risk of delirium and chronic pain. The authors sought to develop and validate preliminary computer vision-based approaches for nociception detection in hospitalized patients. Methods: This was a prospective observational cohort study using red–green–blue camera detection of perioperative patients. Adults (18 yr or older) admitted for surgical procedures to the San Francisco Veterans Affairs Medical Center (San Francisco, California) were included across two study phases: (1) the algorithm development phase and (2) the internal validation phase. Continuous recordings occurred perioperatively across any postoperative setting. The authors inputted facial images into convolutional neural networks using a pretrained backbone to classify (1) the Critical Care Pain Observation Tool (CPOT) and (2) the numeric rating scale. Outcomes were binary pain/no pain. We performed external validation for CPOT and numerical rating scale classification on data from the University of Northern British Columbia (Prince George, Canada)–McMaster University (Hamilton, Canada) and the Delaware Pain Database. Perturbation models were used for explainability. Results: The study included 130 patients for development, 77 patients for the validation cohort, and 25 patients from University of Northern British Columbia–McMaster University and 229 patients from Delaware datasets for external validation. Model areas under the curve of the receiver operating characteristic for CPOT models were 0.71 (95% CI, 0.70 to 0.74) on the development cohort, 0.91 (95% CI, 0.90 to 0.92) on the San Francisco Veterans Affairs Medical Center validation cohort, 0.91 (95% CI, 0.89 to 0.93) on University of Northern British Columbia–McMaster University, and 0.80 (95% CI, 0.75 to 0.85) on Delaware. The numeric rating scale model had lower performance (area under the receiver operating characteristics curve, 0.58 [95% CI, 0.55 to 0.61]). Brier scores improved after calibration across multiple different techniques. Perturbation models for CPOT models revealed eyebrows, nose, lips, and forehead were most important for model prediction. Conclusions: Automated nociception detection using computer vision alone is feasible but requires additional testing and validation given the small datasets used. Future multicenter observational studies are required to better understand the potential for automated continuous assessments for nociception detection in hospitalized patients.


  • Variation in Mentions of Race and Ethnicity in Notes in Intensive Care Units Across a Health Care System

    by Julien Cobert, Edie Espejo, John Boscardin, Hunter Mills, Deepshikha Ashana, Karthik Raghunathan, Timothy A. Heintz, Allyson Cook Chapman, Alex K. Smith, Sei Lee

    Background: Social constructs like race can affect how patients are perceived and impact care. This study investigated whether mentions of race in notes for critically ill patients differed according to patients’ race. Methods: This retrospective cohort study included intensive care unit notes for adults (≥18 years old) admitted to any of 6 intensive care units at University of California, San Francisco, from 2012 through 2020. Notes were linked to National Provider Identifier records to obtain note writer characteristics. Logistic regression analysis with robust SEs clustered on note writers was adjusted for patient-, note- and clinician-level characteristics. Any race or ethnicity mention was the outcome of interest. Results: Among 5573 patients with 292 457 notes by 9742 unique note writers, 3225 patients (57.9%) self-reported their race as White, 997 (17.9%) as Asian, 860 (15.4%) as Latinx, and 491 (8.8%) as Black. Note writers documented race/ethnicity for 20.8% of Black, 10.9% of Latinx, 9.1% of White, and 4.4% of Asian patients. Black patients were more likely than White patients to have race mentioned in notes (adjusted odds ratio, 2.05 [95% CI, 1.49-2.82]). Conclusions: Black patients were more than twice as likely as White patients to have race mentioned in notes. Note language containing information on social constructs has consequences for clinicians and patients reading notes and for algorithms trained on clinical notes.