LRLisa V. Rubenstein

Lisa V. Rubenstein

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Lisa V. Rubenstein has a total of 91 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Measuring Primary Care Productivity in the Era of Interprofessional Team Care: Stakeholder, Scoping Review, and Implementation Perspectives

    by Lisa V. Rubenstein, Sydne J. Newberry, Ishita Ghai, Aneesa Motala, Idamay Curtis, Paul G. Shekelle, Todd H. Wagner, L. Diem Tran, Stephen D. Fihn, Karin M. Nelson

    Policy Points: The economics and outcomes of modern primary care are substantially driven by investment in interprofessional clinical team members aimed at delivering complex, population health–oriented care. Neither interprofessional primary care team investment nor the work products expected in return are well represented in current commonly used productivity metrics. Stakeholder perspective‐guided scoping review followed by expert panel input on measure development showed the feasibility of applying economic methods for assessing primary care productivity relative to multiple high‐value products. Context: Current primary care productivity measures do not account for investment in interprofessional primary care teams in relation to primary care goals and thus are insufficient for assessing and improving primary care efficiency and productivity. We explored alternative productivity measurement methods. Methods: We conducted a scoping review of English language literature between 2008 and 2023 to identify articles that assessed primary care practice productivity and efficiency. We reviewed the full texts of articles to assess their analytic models including inputs, outputs, and context measures. Using scoping review results to inform content, we conducted a modified Delphi expert panel to discuss potential use cases, analytic approaches, and data elements for new primary care productivity measures. Panelists anonymously voted on recommendations for guiding near‐term measure development and testing. Findings: Evidence review identified 25 included studies. The majority (76%, 19/25) used an economic model‐based productivity calculation, predominantly estimated using data envelopment analysis (DEA), with stochastic frontier analysis accounting for most of the remainder. Primary care staffing was the most common input, included in 84% of the 19 economic model studies. As outputs, over half (53%) of studies included measures of quality of care, whereas the same proportion included numbers of clinical activities. No studies used patient‐reported experiences of care. Expert panelists recommended that initial measure development focus on primary care practice efficiency improvement, building the measure on routinely collected health system data, accounting for the clinical team's full‐time equivalent staffing, and incorporating quality of care. Panelists endorsed DEA while also acknowledging that other approaches had potential. Conclusions: We identified measurement approaches that aligned with both economic and foundational primary care principles but none that were implemented for routine use. Opportunities exist to develop metrics that accurately reflect primary care structures, goals, and values.


  • Open Access

    Learning from national implementation of the Veterans Affairs Clinical Resource Hub (CRH) program for improving access to care: protocol for a six year evaluation

    by Lisa V. Rubenstein, Idamay Curtis, Chelle L. Wheat, David E. Grembowski, Susan E. Stockdale, Peter J. Kaboli, Jean Yoon, Bradford L. Felker, Ashok S. Reddy, Karin M. Nelson

    Background: The Veterans Affairs (VA) Clinical Resource Hub (CRH) program aims to improve patient access to care by implementing time-limited, regionally based primary or mental health staffing support to cover local staffing vacancies. VA’s Office of Primary Care (OPC) designed CRH to support more than 1000 geographically disparate VA outpatient sites, many of which are in rural areas, by providing virtual contingency clinical staffing for sites experiencing primary care and mental health staffing deficits. The subsequently funded CRH evaluation, carried out by the VA Primary Care Analytics Team (PCAT), partnered with CRH program leaders and evaluation stakeholders to develop a protocol for a six-year CRH evaluation. The objectives for developing the CRH evaluation protocol were to prospectively: 1) identify the outcomes CRH aimed to achieve, and the key program elements designed to achieve them; 2) specify evaluation designs and data collection approaches for assessing CRH progress and success; and 3) guide the activities of five geographically dispersed evaluation teams. Methods: The protocol documents a multi-method CRH program evaluation design with qualitative and quantitative elements. The evaluation’s overall goal is to assess CRH’s return on investment to the VA and Veterans at six years through synthesis of findings on program effectiveness. The evaluation includes both observational and quasi-experimental elements reflecting impacts at the national, regional, outpatient site, and patient levels. The protocol is based on program evaluation theory, implementation science frameworks, literature on contingency staffing, and iterative review and revision by both research and clinical operations partners. Discussion: Health systems increasingly seek to use data to guide management and decision-making for newly implemented clinical programs and policies. Approaches for planning evaluations to accomplish this goal, however, are not well-established. By publishing the protocol, we aim to increase the validity and usefulness of subsequent evaluation findings. We also aim to provide an example of a program evaluation protocol developed within a learning health systems partnership.


  • JGIM Welcomes Quality Improvement and Implementation Science Submissions on Healthcare Delivery Change

    by Lisa V. Rubenstein, Ian M. Kronish, Rebekah J. Walker, Luci K. Leykum, Michael K. Ong, Steven M. Asch

  • Primary Care Productivity: Findings from the Literature and Perspectives from a Stakeholder Panel

    by Susanne Hempel, Idamay Curtis, Stephan D. Fihn, Annie Brothers, Marjorie Danz, Karin M. Nelson, Aneesa Motala, Lisa V. Rubenstein

    To advance understanding of primary care clinic productivity, the authors reviewed relevant literature and convened a diverse set of stakeholders to explore definitions of productivity, input, and output in primary care; identify relevant tools; and establish consensus on key aspects of primary care productivity.


  • Facilitating ethical quality improvement initiatives: Design and implementation of an initiative-specific ethics committee

    by Melissa M. Bottrell, Alissa Simon, Cynthia Geppert, Evelyn T. Chang, Steven M. Asch, Lisa Rubenstein


  • Open Access

    The Impact of Job Role on Health-Care Workers’ Definitions of Patient-Centered Care

    by Jessica L Moreau, Alison B Hamilton, Elizabeth M Yano, Lisa V Rubenstein, Susan E Stockdale

    While patient-centered care (PCC) is a widely accepted aspect of health-care quality, its definition is still the subject of debate. We investigated health-care workers’ definitions of PCC by level of patient contact in job roles. Our qualitative study involved semi-structured interviews with key stakeholder employees (n = 66) at 6 Veterans’ Affairs health-care locations in Southern California. Interviews were recorded, transcribed, coded for definitions of PCC, and analyzed by participants’ self-described level of patient contact. Stakeholders whose role primarily involved patient contact tended to define PCC through: patient as a person, patient preferences, and shared decision-making. Stakeholders whose role did not primarily involve patient contact tended to define PCC through: patient-centered redesign, customer service, and access to services. Stakeholders with more patient contact emphasized patient-level and interpersonal concepts, while those with less patient contact emphasized system-level and business-oriented concepts. The focus on PCC-as-access may reflect influence of changing institutional climate on definitions of PCC for some stakeholders. To facilitate successful PCC efforts, health-care systems may need to leverage differing but complementary definitions of PCC within its workforce.


  • Open Access

    Eight Priorities for Improving Primary Care Access Management in Healthcare Organizations: Results of a Modified Delphi Stakeholder Panel

    by Lisa Rubenstein, Susanne Hempel, Margie Danz, Danielle Rose, Susan Stockdale, Idamay Curtis, Susan Kirsh

    Objective: To identify priorities for improving healthcare organization management of patient access to primary care based on prior evidence and a stakeholder panel. Background: Studies on healthcare access show its importance for ensuring population health. Few studies show how healthcare organizations can improve access. Methods: We conducted a modified Delphi stakeholder panel anchored by a systematic review. Panelists (N = 20) represented diverse stakeholder groups including patients, providers, policy makers, purchasers, and payers of healthcare services, predominantly from the Veterans Health Administration. A pre-panel survey addressed over 80 aspects of healthcare organization management of access, including defining access management. Panelists discussed survey-based ratings during a 2-day in-person meeting and re-voted afterward. A second panel process focused on each final priority and developed recommendations and suggestions for implementation. Results: The panel achieved consensus on definitions of optimal access and access management on eight urgent and important priorities for guiding access management improvement, and on 1–3 recommendations per priority. Each recommendation is supported by referenced, panel-approved suggestions for implementation. Priorities address two organizational structure targets (interdisciplinary primary care site leadership; clearly identified group practice management structure); four process improvements (patient telephone access management; contingency staffing; nurse management of demand through care coordination; proactive demand management by optimizing provider visit schedules), and two outcomes (quality of patients’ experiences of access; provider and staff morale). Recommendations and suggestions for implementation, including literature references, are summarized in a panelist-approved, ready-to-use tool. Conclusions: A stakeholder panel informed by a pre-panel systematic review identified eight action-oriented priorities for improving access and recommendations for implementing each priority. The resulting tool is suitable for guiding the VA and other integrated healthcare delivery organizations in assessing and initiating improvements in access management, and for supporting continued research.


  • Open Access

    Spread tools: a systematic review of components, uptake, and effectiveness of quality improvement toolkits

    by Susanne Hempel, Claire O’hAnlon, Yee Wei Lim, Margie Danz, Jody Larkin, Lisa Rubenstein

    The objective was to conduct a systematic review of toolkit evaluations intended to spread interventions to improve healthcare quality. We aimed to determine the components, uptake, and effectiveness of publicly available toolkits. We searched PubMed, CINAHL, and the Web of Science from 2005 to May 2018 for evaluations of publicly available toolkits, used a forward search of known toolkits, screened references, and contacted topic experts. Two independent reviewers screened publications for inclusion. One reviewer abstracted data and appraised the studies, checked by a second reviewer; reviewers resolved disagreements through discussion. Findings, summarized in comprehensive evidence tables and narrative synthesis addressed the uptake and utility, procedural and organizational outcomes, provider outcomes, and patient outcomes. In total, 77 studies evaluating 72 toolkits met inclusion criteria. Toolkits addressed a variety of quality improvement approaches and focused on clinical topics such as weight management, fall prevention, vaccination, hospital-acquired infections, pain management, and patient safety. Most toolkits included introductory and implementation material (e.g., research summaries) and healthcare provider tools (e.g., care plans), and two-thirds included material for patients (e.g., information leaflets). Pre-post studies were most common (55%); 10% were single hospital evaluations and the number of participating staff ranged from 17 to 704. Uptake data were limited and toolkit uptake was highly variable. Studies generally indicated high satisfaction with toolkits, but the perceived usefulness of individual tools varied. Across studies, 57% reported on adherence to clinical procedures and toolkit effects were positive. Provider data were reported in 40% of studies but were primarily self-reported changes. Only 29% reported patient data and, overall, results from robust study designs are missing from the evidence base. The review documents publicly available toolkits and their components. Available uptake data are limited but indicate variability. High satisfaction with toolkits can be achieved but the usefulness of individual tools may vary. The existing evidence base on the effectiveness of toolkits remains limited. While emerging evidence indicates positive effects on clinical processes, more research on toolkit value and what affects it is needed, including linking toolkits to objective provider behavior measures and patient outcomes. PROSPERO registration number: PROSPERO 2014: CRD42014013930 .


  • Open Access

    Long-term impact of evidence-based quality improvement for facilitating medical home implementation on primary care health professional morale

    by Lisa S. Meredith, Benjamin Batorsky, Matthew Cefalu, Jill E. Darling, Susan E. Stockdale, Elizabeth M. Yano, Lisa V. Rubenstein

    Poor morale among primary care providers (PCPs) and staff can undermine the success of patient-centered care models such as the patient-centered medical home that rely on highly coordinated inter-professional care teams. Medical home literature hypothesizes that participation in quality improvement can ease medical home transformation. No studies, however, have assessed the impact of quality improvement participation on morale (e.g., burnout or dissatisfaction) during transformation. The objective of this study is to examine whether primary care practices participating in evidence-based quality improvement (EBQI) during medical home transformation reduced burnout and increased satisfaction over time compared to non-participating practices. We used a longitudinal quasi-experimental design to examine the impact of EBQI (vs. no EBQI), a multi-level, interdisciplinary approach for engaging frontline primary care practices in developing evidence-based improvement innovations and tools for spread on PCP and staff morale following the 2010 national implementation of the medical home model in the Veterans Health Administration. The sample included 356 primary care employees (107 primary care providers and 249 staff) from 23 primary care practices (6 intervention and 17 comparison) within one Veterans Health Administration region. Three intervention practices began EBQI in 2011 (early) and three more began EBQI in 2012 (late). Three waves of surveys were administered across 42 months beginning in November 2011 and ending in January 2016 approximately 2 years 18 months apart. We used repeated measures analysis of the survey data on medical home teams. Main outcome measures were the emotional exhaustion subscale from the Maslach Burnout Inventory, and job satisfaction. Six of 26 approved EBQI innovations directly addressed provider and staff morale; all 26 addressed medical home implementation challenges. Survey rates were 63% for baseline and 48% for both follow-up waves. Age was associated with lower burnout among PCPs (p = .039) and male PCPs had higher satisfaction (p = .037). Controlling for practice and PCP/staff characteristics, burnout increased by 5 points for PCPs in comparison practices (p = .024) and decreased by 1.4 points for early and 6.8 points (p = .039) for the late EBQI practices. Engaging PCPs and staff in EBQI reduced burnout over time during medical home transformation.