Charlene R. Weir

The heuristic evaluation (HE) method is one of the most common in the suite of tools for usability evaluations because it is a fast, inexpensive and resource-efficient process in relation to the many usability issues it generates. The method emphasizes completely independent initial expert evaluations. Inter-rater reliability and agreement coefficients are not calculated. The variability across evaluators, even dual domain experts, can be significant as is seen in the case study here. The implications of this wide variability mean that results are unique to each HE, results are not readily reproducible and HE research on usability is not yet creating a uniform body of knowledge. We offer recommendations to improve the science by incorporating selected techniques from qualitative research: calculating inter-rater reliability and agreement scores, creating a codebook to define concepts/categories and offering crucial information about raters' backgrounds, agreement techniques and the evaluation setting.

How can health sciences librarians and biomedical informaticians offer relevant support to Clinical and Translational Science Award (CTSA) personnel?The Spencer S. Eccles Health Sciences Library and the associate vice president for information technology ...

Objective The evidence base for information technology (IT) has been criticized, especially with the current emphasis on translational science. The purpose of this paper is to present an analysis of the role of IT in the implementation of a geriatric education and quality improvement (QI) intervention. Design A mixed-method three-group comparative design was used. The PRECEDE/PROCEED implementation model was used to qualitatively identify key factors in the implementation process. These results were further explored in a quantitative analysis. Method Thirty-three primary care clinics at three institutions (Intermountain Healthcare, VA Salt Lake City Health Care System, and University of Utah) participated. The program consisted of an onsite, didactic session, QI planning and 6 months of intense implementation support. Results Completion rate was 82% with an average improvement rate of 21%. Important predisposing factors for success included an established electronic record and a culture of quality. The reinforcing and enabling factors included free continuing medical education credits, feedback, IT access, and flexible support. The relationship between IT and QI emerged as a central factor. Quantitative analysis found significant differences between institutions for pre–post changes even after the number and category of implementation strategies had been controlled for. Conclusions The analysis illustrates the complex dependence between IT interventions, institutional characteristics, and implementation practices. Access to IT tools and data by individual clinicians may be a key factor for the success of QI projects. Institutions vary widely in the degree of access to IT tools and support. This article suggests that more attention be paid to the QI and IT department relationship.

Previous studies have described the determinants of successful information technology (IT) implementation. In 2003, Kukafka et al. integrated several theoretical perspectives and proposed a framework for IT implementation. This framework is applicable to IT implementation in general but lacks the identification of factors affecting adoption, which are specific to the technology under consideration. We developed and validated a model that specifically identifies factors associated with clinicians' adoption of computerized protocols. The purpose of this paper is to identify the relations between the specific factors associated with intention to use computerized protocols and the high level variables that constitute the framework proposed by Kukafka et al. Incorporation of a specific model into a general schema for IT implementation allows implementers to assess the specific individual, organizational and environmental changes required to bring about successful implementation of computerized protocols. An understanding of clinicians' perceptions specific to the technology in use will allow its seamless integration into an organization's healthcare IT plan. Strategic planning requires enhancing the framework with additional detail related to the specific technology under consideration.