Corey M. Johnson
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Corey M. Johnson has a total of 36 co-authors. Here are the top co-authors by number of shared publications:
- Blake Lee Galbreath
Washington State University
4 shared publications - Jen Saulnier Lange
Washington State University
3 shared publications - Alex Merrill
Washington State University
2 shared publications B. Jane Scales
2 shared publications- Elizabeth Blakesley
Washington State University
2 shared publications Elizabeth Blakesley Lindsay
2 shared publications
Latest Publications
- Open Access
- 1 September 2026
- Published by University of Alberta Libraries in Evidence Based Library and Information Practice
- Vol. 21 (3) , 5-30
Non-Local Librarian Performance Evaluation in the Chat Reference Setting
Objective – This research investigates how well non-local chat reference operators perform in their work with local library patrons at an R1 academic institution in the United States. More specifically, the authors sought to learn how this particular reference librarian population (co-op librarians) handled circumstances surrounding when to create a ticket for a transaction and which positive reference strategies were not utilized that should have been. In addition, the researchers rated the co-op librarians’ overall success in answering local reference questions and the kinds and frequency of false or incorrect information that was offered. Finally, the authors used the length of the reference transaction (by time and message count), reference query type, medical/health related questions, and patron satisfaction (both rater perceived and patron self-reported) as variables to learn more about situations where co-op librarians were more and less successful. Methods – The researchers examined one year’s worth of co-op librarian chat reference records from April 2023 to March 2024. The authors created a rating form to capture structural session information like the length of the transaction, transaction day of week/time, query type, if a ticket was created, and patron self-reported satisfaction. In addition, the form solicited subjective assessment information from the researchers covering the topics detailed in the “Objective” section above. The researchers tabulated aggregated frequencies across the data produced, in addition to running a number of tests for statistical significance. Results – The researchers were surprised to see that about one third of co-op librarian transactions happened during times when local librarians were scheduled to be available to answer these questions. They also predicted that co-op librarian performance would decline when faced with medical/health questions, but largely that was not the case. It was discovered that co-op librarians made referrals about one third of the time, and that among the two thirds that were not referred, about one third of those should have been. Overall, co-op librarians were rated as highly successful with about three quarters of their transactions in the top two of a five-point rating scale. Conclusion – A number of overarching conclusions arose from this study, each with implications for chat reference practice. co-op librarians perform well overall, despite the challenges of frequently receiving questions with local aspects. They serve as a good backup for local librarians when the latter miss or are too busy to pick up chats. They do not shy away from attempting to answer questions, largely employ best practices in their work, infrequently inject false information into transactions, usually garner high patron satisfaction levels, and most importantly are largely successful in answering patron questions. The main struggles co-op librarians have involve dealing with account and interlibrary loan focused questions, and they also could be making referrals/tickets more frequently.
- 1 September 2026
- Published by University of Alberta Libraries in Evidence Based Library and Information Practice
- Vol. 21 (3) , 5-30
Open AccessNon-Local Librarian Performance Evaluation in the Chat Reference Setting
Objective – This research investigates how well non-local chat reference operators perform in their work with local library patrons at an R1 academic institution in the United States. More specifically, the authors sought to learn how this particular reference librarian population (co-op librarians) handled circumstances surrounding when to create a ticket for a transaction and which positive reference strategies were not utilized that should have been. In addition, the researchers rated the co-op librarians’ overall success in answering local reference questions and the kinds and frequency of false or incorrect information that was offered. Finally, the authors used the length of the reference transaction (by time and message count), reference query type, medical/health related questions, and patron satisfaction (both rater perceived and patron self-reported) as variables to learn more about situations where co-op librarians were more and less successful. Methods – The researchers examined one year’s worth of co-op librarian chat reference records from April 2023 to March 2024. The authors created a rating form to capture structural session information like the length of the transaction, transaction day of week/time, query type, if a ticket was created, and patron self-reported satisfaction. In addition, the form solicited subjective assessment information from the researchers covering the topics detailed in the “Objective” section above. The researchers tabulated aggregated frequencies across the data produced, in addition to running a number of tests for statistical significance. Results – The researchers were surprised to see that about one third of co-op librarian transactions happened during times when local librarians were scheduled to be available to answer these questions. They also predicted that co-op librarian performance would decline when faced with medical/health questions, but largely that was not the case. It was discovered that co-op librarians made referrals about one third of the time, and that among the two thirds that were not referred, about one third of those should have been. Overall, co-op librarians were rated as highly successful with about three quarters of their transactions in the top two of a five-point rating scale. Conclusion – A number of overarching conclusions arose from this study, each with implications for chat reference practice. co-op librarians perform well overall, despite the challenges of frequently receiving questions with local aspects. They serve as a good backup for local librarians when the latter miss or are too busy to pick up chats. They do not shy away from attempting to answer questions, largely employ best practices in their work, infrequently inject false information into transactions, usually garner high patron satisfaction levels, and most importantly are largely successful in answering patron questions. The main struggles co-op librarians have involve dealing with account and interlibrary loan focused questions, and they also could be making referrals/tickets more frequently.
- Open Access
- 15 December 2025
- Published by American Library Association in Information Technology and Libraries
- Vol. 44 (4)
AI-Infused Discovery Environments
Although still in its infancy, artificial intelligence (AI) is rapidly making inroads into most facets of the library and education spheres. This paper outlines steps taken to examine Primo Research Assistant, an AI-infused discovery environment, for potential deployment at a large US public research university. The researchers aimed to evaluate the quality and relevance of the AI results in comparison to sources retrieved from the conventional search functionality, as well as the AI system’s multi-paragraph overview reply to the search query. As a starting point, the authors collected 103 search strings from a Primo Zero Result Searches report to approximate a corpus of natural language search queries. For the same research area, it was discovered that there was only limited overlap between the titles returned by the AI tool versus the current discovery layer. The researchers did not find appreciable differences in the numbers of topic-relevant sources between the AI and non-AI search products (Yes = 46.3% vs. Yes = 45.6%, respectively). The overview summary is largely helpful in terms of learning more details about the recommended sources, but it also sometimes misrepresents connections between the sources and the research topic. Given the overall conclusion that the AI system did not constitute a clear advancement or decline in effective information retrieval, the authors will turn to usability testing to aid them in further implementation decisions.
- 15 December 2025
- Published by American Library Association in Information Technology and Libraries
- Vol. 44 (4)
Open AccessAI-Infused Discovery Environments
Although still in its infancy, artificial intelligence (AI) is rapidly making inroads into most facets of the library and education spheres. This paper outlines steps taken to examine Primo Research Assistant, an AI-infused discovery environment, for potential deployment at a large US public research university. The researchers aimed to evaluate the quality and relevance of the AI results in comparison to sources retrieved from the conventional search functionality, as well as the AI system’s multi-paragraph overview reply to the search query. As a starting point, the authors collected 103 search strings from a Primo Zero Result Searches report to approximate a corpus of natural language search queries. For the same research area, it was discovered that there was only limited overlap between the titles returned by the AI tool versus the current discovery layer. The researchers did not find appreciable differences in the numbers of topic-relevant sources between the AI and non-AI search products (Yes = 46.3% vs. Yes = 45.6%, respectively). The overview summary is largely helpful in terms of learning more details about the recommended sources, but it also sometimes misrepresents connections between the sources and the research topic. Given the overall conclusion that the AI system did not constitute a clear advancement or decline in effective information retrieval, the authors will turn to usability testing to aid them in further implementation decisions.
- Open Access
- 17 June 2024
- Published by American Library Association in Information Technology and Libraries
- Vol. 43 (2)
Framework for Measuring Relevancy in Discovery Environments
Institutional discovery environments now serve as central resource databases for researchers in the academic environment. Over the last several decades, there have been numerous discovery layer research inquiries centering primarily on user satisfaction measures of discovery system effectiveness. This study focuses on the creation of a largely automated method for evaluating discovery layer quality, utilizing the bibliographic sources from student research projects. Building on past research, the current study replaces a semiautomated Excel Fuzzy Lookup Add-In process witha fully scripted R-based approach, which employs the stringdist R package and applies the Jaro-Winkler distance metric as the matching evaluator. The researchers consider the error rate incurred by relying solely on an automated matching metric. They also use Open Refine for normalization processes and package the tools together on an OSF site for other institutions to use. Since the R-based approach does not require special processing or time and can be reproduced with minimal effort, it will allow future studies and users of our method to capture larger sample sizes, boosting validity. While the assessment process has been streamlined and shows promise, there remain issues in establishing solid connections between research paper bibliographies and discovery layer use. Subsequent research will focus on creating alternatives to paper titles as search proxies that better resemble genuine information-seeking behavior and comparing undergraduate and graduate student interactions within discovery environments.
- 17 June 2024
- Published by American Library Association in Information Technology and Libraries
- Vol. 43 (2)
Open AccessFramework for Measuring Relevancy in Discovery Environments
Institutional discovery environments now serve as central resource databases for researchers in the academic environment. Over the last several decades, there have been numerous discovery layer research inquiries centering primarily on user satisfaction measures of discovery system effectiveness. This study focuses on the creation of a largely automated method for evaluating discovery layer quality, utilizing the bibliographic sources from student research projects. Building on past research, the current study replaces a semiautomated Excel Fuzzy Lookup Add-In process witha fully scripted R-based approach, which employs the stringdist R package and applies the Jaro-Winkler distance metric as the matching evaluator. The researchers consider the error rate incurred by relying solely on an automated matching metric. They also use Open Refine for normalization processes and package the tools together on an OSF site for other institutions to use. Since the R-based approach does not require special processing or time and can be reproduced with minimal effort, it will allow future studies and users of our method to capture larger sample sizes, boosting validity. While the assessment process has been streamlined and shows promise, there remain issues in establishing solid connections between research paper bibliographies and discovery layer use. Subsequent research will focus on creating alternatives to paper titles as search proxies that better resemble genuine information-seeking behavior and comparing undergraduate and graduate student interactions within discovery environments.
- 21 January 2024
- Published by Taylor & Francis in The Reference Librarian
- Vol. 65 (1-2) , 34-58
Service, Interrupted: Analyzing Chat Reference Referrals
- 21 January 2024
- Published by Taylor & Francis in The Reference Librarian
- Vol. 65 (1-2) , 34-58
Service, Interrupted: Analyzing Chat Reference Referrals
- 29 November 2021
- Published by Taylor & Francis in Geopolitics
- Vol. 27 (1) , 1-4
Editorial
After the challenges of the pandemic over the past two years, we want to begin by acknowledging the contributions of authors, reviewers, and editors during these difficult times. Thank you for supp...
- 29 November 2021
- Published by Taylor & Francis in Geopolitics
- Vol. 27 (1) , 1-4
Editorial
After the challenges of the pandemic over the past two years, we want to begin by acknowledging the contributions of authors, reviewers, and editors during these difficult times. Thank you for supp...
- Open Access
- 15 June 2021
- Published by American Library Association in Information Technology and Libraries
- Vol. 40 (2)
Framework for Measuring Relevancy in Discovery Environments
Discovery environments are ubiquitous in academic libraries but studying their effectiveness and use in an academic environment has mostly centered around user satisfaction, experience, and task analysis. This study aims to create a quantitative, reproducible framework to test the relevancy of results and the overall success of Washington State University’s discovery environment (Primo by Ex Libris). Within this framework, the authors use bibliographic citations from student research papers submitted as part of a required university class as the proxy for relevancy. In the context of this study, the researchers created a testing model that includes: (1) a process to produce machine-generated keywords from a corpus of research papers to compare against a set of human-created keywords, (2) a machine process to query a discovery environment to produce search result lists to compare against citation lists, and (3) four metrics to measure the comparative success of different search strategies and the relevancy of the results. This framework is used to move beyond a sentiment or task-based analysis to measure if materials cited in student papers appear in the results list of a production discovery environment. While this initial test of the framework produced fewer matches between researcher-generated search results and student bibliography sources than expected, the authors note that faceted searches represent a greater success rate when compared to open-ended searches. Future work will include comparative (A/B) testing of commonly deployed discovery layer configurations and limiters to measure the impact of local decisions on discovery layer efficacy as well as noting where in the results list a citation match occurs.
- 15 June 2021
- Published by American Library Association in Information Technology and Libraries
- Vol. 40 (2)
Open AccessFramework for Measuring Relevancy in Discovery Environments
Discovery environments are ubiquitous in academic libraries but studying their effectiveness and use in an academic environment has mostly centered around user satisfaction, experience, and task analysis. This study aims to create a quantitative, reproducible framework to test the relevancy of results and the overall success of Washington State University’s discovery environment (Primo by Ex Libris). Within this framework, the authors use bibliographic citations from student research papers submitted as part of a required university class as the proxy for relevancy. In the context of this study, the researchers created a testing model that includes: (1) a process to produce machine-generated keywords from a corpus of research papers to compare against a set of human-created keywords, (2) a machine process to query a discovery environment to produce search result lists to compare against citation lists, and (3) four metrics to measure the comparative success of different search strategies and the relevancy of the results. This framework is used to move beyond a sentiment or task-based analysis to measure if materials cited in student papers appear in the results list of a production discovery environment. While this initial test of the framework produced fewer matches between researcher-generated search results and student bibliography sources than expected, the authors note that faceted searches represent a greater success rate when compared to open-ended searches. Future work will include comparative (A/B) testing of commonly deployed discovery layer configurations and limiters to measure the impact of local decisions on discovery layer efficacy as well as noting where in the results list a citation match occurs.
- Open Access
- 7 May 2021
- Published by University of Michigan Library in Weave: Journal of Library User Experience
- Vol. 4 (1)
Implementing Universal Design for Learning Elements in the Online Learning Materials of a First-Year Required Course
Universal Design for Learning (UDL) is a pedagogical design theory that accounts for variability across and within learners, both in physical classrooms and with online learning materials. Librarians at Washington State University implemented elements reflective of specific UDL checkpoints in a heavily-used LibGuide for a required undergraduate course. Online learning objects create opportunities for meeting UDL’s goals of not only multiple means of representation, but also engagement and action and expression extending beyond the baseline obligation to meet standard accessibility guidelines. We discuss our implementation of UDL elements, changes in LibGuide use before and after the introduction of UDL features, and usability study results, focusing on how students who face learning barriers view UDL. We explain considerations for successfully implementing UDL in online materials and the potential benefits.
- 7 May 2021
- Published by University of Michigan Library in Weave: Journal of Library User Experience
- Vol. 4 (1)
Open AccessImplementing Universal Design for Learning Elements in the Online Learning Materials of a First-Year Required Course
Universal Design for Learning (UDL) is a pedagogical design theory that accounts for variability across and within learners, both in physical classrooms and with online learning materials. Librarians at Washington State University implemented elements reflective of specific UDL checkpoints in a heavily-used LibGuide for a required undergraduate course. Online learning objects create opportunities for meeting UDL’s goals of not only multiple means of representation, but also engagement and action and expression extending beyond the baseline obligation to meet standard accessibility guidelines. We discuss our implementation of UDL elements, changes in LibGuide use before and after the introduction of UDL features, and usability study results, focusing on how students who face learning barriers view UDL. We explain considerations for successfully implementing UDL in online materials and the potential benefits.
- 2 October 2019
- Published by Taylor & Francis in Public Services Quarterly
- Vol. 15 (4) , 300-318
An Analysis of Help Seeking Behaviors in a Required First Year Course
The aim of this study is to show student perception of the value of library reference and consultation work in the building of information literacy skills. Using student answers to questions about library research help received in a required first year course, the authors analyzed motivations for use of librarian aid, and the types of assistance sought and that ideally could have been utilized. The authors found students primarily received librarian help with locating books, and that students often believed they would have benefited from more in-depth assistance but did not ask for it.
- 2 October 2019
- Published by Taylor & Francis in Public Services Quarterly
- Vol. 15 (4) , 300-318
An Analysis of Help Seeking Behaviors in a Required First Year Course
The aim of this study is to show student perception of the value of library reference and consultation work in the building of information literacy skills. Using student answers to questions about library research help received in a required first year course, the authors analyzed motivations for use of librarian aid, and the types of assistance sought and that ideally could have been utilized. The authors found students primarily received librarian help with locating books, and that students often believed they would have benefited from more in-depth assistance but did not ask for it.
- Open Access
- 18 June 2018
- Published by American Library Association in Information Technology and Libraries
- Vol. 37 (2) , 10-33
Primo New User Interface: Usability Testing and Local Customizations Implemented in Response
Washington State University was the first library system of its 39-member consortium to migrate to Primo New User Interface. Following this migration, we conducted a usability study in July 2017 to better understand how our users fared when the new user interface deviated significantly from the classic interface. From this study, we learned that users had little difficulty using basic and advanced search, signing into and out of primo, and navigating their account. In other areas, where the difference between the two interfaces was more pronounced, study participants experienced more difficulty. Finally, we present customizations implemented at Washington State University to the design of the interface to help alleviate the observed issues.
- 18 June 2018
- Published by American Library Association in Information Technology and Libraries
- Vol. 37 (2) , 10-33
Open AccessPrimo New User Interface: Usability Testing and Local Customizations Implemented in Response
Washington State University was the first library system of its 39-member consortium to migrate to Primo New User Interface. Following this migration, we conducted a usability study in July 2017 to better understand how our users fared when the new user interface deviated significantly from the classic interface. From this study, we learned that users had little difficulty using basic and advanced search, signing into and out of primo, and navigating their account. In other areas, where the difference between the two interfaces was more pronounced, study participants experienced more difficulty. Finally, we present customizations implemented at Washington State University to the design of the interface to help alleviate the observed issues.
- Open Access
- 1 January 2017
- Published by Portland State University Library in Communications in Information Literacy
- Vol. 11 (2) , 339-353
Measuring Library Impacts through First Year Course Assessment
This study shows the value of library instruction in the building of first-year students’ information literacy skills and it illustrates librarians as partners in leading student learning outcome assessment. Using research papers from a required first-year course, raters from units across the institution evaluated student information literacy (IL) skill development. Students performed at a “Proficient First Year” level for most information literacy skill areas. The authors found there was a significant correlation between IL skill development and participation in one or more library instruction sessions. For this reason, the authors posit that liaison librarians are in a stronger and more stable collaborative position when they can demonstrate that their work has positive correlations with student learning.
- 1 January 2017
- Published by Portland State University Library in Communications in Information Literacy
- Vol. 11 (2) , 339-353
Open AccessMeasuring Library Impacts through First Year Course Assessment
This study shows the value of library instruction in the building of first-year students’ information literacy skills and it illustrates librarians as partners in leading student learning outcome assessment. Using research papers from a required first-year course, raters from units across the institution evaluated student information literacy (IL) skill development. Students performed at a “Proficient First Year” level for most information literacy skill areas. The authors found there was a significant correlation between IL skill development and participation in one or more library instruction sessions. For this reason, the authors posit that liaison librarians are in a stronger and more stable collaborative position when they can demonstrate that their work has positive correlations with student learning.