Anjana Susarla

The use of video-sharing platforms for disseminating healthcare information is rapidly increasing worldwide. However, professionally produced videos from healthcare organizations cover only a small fraction of known health conditions. In contrast, user-generated videos often address a broader range of health topics and offer personal insights into the daily management of complex diseases, although their educational value remains uncertain. With the growing popularity of self-care videos, there is a critical need to automate manual evaluation methods to ensure these videos meet established health education guidelines. In this study, we propose a hierarchical co-attention mechanism within a transformer-based approach to assess the actionability of health-related videos, using criteria from the Patient Education Materials Assessment Tool (PEMAT). Our model leverages co-attention to dynamically align textual and visual features, enabling a more nuanced understanding of multimodal content. Extensive experiments on 1,795 YouTube videos demonstrate the superior performance of our approach, achieving 82.2% AUC and outperforming state-of-the-art video classifiers. This research contributes to the expanding literature on transformer architectures for video classification and extends the application of PEMAT guidelines to a larger data set of health-related content. By automating the evaluation process, our model overcomes the limitations of manual assessments, offering a scalable and efficient solution for ensuring the quality and reliability of healthcare information on digital platforms. History: This paper has been accepted by Kaushik Dutta for the Special Issue on Responsible AI and Data Science for Social Good. Funding: This research was supported in part by the National Library of Medicine [Grant R01LM013443] of the National Institutes of Health. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2023.0493 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2023.0493 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

The software and data in this repository are a snapshot of the software and data that were used in the research reported on in the paper: CAAC: Co-attentive Actionability Classification for Assessing Patient Education Videos by Krishna Pothugunta, Xiao Liu, Anjana Susarla and Rema Padman.

Generative artificial intelligence (AI) is not merely changing how information systems (IS) research gets done—it is reshaping what research can be. We stand at a pivotal moment where machines can help generate hypotheses, synthesize vast literatures, and identify patterns that would take human researchers months to uncover. Yet, this unprecedented capability presents equally unprecedented risks to scholarly integrity. Because the field is uniquely positioned to understand sociotechnical transformations, IS research faces an extraordinary opportunity to pioneer “inventing with machines” while preserving the human insight and oversight that gives scholarship, as currently defined, its meaning. This transformation demands more than tool adoption. It requires a reimagination of scholarly infrastructure, norms, and practice. However, this transformation of research tooling creates a dangerous paradox: Powerful AI tools are now accessible to researchers who lack the technical literacy to understand and use them responsibly, threatening everything from citation accuracy to theoretical validity. Yet within this paradox lies the potential for revolutionary advances in how we craft our future as scholars. Informed by the sociotechnical perspective, we argue that the path forward requires coordinated community action that goes far beyond individual skill development. The IS community must lead the development of specialized AI tools that consider our theoretical traditions, create educational frameworks that preserve scholarly values while embracing computational capabilities, and pioneer review processes that harness AI’s analytical power without ceding human control, at least, in the short run. Success will determine not only the future of IS scholarship but our field’s capacity to guide other disciplines through this fundamental transformation of academic practice. The era of human-AI collaboration in research has already begun. How we govern and guide it will define the next generation of scholarly discovery.

Background: An estimated 93% of adults in the United States access the internet, with up to 80% looking for health information. However, only 12% of US adults are proficient enough in health literacy to interpret health information and make informed health care decisions meaningfully. With the vast amount of health information available in multimedia formats on social media platforms such as YouTube and Facebook, there is an urgent need and a unique opportunity to design an automated approach to curate online health information using multiple criteria to meet the health literacy needs of a diverse population. Objective: This study aimed to develop an automated approach to assessing the understandability of patient educational videos according to the Patient Education Materials Assessment Tool (PEMAT) guidelines and evaluating the impact of video understandability on viewer engagement. We also offer insights for content creators and health care organizations on how to improve engagement with these educational videos on user-generated content platforms. Methods: We developed a human-in-the-loop, augmented intelligence approach that explicitly focused on the human-algorithm interaction, combining PEMAT-based patient education constructs mapped to features extracted from the videos, annotations of the videos by domain experts, and cotraining methods from machine learning to assess the understandability of videos on diabetes and classify them. We further examined the impact of understandability on several dimensions of viewer engagement with the videos. Results: We collected 9873 YouTube videos on diabetes using search keywords extracted from a patient-oriented forum and reviewed by a medical expert. Our machine learning methods achieved a weighted precision of 0.84, a weighted recall of 0.79, and an F1-score of 0.81 in classifying video understandability and could effectively identify patient educational videos that medical experts would like to recommend for patients. Videos rated as highly understandable had an average higher view count (average treatment effect [ATE]=2.55; P<.001), like count (ATE=2.95; P<.001), and comment count (ATE=3.10; P<.001) than less understandable videos. In addition, in a user study, 4 medical experts recommended 72% (144/200) of the top 10 videos ranked by understandability compared to 40% (80/200) of the top 10 videos ranked by YouTube’s default algorithm for 20 ramdomly selected search keywords. Conclusions: We developed a human-in-the-loop, scalable algorithm to assess the understandability of health information on YouTube. Our method optimally combines expert input with algorithmic support, enhancing engagement and aiding medical experts in recommending educational content. This solution also guides health care organizations in creating effective patient education materials for underserved health topics.

BACKGROUND An estimated 93% of adults in the United States access the internet, with up to 80% looking for health information. However, only 12% of US adults are proficient enough in health literacy to interpret health information and make informed health care decisions meaningfully. With the vast amount of health information available in multimedia formats on social media platforms such as YouTube and Facebook, there is an urgent need and a unique opportunity to design an automated approach to curate online health information using multiple criteria to meet the health literacy needs of a diverse population. OBJECTIVE This study aimed to develop an automated approach to assessing the understandability of patient educational videos according to the Patient Education Materials Assessment Tool (PEMAT) guidelines and evaluating the impact of video understandability on viewer engagement. We also offer insights for content creators and health care organizations on how to improve engagement with these educational videos on user-generated content platforms. METHODS We developed a human-in-the-loop, augmented intelligence approach that explicitly focused on the human-algorithm interaction, combining PEMAT-based patient education constructs mapped to features extracted from the videos, annotations of the videos by domain experts, and cotraining methods from machine learning to assess the understandability of videos on diabetes and classify them. We further examined the impact of understandability on several dimensions of viewer engagement with the videos. RESULTS We collected 9873 YouTube videos on diabetes using search keywords extracted from a patient-oriented forum and reviewed by a medical expert. Our machine learning methods achieved a weighted precision of 0.84, a weighted recall of 0.79, and an F1-score of 0.81 in classifying video understandability and could effectively identify patient educational videos that medical experts would like to recommend for patients. Videos rated as highly understandable had an average higher view count (average treatment effect [ATE]=2.55; P<.001), like count (ATE=2.95; P<.001), and comment count (ATE=3.10; P<.001) than less understandable videos. In addition, in a user study, 4 medical experts recommended 72% (144/200) of the top 10 videos ranked by understandability compared to 40% (80/200) of the top 10 videos ranked by YouTube’s default algorithm for 20 ramdomly selected search keywords. CONCLUSIONS We developed a human-in-the-loop, scalable algorithm to assess the understandability of health information on YouTube. Our method optimally combines expert input with algorithmic support, enhancing engagement and aiding medical experts in recommending educational content. This solution also guides health care organizations in creating effective patient education materials for underserved health topics.

Funding: A. Susarla was funded by an R01 grant from the National Library of Medicine, through [Grant R01LM013443].

Studies suggest that one in three US adults use the Internet to diagnose or learn about a health concern. However, such access to health information online could exacerbate the disparities in health information availability and use. Health information seeking behavior (HISB) refers to the ways in which individuals seek information about their health, risks, illnesses, and health-protective behaviors. For patients engaging in searches for health information on digital media platforms, health literacy divides can be exacerbated both by their own lack of knowledge and by algorithmic recommendations, with results that disproportionately impact disadvantaged populations, minorities, and low health literacy users. This study reports on an exploratory investigation of the above challenges by examining whether responsible and representative recommendations can be generated using advanced analytic methods applied to a large corpus of videos and their metadata on a chronic condition (diabetes) from the YouTube social media platform. The paper focusses on biases associated with demographic characters of actors using videos on diabetes that were retrieved and curated for multiple criteria such as encoded medical content and their understandability to address patient education and population health literacy needs. This approach offers an immense opportunity for innovation in human-in-the-loop, augmented-intelligence, bias-aware and responsible algorithmic recommendations by combining the perspectives of health professionals and patients into a scalable and generalizable machine learning framework for patient empowerment and improved health outcomes.

This study examines the dynamic relationships between firm-generated content (FGC), user-generated content (UGC), traditional media, and offline light vehicle sales. Data were collected from the official Facebook and Twitter pages of 30 U.S. car brands from 2009 to 2015. Our results suggest that Facebook and Twitter are heterogeneous in terms of their effect on offline vehicle sales; FGC is more effective than UGC for influencing offline light vehicle sales; viral impressions from Facebook and Twitter are essential, although effects vary for the various social media platforms, FGC, and UGC; and a firm’s marketing efforts and UGC both have a long-term effect on sales, with the long-term effect of a firm’s marketing efforts outlasting that of UGC. We also documented the within-Twitter synergistic effect between FGC and UGC for offline car sales and cross-channel substitution relationships between FGC and both Facebook and traditional media and Twitter and traditional media. Our study implies that managers who attempt to maximize multichannel marketing for offline sales of durable goods should consider (1) the nature of each platform, (2) the number of potential audiences each platform can reach, and (3) the user basis of each platform.