Glenn Fernandes
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Glenn Fernandes has a total of 28 co-authors. Here are the top co-authors by number of shared publications:
- Nabil Alshurafa
Preventive Medicine and Computer Science, Northwestern University, Chicago, Illinois, USA
8 shared publications - Christopher Romano
Northwestern University, Chicago, Illinois, USA
3 shared publications - Jacob Michael Schauer
Division of Gastroenterology and Hepatology, Department of Medicine, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
3 shared publications - Mahdi Pedram
Northwestern University, United States
3 shared publications - Adnan Darwiche
Computer Science Department University of California Los Angeles USA
2 shared publications - Aggelos K. Katsaggelos
Northwestern University
2 shared publications
Latest Publications
- Open Access
- 27 April 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Computing for Healthcare
“I Don’t Like Being Told Just What to Do; I Need to Know Why”: Patient Expectations of Machine Learning-Driven Just-in-Time-Adaptive Interventions (JITAIs) for Prenatal Stress Management
Just-in-time-adaptive interventions (JITAIs) have advanced the ways in which technology supports behavior change, providing tailored interventions at the most appropriate time. With machine learning (ML), these interventions are becoming increasingly sophisticated. However, significant efforts are required to fully understand and address individual expectations of ML-driven JITAIs. In this work, we recruited 20 pregnant people to explore their interest in and expectations of a JITAI designed for prenatal stress prediction and prevention, an area of global public health significance. Using design sessions that combined reflection and design feedback activities, we found that participants were interested in using ML-driven JITAI designs that (1) fit into their daily life by offering flexible engagement, (2) supported them in building a mental model of the underlying JITAI functionality, (3) differentiated between non-adherence and non-compliance, and (4) illustrated apparent reciprocal learning. We use these results to propose designs for future iterations of prenatal stress interventions, particularly those leveraging increasingly complex computational technologies.
- 27 April 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Computing for Healthcare
Open Access“I Don’t Like Being Told Just What to Do; I Need to Know Why”: Patient Expectations of Machine Learning-Driven Just-in-Time-Adaptive Interventions (JITAIs) for Prenatal Stress Management
Just-in-time-adaptive interventions (JITAIs) have advanced the ways in which technology supports behavior change, providing tailored interventions at the most appropriate time. With machine learning (ML), these interventions are becoming increasingly sophisticated. However, significant efforts are required to fully understand and address individual expectations of ML-driven JITAIs. In this work, we recruited 20 pregnant people to explore their interest in and expectations of a JITAI designed for prenatal stress prediction and prevention, an area of global public health significance. Using design sessions that combined reflection and design feedback activities, we found that participants were interested in using ML-driven JITAI designs that (1) fit into their daily life by offering flexible engagement, (2) supported them in building a mental model of the underlying JITAI functionality, (3) differentiated between non-adherence and non-compliance, and (4) illustrated apparent reciprocal learning. We use these results to propose designs for future iterations of prenatal stress interventions, particularly those leveraging increasingly complex computational technologies.
- Open Access
- 2 December 2025
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Vol. 9 (4) , 1-25
THOR: Thermal-Guided Hand-Object Reasoning via Adaptive Vision Sampling
Wearable cameras are increasingly used as an observational and interventional tool for human behaviors by providing detailed visual data of hand-related activities. This data can be leveraged to facilitate memory recall for logging of behavior or timely interventions aimed at improving health. However, continuous processing of RGB images from these cameras consumes significant power impacting battery lifetime, generates a large volume of unnecessary video data for post-processing, raises privacy concerns, and requires substantial computational resources for real-time analysis. We introduce THOR, a real-time adaptive spatio-temporal RGB frame sampling method that leverages thermal sensing to capture hand-object patches and classify them in real time. We use low-resolution thermal camera data to identify moments when a person switches from one hand-related activity to another and adjust the RGB frame sampling rate by increasing it during activity transitions and reducing it during periods of sustained activity (when the system has enough information to identify the activity). Additionally, we use the thermal cues from the hand to localize the region of interest (i.e. , the hand-object interaction) in each RGB frame, allowing the system to crop and process only the necessary part of the image for activity recognition. We develop a wearable device to validate our method through an in-the-wild study with 14 participants and over 30 activities, and further evaluate it on Ego4D (923 participants across 9 countries, totaling 3,670 hours of video). Our results show that using only 3% of the original RGB video data, our method captures all the activity segments, and achieves a hand-related activity recognition F1-score (95%) comparable to using the entire RGB video (94%). Our work provides a more practical path for the longitudinal use of wearable cameras to monitor hand-related activities and health-risk behaviors in real time.
- 2 December 2025
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Vol. 9 (4) , 1-25
Open AccessTHOR: Thermal-Guided Hand-Object Reasoning via Adaptive Vision Sampling
Wearable cameras are increasingly used as an observational and interventional tool for human behaviors by providing detailed visual data of hand-related activities. This data can be leveraged to facilitate memory recall for logging of behavior or timely interventions aimed at improving health. However, continuous processing of RGB images from these cameras consumes significant power impacting battery lifetime, generates a large volume of unnecessary video data for post-processing, raises privacy concerns, and requires substantial computational resources for real-time analysis. We introduce THOR, a real-time adaptive spatio-temporal RGB frame sampling method that leverages thermal sensing to capture hand-object patches and classify them in real time. We use low-resolution thermal camera data to identify moments when a person switches from one hand-related activity to another and adjust the RGB frame sampling rate by increasing it during activity transitions and reducing it during periods of sustained activity (when the system has enough information to identify the activity). Additionally, we use the thermal cues from the hand to localize the region of interest (i.e. , the hand-object interaction) in each RGB frame, allowing the system to crop and process only the necessary part of the image for activity recognition. We develop a wearable device to validate our method through an in-the-wild study with 14 participants and over 30 activities, and further evaluate it on Ego4D (923 participants across 9 countries, totaling 3,670 hours of video). Our results show that using only 3% of the original RGB video data, our method captures all the activity segments, and achieves a hand-related activity recognition F1-score (95%) comparable to using the entire RGB video (94%). Our work provides a more practical path for the longitudinal use of wearable cameras to monitor hand-related activities and health-risk behaviors in real time.
- 11 November 2025
- Published by Springer Nature
- p. 225-232
GULP-KG: Graphical Understanding of Lifestyle Patterns with Knowledge Graphs
- 11 November 2025
- Published by Springer Nature
- p. 225-232
GULP-KG: Graphical Understanding of Lifestyle Patterns with Knowledge Graphs
- Open Access
- 22 August 2024
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Vol. 8 (3) , 1-48
HabitSense: A Privacy-Aware, AI-Enhanced Multimodal Wearable Platform for mHealth Applications
Wearable cameras provide an objective method to visually confirm and automate the detection of health-risk behaviors such as smoking and overeating, which is critical for developing and testing adaptive treatment interventions. Despite the potential of wearable camera systems, adoption is hindered by inadequate clinician input in the design, user privacy concerns, and user burden. To address these barriers, we introduced HabitSense, an open-source1, multi-modal neck-worn platform developed with input from focus groups with clinicians (N=36) and user feedback from in-wild studies involving 105 participants over 35 days. Optimized for monitoring health-risk behaviors, the platform utilizes RGB, thermal, and inertial measurement unit sensors to detect eating and smoking events in real time. In a 7-day study involving 15 participants, HabitSense recorded 768 hours of footage, capturing 420.91 minutes of hand-to-mouth gestures associated with eating and smoking data crucial for training machine learning models, achieving a 92% F1-score in gesture recognition. To address privacy concerns, the platform records only during likely health-risk behavior events using SECURE, a smart activation algorithm. Additionally, HabitSense employs on-device obfuscation algorithms that selectively obfuscate the background during recording, maintaining individual privacy while leaving gestures related to health-risk behaviors unobfuscated. Our implementation of SECURE has resulted in a 48% reduction in storage needs and a 30% increase in battery life. This paper highlights the critical roles of clinician feedback, extensive field testing, and privacy-enhancing algorithms in developing an unobtrusive, lightweight, and reproducible wearable system that is both feasible and acceptable for monitoring health-risk behaviors in real-world settings.
- 22 August 2024
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Vol. 8 (3) , 1-48
Open AccessHabitSense: A Privacy-Aware, AI-Enhanced Multimodal Wearable Platform for mHealth Applications
Wearable cameras provide an objective method to visually confirm and automate the detection of health-risk behaviors such as smoking and overeating, which is critical for developing and testing adaptive treatment interventions. Despite the potential of wearable camera systems, adoption is hindered by inadequate clinician input in the design, user privacy concerns, and user burden. To address these barriers, we introduced HabitSense, an open-source1, multi-modal neck-worn platform developed with input from focus groups with clinicians (N=36) and user feedback from in-wild studies involving 105 participants over 35 days. Optimized for monitoring health-risk behaviors, the platform utilizes RGB, thermal, and inertial measurement unit sensors to detect eating and smoking events in real time. In a 7-day study involving 15 participants, HabitSense recorded 768 hours of footage, capturing 420.91 minutes of hand-to-mouth gestures associated with eating and smoking data crucial for training machine learning models, achieving a 92% F1-score in gesture recognition. To address privacy concerns, the platform records only during likely health-risk behavior events using SECURE, a smart activation algorithm. Additionally, HabitSense employs on-device obfuscation algorithms that selectively obfuscate the background during recording, maintaining individual privacy while leaving gestures related to health-risk behaviors unobfuscated. Our implementation of SECURE has resulted in a 48% reduction in storage needs and a 30% increase in battery life. This paper highlights the critical roles of clinician feedback, extensive field testing, and privacy-enhancing algorithms in developing an unobtrusive, lightweight, and reproducible wearable system that is both feasible and acceptable for monitoring health-risk behaviors in real-world settings.
- Open Access
- 6 September 2023
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 25, e42047
An Explainable Artificial Intelligence Software Tool for Weight Management Experts (PRIMO): Mixed Methods Study
Background: Predicting the likelihood of success of weight loss interventions using machine learning (ML) models may enhance intervention effectiveness by enabling timely and dynamic modification of intervention components for nonresponders to treatment. However, a lack of understanding and trust in these ML models impacts adoption among weight management experts. Recent advances in the field of explainable artificial intelligence enable the interpretation of ML models, yet it is unknown whether they enhance model understanding, trust, and adoption among weight management experts. Objective: This study aimed to build and evaluate an ML model that can predict 6-month weight loss success (ie, ≥7% weight loss) from 5 engagement and diet-related features collected over the initial 2 weeks of an intervention, to assess whether providing ML-based explanations increases weight management experts’ agreement with ML model predictions, and to inform factors that influence the understanding and trust of ML models to advance explainability in early prediction of weight loss among weight management experts. Methods: We trained an ML model using the random forest (RF) algorithm and data from a 6-month weight loss intervention (N=419). We leveraged findings from existing explainability metrics to develop Prime Implicant Maintenance of Outcome (PRIMO), an interactive tool to understand predictions made by the RF model. We asked 14 weight management experts to predict hypothetical participants’ weight loss success before and after using PRIMO. We compared PRIMO with 2 other explainability methods, one based on feature ranking and the other based on conditional probability. We used generalized linear mixed-effects models to evaluate participants’ agreement with ML predictions and conducted likelihood ratio tests to examine the relationship between explainability methods and outcomes for nested models. We conducted guided interviews and thematic analysis to study the impact of our tool on experts’ understanding and trust in the model. Results: Our RF model had 81% accuracy in the early prediction of weight loss success. Weight management experts were significantly more likely to agree with the model when using PRIMO (χ2=7.9; P=.02) compared with the other 2 methods with odds ratios of 2.52 (95% CI 0.91-7.69) and 3.95 (95% CI 1.50-11.76). From our study, we inferred that our software not only influenced experts’ understanding and trust but also impacted decision-making. Several themes were identified through interviews: preference for multiple explanation types, need to visualize uncertainty in explanations provided by PRIMO, and need for model performance metrics on similar participant test instances. Conclusions: Our results show the potential for weight management experts to agree with the ML-based early prediction of success in weight loss treatment programs, enabling timely and dynamic modification of intervention components to enhance intervention effectiveness. Our findings provide methods for advancing the understandability and trust of ML models among weight management experts.
- 6 September 2023
- Published by JMIR Publications Inc. in Journal of Medical Internet Research
- Vol. 25, e42047
Open AccessAn Explainable Artificial Intelligence Software Tool for Weight Management Experts (PRIMO): Mixed Methods Study
Background: Predicting the likelihood of success of weight loss interventions using machine learning (ML) models may enhance intervention effectiveness by enabling timely and dynamic modification of intervention components for nonresponders to treatment. However, a lack of understanding and trust in these ML models impacts adoption among weight management experts. Recent advances in the field of explainable artificial intelligence enable the interpretation of ML models, yet it is unknown whether they enhance model understanding, trust, and adoption among weight management experts. Objective: This study aimed to build and evaluate an ML model that can predict 6-month weight loss success (ie, ≥7% weight loss) from 5 engagement and diet-related features collected over the initial 2 weeks of an intervention, to assess whether providing ML-based explanations increases weight management experts’ agreement with ML model predictions, and to inform factors that influence the understanding and trust of ML models to advance explainability in early prediction of weight loss among weight management experts. Methods: We trained an ML model using the random forest (RF) algorithm and data from a 6-month weight loss intervention (N=419). We leveraged findings from existing explainability metrics to develop Prime Implicant Maintenance of Outcome (PRIMO), an interactive tool to understand predictions made by the RF model. We asked 14 weight management experts to predict hypothetical participants’ weight loss success before and after using PRIMO. We compared PRIMO with 2 other explainability methods, one based on feature ranking and the other based on conditional probability. We used generalized linear mixed-effects models to evaluate participants’ agreement with ML predictions and conducted likelihood ratio tests to examine the relationship between explainability methods and outcomes for nested models. We conducted guided interviews and thematic analysis to study the impact of our tool on experts’ understanding and trust in the model. Results: Our RF model had 81% accuracy in the early prediction of weight loss success. Weight management experts were significantly more likely to agree with the model when using PRIMO (χ2=7.9; P=.02) compared with the other 2 methods with odds ratios of 2.52 (95% CI 0.91-7.69) and 3.95 (95% CI 1.50-11.76). From our study, we inferred that our software not only influenced experts’ understanding and trust but also impacted decision-making. Several themes were identified through interviews: preference for multiple explanation types, need to visualize uncertainty in explanations provided by PRIMO, and need for model performance metrics on similar participant test instances. Conclusions: Our results show the potential for weight management experts to agree with the ML-based early prediction of success in weight loss treatment programs, enabling timely and dynamic modification of intervention components to enhance intervention effectiveness. Our findings provide methods for advancing the understandability and trust of ML models among weight management experts.
- Open Access
- 19 April 2023
- Published by Association for Computing Machinery (ACM)
- Vol. 2023, 1-8
Is cartoonized life-vlogging the key to increasing adoption of activity-oriented wearable camera systems?
Health science researchers studying human behavior rely on wearable cameras to visually confirm behaviors in real-world settings. However, privacy concerns significantly impede their adoption. Lens orientation and activity-oriented cameras have potential in balancing the need to visually validate the wearers’ activities while reducing privacy concerns. To increase adoption and further alleviate privacy concerns while maintaining utility, generative stylizing approaches, like cartooning using generative adversarial networks (GANs), have recently shown promise. We investigate different cartoon-based obfuscation of activity-oriented footage through two studies. The first deploys crowdsourcing methods (n=60), while the second is experiential, where participants (n=49) don the device for an entire day and report concerns on their footage. Our findings support that cartoonization of activity-oriented data significantly reduces privacy concerns, particularly among bystanders in high privacy-concerning scenarios, while maintaining context verification (90% of participants). Through thematic analysis, we provide further insight for the community on best practices for cartoonization of activity-oriented videos.
- 19 April 2023
- Published by Association for Computing Machinery (ACM)
- Vol. 2023, 1-8
Open AccessIs cartoonized life-vlogging the key to increasing adoption of activity-oriented wearable camera systems?
Health science researchers studying human behavior rely on wearable cameras to visually confirm behaviors in real-world settings. However, privacy concerns significantly impede their adoption. Lens orientation and activity-oriented cameras have potential in balancing the need to visually validate the wearers’ activities while reducing privacy concerns. To increase adoption and further alleviate privacy concerns while maintaining utility, generative stylizing approaches, like cartooning using generative adversarial networks (GANs), have recently shown promise. We investigate different cartoon-based obfuscation of activity-oriented footage through two studies. The first deploys crowdsourcing methods (n=60), while the second is experiential, where participants (n=49) don the device for an entire day and report concerns on their footage. Our findings support that cartoonization of activity-oriented data significantly reduces privacy concerns, particularly among bystanders in high privacy-concerning scenarios, while maintaining context verification (90% of participants). Through thematic analysis, we provide further insight for the community on best practices for cartoonization of activity-oriented videos.
- Open Access
- 19 April 2023
- Published by Association for Computing Machinery (ACM)
- Vol. 2023, 1-8
Experience: Barriers and Opportunities of Wearables for Eating Research
Wearable devices have long held the potential to provide real-time objective measures of behavior. However, due to challenges in real-world deployment, these systems are rarely tested rigorously in free-living settings. To reduce this challenge for future researchers, in this paper, we describe our experience developing several generations of a multi-sensor, neck-worn eating-detection system that has been tested with 130 participants across multiple studies in both laboratory and free-living settings. We describe the challenges faced in the development and deployment of the system by (1) presenting example deployment details captured either by the sensing system or the ground truth collector and (2) using structured interviews and surveys with developers and stakeholders of the system, collecting qualitative data on their experience. We performed thematic analysis and provided detailed lessons learned explaining factors that impact the experience of building and deploying such a wearable in a free-living setting, reducing challenges for future researchers. We believe that our experience will help future researchers develop successful mobile health (mHealth) systems that translate into reliable free-living deployments.
- 19 April 2023
- Published by Association for Computing Machinery (ACM)
- Vol. 2023, 1-8
Open AccessExperience: Barriers and Opportunities of Wearables for Eating Research
Wearable devices have long held the potential to provide real-time objective measures of behavior. However, due to challenges in real-world deployment, these systems are rarely tested rigorously in free-living settings. To reduce this challenge for future researchers, in this paper, we describe our experience developing several generations of a multi-sensor, neck-worn eating-detection system that has been tested with 130 participants across multiple studies in both laboratory and free-living settings. We describe the challenges faced in the development and deployment of the system by (1) presenting example deployment details captured either by the sensing system or the ground truth collector and (2) using structured interviews and surveys with developers and stakeholders of the system, collecting qualitative data on their experience. We performed thematic analysis and provided detailed lessons learned explaining factors that impact the experience of building and deploying such a wearable in a free-living setting, reducing challenges for future researchers. We believe that our experience will help future researchers develop successful mobile health (mHealth) systems that translate into reliable free-living deployments.
- Open Access
- 19 August 2022
- Published by JMIR Publications Inc. in JMIR Preprints
An Explainable Artificial Intelligence Software Tool for Weight Management Experts (PRIMO): Mixed Methods Study (Preprint)
BACKGROUND Predicting the likelihood of success of weight loss interventions using machine learning (ML) models may enhance intervention effectiveness by enabling timely and dynamic modification of intervention components for nonresponders to treatment. However, a lack of understanding and trust in these ML models impacts adoption among weight management experts. Recent advances in the field of explainable artificial intelligence enable the interpretation of ML models, yet it is unknown whether they enhance model understanding, trust, and adoption among weight management experts. OBJECTIVE This study aimed to build and evaluate an ML model that can predict 6-month weight loss success (ie, ≥7% weight loss) from 5 engagement and diet-related features collected over the initial 2 weeks of an intervention, to assess whether providing ML-based explanations increases weight management experts’ agreement with ML model predictions, and to inform factors that influence the understanding and trust of ML models to advance explainability in early prediction of weight loss among weight management experts. METHODS We trained an ML model using the random forest (RF) algorithm and data from a 6-month weight loss intervention (N=419). We leveraged findings from existing explainability metrics to develop Prime Implicant Maintenance of Outcome (PRIMO), an interactive tool to understand predictions made by the RF model. We asked 14 weight management experts to predict hypothetical participants’ weight loss success before and after using PRIMO. We compared PRIMO with 2 other explainability methods, one based on feature ranking and the other based on conditional probability. We used generalized linear mixed-effects models to evaluate participants’ agreement with ML predictions and conducted likelihood ratio tests to examine the relationship between explainability methods and outcomes for nested models. We conducted guided interviews and thematic analysis to study the impact of our tool on experts’ understanding and trust in the model. RESULTS Our RF model had 81% accuracy in the early prediction of weight loss success. Weight management experts were significantly more likely to agree with the model when using PRIMO (χ2=7.9; P=.02) compared with the other 2 methods with odds ratios of 2.52 (95% CI 0.91-7.69) and 3.95 (95% CI 1.50-11.76). From our study, we inferred that our software not only influenced experts’ understanding and trust but also impacted decision-making. Several themes were identified through interviews: preference for multiple explanation types, need to visualize uncertainty in explanations provided by PRIMO, and need for model performance metrics on similar participant test instances. CONCLUSIONS Our results show the potential for weight management experts to agree with the ML-based early prediction of success in weight loss treatment programs, enabling timely and dynamic modification of intervention components to enhance intervention effectiveness. Our findings provide methods for advancing the understandability and trust of ML models among weight management experts.
- 19 August 2022
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessAn Explainable Artificial Intelligence Software Tool for Weight Management Experts (PRIMO): Mixed Methods Study (Preprint)
BACKGROUND Predicting the likelihood of success of weight loss interventions using machine learning (ML) models may enhance intervention effectiveness by enabling timely and dynamic modification of intervention components for nonresponders to treatment. However, a lack of understanding and trust in these ML models impacts adoption among weight management experts. Recent advances in the field of explainable artificial intelligence enable the interpretation of ML models, yet it is unknown whether they enhance model understanding, trust, and adoption among weight management experts. OBJECTIVE This study aimed to build and evaluate an ML model that can predict 6-month weight loss success (ie, ≥7% weight loss) from 5 engagement and diet-related features collected over the initial 2 weeks of an intervention, to assess whether providing ML-based explanations increases weight management experts’ agreement with ML model predictions, and to inform factors that influence the understanding and trust of ML models to advance explainability in early prediction of weight loss among weight management experts. METHODS We trained an ML model using the random forest (RF) algorithm and data from a 6-month weight loss intervention (N=419). We leveraged findings from existing explainability metrics to develop Prime Implicant Maintenance of Outcome (PRIMO), an interactive tool to understand predictions made by the RF model. We asked 14 weight management experts to predict hypothetical participants’ weight loss success before and after using PRIMO. We compared PRIMO with 2 other explainability methods, one based on feature ranking and the other based on conditional probability. We used generalized linear mixed-effects models to evaluate participants’ agreement with ML predictions and conducted likelihood ratio tests to examine the relationship between explainability methods and outcomes for nested models. We conducted guided interviews and thematic analysis to study the impact of our tool on experts’ understanding and trust in the model. RESULTS Our RF model had 81% accuracy in the early prediction of weight loss success. Weight management experts were significantly more likely to agree with the model when using PRIMO (χ2=7.9; P=.02) compared with the other 2 methods with odds ratios of 2.52 (95% CI 0.91-7.69) and 3.95 (95% CI 1.50-11.76). From our study, we inferred that our software not only influenced experts’ understanding and trust but also impacted decision-making. Several themes were identified through interviews: preference for multiple explanation types, need to visualize uncertainty in explanations provided by PRIMO, and need for model performance metrics on similar participant test instances. CONCLUSIONS Our results show the potential for weight management experts to agree with the ML-based early prediction of success in weight loss treatment programs, enabling timely and dynamic modification of intervention components to enhance intervention effectiveness. Our findings provide methods for advancing the understandability and trust of ML models among weight management experts.