Hong Jia
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Hong Jia has a total of 42 co-authors. Here are the top co-authors by number of shared publications:
- Ting Dang
The University of Melbourne, Australia
7 shared publications - Vassilis Kostakos
School of Computing and Information Systems, University of Melbourne, Melbourne, Australia
7 shared publications - Simon D'ALfonso
School of Computing and Information Systems, University of Melbourne, Melbourne, Australia
6 shared publications - Tianyi Zhang
The University of Melbourne, Melbourne, Australia
5 shared publications - Wen Hu
University of New South Wales
4 shared publications - Branislav Kusy
DATA61, CSIRO
3 shared publications
Latest Publications
- Open Access
- 20 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 69-74
Multi-Task Mental Health Detection with Large Language Models under Class Imbalance
- 20 June 2026
- Published by Association for Computing Machinery (ACM)
- p. 69-74
Open AccessMulti-Task Mental Health Detection with Large Language Models under Class Imbalance
- Open Access
- 15 June 2026
- Published by Elsevier in International Journal of Human-Computer Studies
- Vol. 215
From prediction to explanation: Using screen text to understand smartphone use and user behaviour
- 15 June 2026
- Published by Elsevier in International Journal of Human-Computer Studies
- Vol. 215
Open AccessFrom prediction to explanation: Using screen text to understand smartphone use and user behaviour
- Open Access
- 17 March 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Computing for Healthcare
- Vol. 7 (2) , 1-30
NarrativeSense: Predicting Affective States in University Students through Smartphone Sensing and Contextual Narratives
Mental health challenges are increasingly prevalent among university students, yet often go undetected due to reliance on traditional assessments that are subjective, infrequent, and lack behavioral context. Digital phenotyping through passively collected smartphone data offers a scalable alternative, but existing approaches often fail to integrate predictive accuracy with narrative-based insights. To overcome these limitations, we present NarrativeSense, a novel framework that combines machine learning models with narrative-based descriptions of daily life events inferred from smartphone sensing data to predict weekly affective states. The system incorporates language model components to transform behavioral patterns into contextualized, human-readable narratives that ground affective predictions in everyday experiences. This narrative layer complements structured prediction by offering intuitive, user-centered insights. Applied to longitudinal data from 58 university students over 119 days, NarrativeSense outperforms baseline machine learning models, standalone LLMs, and ensemble methods, while providing richer insights. Our findings demonstrate the potential of narrative-enhanced digital phenotyping for scalable and explainable mental health monitoring in educational and clinical settings.
- 17 March 2026
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Computing for Healthcare
- Vol. 7 (2) , 1-30
Open AccessNarrativeSense: Predicting Affective States in University Students through Smartphone Sensing and Contextual Narratives
Mental health challenges are increasingly prevalent among university students, yet often go undetected due to reliance on traditional assessments that are subjective, infrequent, and lack behavioral context. Digital phenotyping through passively collected smartphone data offers a scalable alternative, but existing approaches often fail to integrate predictive accuracy with narrative-based insights. To overcome these limitations, we present NarrativeSense, a novel framework that combines machine learning models with narrative-based descriptions of daily life events inferred from smartphone sensing data to predict weekly affective states. The system incorporates language model components to transform behavioral patterns into contextualized, human-readable narratives that ground affective predictions in everyday experiences. This narrative layer complements structured prediction by offering intuitive, user-centered insights. Applied to longitudinal data from 58 university students over 119 days, NarrativeSense outperforms baseline machine learning models, standalone LLMs, and ensemble methods, while providing richer insights. Our findings demonstrate the potential of narrative-enhanced digital phenotyping for scalable and explainable mental health monitoring in educational and clinical settings.
- Open Access
- 13 March 2026
- Published by Elsevier in Pervasive and Mobile Computing
- Vol. 119
A cascade framework for on-device uncertainty-aware event detection on microcontrollers
- 13 March 2026
- Published by Elsevier in Pervasive and Mobile Computing
- Vol. 119
Open AccessA cascade framework for on-device uncertainty-aware event detection on microcontrollers
- Open Access
- 12 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 1569-1575
Behavioral Indicators of Loneliness: Predicting University Students' Loneliness Scores from Smartphone Sensing Data
- 12 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 1569-1575
Open AccessBehavioral Indicators of Loneliness: Predicting University Students' Loneliness Scores from Smartphone Sensing Data
- Open Access
- 12 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 1565-1568
WellComp 2025: Eighth International Workshop on Computing and Software Systems for Well-Being
- 12 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 1565-1568
Open AccessWellComp 2025: Eighth International Workshop on Computing and Software Systems for Well-Being
- Open Access
- 12 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 1597-1604
From Patient Burdens to User Agency: Designing for Real-Time Protection Support in Online Health Consultations
- 12 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 1597-1604
Open AccessFrom Patient Burdens to User Agency: Designing for Real-Time Protection Support in Online Health Consultations
- Open Access
- 6 May 2025
- Published by Association for Computing Machinery (ACM)
- p. 158-171
LightLLM: A Versatile Large Language Model for Predictive Light Sensing
- 6 May 2025
- Published by Association for Computing Machinery (ACM)
- p. 158-171
Open AccessLightLLM: A Versatile Large Language Model for Predictive Light Sensing
- 31 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Pervasive Computing
- Vol. 23 (4) , 87-93
Multimodal Large Language Models in Human-Centered Health: Practical Insights
Multimodal data offer a comprehensive view of human health by integrating diverse sources, such as text, medical images, physiological signals, and behavioral data. Recent advancements in large language models (LLMs) have led to the development of multimodal large language models (MLLMs), which leverage the text understanding capabilities of LLMs and integrate them with other modalities. While MLLMs show great promise for human-centered health applications, the practical challenges of implementing them in the healthcare sector remain largely unexplored. This article discusses these practical considerations and the future potential of MLLMs in transforming human-centered healthcare.
- 31 January 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Pervasive Computing
- Vol. 23 (4) , 87-93
Multimodal Large Language Models in Human-Centered Health: Practical Insights
Multimodal data offer a comprehensive view of human health by integrating diverse sources, such as text, medical images, physiological signals, and behavioral data. Recent advancements in large language models (LLMs) have led to the development of multimodal large language models (MLLMs), which leverage the text understanding capabilities of LLMs and integrate them with other modalities. While MLLMs show great promise for human-centered health applications, the practical challenges of implementing them in the healthcare sector remain largely unexplored. This article discusses these practical considerations and the future potential of MLLMs in transforming human-centered healthcare.
- Open Access
- 4 December 2024
- Published by Association for Computing Machinery (ACM)
- p. 2347-2352
AutoJournaling: A Context-Aware Journaling System Leveraging MLLMs on Smartphone Screenshots
Journaling offers significant benefits, including fostering self-reflection, enhancing writing skills, and aiding in mood monitoring. However, many people abandon the practice because traditional journaling is time-consuming, and detailed life events may be overlooked if not recorded promptly. Given that smartphones are the most widely used devices for entertainment, work, and socialization, they present an ideal platform for innovative approaches to journaling. Despite their ubiquity, the potential of using digital phenotyping, a method of unobtrusively collecting data from digital devices to gain insights into psychological and behavioral patterns, for automated journal generation has been largely underexplored. In this study, we propose AutoJournaling, the first-of-its-kind system that automatically generates journals by collecting and analyzing screenshots from smartphones. This system captures life events and corresponding emotions, offering a novel approach to digital phenotyping. We evaluated AutoJournaling by collecting screenshots every 3 seconds from three students over five days, demonstrating its feasibility and accuracy. AutoJournaling is the first framework to utilize seamlessly collected screenshots for journal generation, providing new insights into psychological states through digital phenotyping.
- 4 December 2024
- Published by Association for Computing Machinery (ACM)
- p. 2347-2352
Open AccessAutoJournaling: A Context-Aware Journaling System Leveraging MLLMs on Smartphone Screenshots
Journaling offers significant benefits, including fostering self-reflection, enhancing writing skills, and aiding in mood monitoring. However, many people abandon the practice because traditional journaling is time-consuming, and detailed life events may be overlooked if not recorded promptly. Given that smartphones are the most widely used devices for entertainment, work, and socialization, they present an ideal platform for innovative approaches to journaling. Despite their ubiquity, the potential of using digital phenotyping, a method of unobtrusively collecting data from digital devices to gain insights into psychological and behavioral patterns, for automated journal generation has been largely underexplored. In this study, we propose AutoJournaling, the first-of-its-kind system that automatically generates journals by collecting and analyzing screenshots from smartphones. This system captures life events and corresponding emotions, offering a novel approach to digital phenotyping. We evaluated AutoJournaling by collecting screenshots every 3 seconds from three students over five days, demonstrating its feasibility and accuracy. AutoJournaling is the first framework to utilize seamlessly collected screenshots for journal generation, providing new insights into psychological states through digital phenotyping.