Shiquan Zhang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Shiquan Zhang has a total of 34 co-authors. Here are the top co-authors by number of shared publications:
- Hong Jia
University of Auckland, Auckland, New Zealand
3 shared publications - Le Fang
School of Computing and Information Systems, University of Melbourne, Melbourne, Australia
3 shared publications - Vassilis Kostakos
School of Computing and Information Systems, University of Melbourne, Melbourne, Australia
3 shared publications - Jorge Goncalves
University of Melbourne, Melbourne, Australia
2 shared publications - Lijuan Du
Department of Ultrasound, Beijing Tiantan Hospital, Capital Medical University, Beijing, China
2 shared publications - Simon D'ALfonso
School of Computing and Information Systems, University of Melbourne, Melbourne, Australia
2 shared publications
Latest Publications
- Open Access
- 25 April 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-14
Raising Awareness of Location Information Vulnerabilities in Social Media Photos using LLMs
- 25 April 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-14
Open AccessRaising Awareness of Location Information Vulnerabilities in Social Media Photos using LLMs
- Open Access
- 4 December 2024
- Published by Association for Computing Machinery (ACM)
- p. 2347-2352
AutoJournaling: A Context-Aware Journaling System Leveraging MLLMs on Smartphone Screenshots
Journaling offers significant benefits, including fostering self-reflection, enhancing writing skills, and aiding in mood monitoring. However, many people abandon the practice because traditional journaling is time-consuming, and detailed life events may be overlooked if not recorded promptly. Given that smartphones are the most widely used devices for entertainment, work, and socialization, they present an ideal platform for innovative approaches to journaling. Despite their ubiquity, the potential of using digital phenotyping, a method of unobtrusively collecting data from digital devices to gain insights into psychological and behavioral patterns, for automated journal generation has been largely underexplored. In this study, we propose AutoJournaling, the first-of-its-kind system that automatically generates journals by collecting and analyzing screenshots from smartphones. This system captures life events and corresponding emotions, offering a novel approach to digital phenotyping. We evaluated AutoJournaling by collecting screenshots every 3 seconds from three students over five days, demonstrating its feasibility and accuracy. AutoJournaling is the first framework to utilize seamlessly collected screenshots for journal generation, providing new insights into psychological states through digital phenotyping.
- 4 December 2024
- Published by Association for Computing Machinery (ACM)
- p. 2347-2352
Open AccessAutoJournaling: A Context-Aware Journaling System Leveraging MLLMs on Smartphone Screenshots
Journaling offers significant benefits, including fostering self-reflection, enhancing writing skills, and aiding in mood monitoring. However, many people abandon the practice because traditional journaling is time-consuming, and detailed life events may be overlooked if not recorded promptly. Given that smartphones are the most widely used devices for entertainment, work, and socialization, they present an ideal platform for innovative approaches to journaling. Despite their ubiquity, the potential of using digital phenotyping, a method of unobtrusively collecting data from digital devices to gain insights into psychological and behavioral patterns, for automated journal generation has been largely underexplored. In this study, we propose AutoJournaling, the first-of-its-kind system that automatically generates journals by collecting and analyzing screenshots from smartphones. This system captures life events and corresponding emotions, offering a novel approach to digital phenotyping. We evaluated AutoJournaling by collecting screenshots every 3 seconds from three students over five days, demonstrating its feasibility and accuracy. AutoJournaling is the first framework to utilize seamlessly collected screenshots for journal generation, providing new insights into psychological states through digital phenotyping.
- Open Access
- 5 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 196-200
ScreenTK: Seamless Detection of Time-Killing Moments Using Continuous Mobile Screen Text and On-Device LLMs
Smartphones have become essential to people's digital lives, providing a continuous stream of information and connectivity. However, this constant flow often depletes users' limited attentional resources and time, leading to decreased productivity and increased stress levels. This issue underscores the need for tools that empowers users to maximize their potential for achieving personal objectives. One effective approach is to identify "time-killing" moments--a specific type of attention surplus--during which users seek to fill perceived free time without a specific purpose. Recent work has utilized screenshots taken every 5 seconds to detect time-killing activities on smartphones. However, this method often misses to capture phone usage between intervals. We demonstrate that up to 50% of time-killing instances go undetected using screenshots, leading to substantial gaps in understanding user behavior. To address this limitation, we propose a method called ScreenTK that detects time-killing moments by leveraging continuous screen text monitoring and on-device large language models (LLMs). Screen text contains more comprehensive information than screenshots and allows LLMs to summarize detailed phone usage. To verify our framework, we conducted experiments with six participants, capturing 1,034 records of different time-killing moments. Initial results show that our framework outperforms state-of-the-art solutions by 38% in our case study.
- 5 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 196-200
Open AccessScreenTK: Seamless Detection of Time-Killing Moments Using Continuous Mobile Screen Text and On-Device LLMs
Smartphones have become essential to people's digital lives, providing a continuous stream of information and connectivity. However, this constant flow often depletes users' limited attentional resources and time, leading to decreased productivity and increased stress levels. This issue underscores the need for tools that empowers users to maximize their potential for achieving personal objectives. One effective approach is to identify "time-killing" moments--a specific type of attention surplus--during which users seek to fill perceived free time without a specific purpose. Recent work has utilized screenshots taken every 5 seconds to detect time-killing activities on smartphones. However, this method often misses to capture phone usage between intervals. We demonstrate that up to 50% of time-killing instances go undetected using screenshots, leading to substantial gaps in understanding user behavior. To address this limitation, we propose a method called ScreenTK that detects time-killing moments by leveraging continuous screen text monitoring and on-device large language models (LLMs). Screen text contains more comprehensive information than screenshots and allows LLMs to summarize detailed phone usage. To verify our framework, we conducted experiments with six participants, capturing 1,034 records of different time-killing moments. Initial results show that our framework outperforms state-of-the-art solutions by 38% in our case study.
- Open Access
- 5 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 186-190
Enabling On-Device LLMs Personalization with Smartphone Sensing
This demo presents a novel end-to-end framework that combines on-device large language models (LLMs) with smartphone sensing technologies to achieve context-aware and personalized services. The framework addresses critical limitations of current personalization solutions via cloud LLMs, such as privacy concerns, latency and cost, and limited personal information. To achieve this, we innovatively proposed deploying LLMs on smartphones with multimodal sensor data through context-aware sensing and customized prompt engineering, ensuring privacy and enhancing personalization performance. A case study involving a university student demonstrated the capability of the framework to provide tailored recommendations. In addition, we show that the framework achieves the best trade-off in privacy, performance, latency, cost, battery and energy consumption between on-device and cloud LLMs. To the best of our knowledge, this is the first framework to provide on-device LLMs personalization with smartphone sensing. Future work will incorporate more diverse sensor data and involve extensive user studies to enhance personalization. Our proposed framework has the potential to substantially improve user experiences across domains including healthcare, productivity, and entertainment.
- 5 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 186-190
Open AccessEnabling On-Device LLMs Personalization with Smartphone Sensing
This demo presents a novel end-to-end framework that combines on-device large language models (LLMs) with smartphone sensing technologies to achieve context-aware and personalized services. The framework addresses critical limitations of current personalization solutions via cloud LLMs, such as privacy concerns, latency and cost, and limited personal information. To achieve this, we innovatively proposed deploying LLMs on smartphones with multimodal sensor data through context-aware sensing and customized prompt engineering, ensuring privacy and enhancing personalization performance. A case study involving a university student demonstrated the capability of the framework to provide tailored recommendations. In addition, we show that the framework achieves the best trade-off in privacy, performance, latency, cost, battery and energy consumption between on-device and cloud LLMs. To the best of our knowledge, this is the first framework to provide on-device LLMs personalization with smartphone sensing. Future work will incorporate more diverse sensor data and involve extensive user studies to enhance personalization. Our proposed framework has the potential to substantially improve user experiences across domains including healthcare, productivity, and entertainment.
- Open Access
- 1 January 2021
- Published by XMLink in Korean Journal of Radiology
- Vol. 22 (11) , 1841-1849
Quantitative Evaluation of Gastrocnemius Medialis Stiffness During Passive Stretching Using Shear Wave Elastography in Patients with Parkinson's Disease: A Prospective Preliminary Study
Yin L, et al. Korean J Radiol. 2021 Nov;22(11):1841-1849. https://doi.org/10.3348/kjr.2020.1338
- 1 January 2021
- Published by XMLink in Korean Journal of Radiology
- Vol. 22 (11) , 1841-1849
Open AccessQuantitative Evaluation of Gastrocnemius Medialis Stiffness During Passive Stretching Using Shear Wave Elastography in Patients with Parkinson's Disease: A Prospective Preliminary Study
Yin L, et al. Korean J Radiol. 2021 Nov;22(11):1841-1849. https://doi.org/10.3348/kjr.2020.1338