Yue Zhang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Yue Zhang has a total of 28 co-authors. Here are the top co-authors by number of shared publications:
- Shijia Pan
University of California, Merced, Merced, CA, USA
10 shared publications - Zhizhang Hu
Computer Science and Engineering, University of California Merced
6 shared publications - Alyssa Weakley
University of California, Davis, Davis, California, USA
2 shared publications - Dong Yoon Lee
University of California, Merced, Merced, California, USA
2 shared publications - Hao-Chuan Wang
University of California, Davis, Davis, CA, USA
2 shared publications - Shiwei Fang
Augusta University, Augusta, GA, USA
2 shared publications
Latest Publications
- Open Access
- 3 June 2024
- Published by Association for Computing Machinery (ACM)
- p. 24-29
Are Large Language Models Capable of Causal Reasoning for Sensing Data Analysis?
The correlation analysis between socioeconomic factors and environmental impact is essential for policy making to ensure sustainability and economic development simultaneously. With the development of Internet of Things (IoT), citizen science IoT monitoring provides valuable environmental measurements, such as PM 2.5 for air quality monitoring. However, socioeconomic factors are usually interconnected and confound each other, making accurate correlation analysis challenging. To isolate this information on an individual socioeconomic factor, we need to mitigate the confounding effect (e.g., propensity score matching) of other factors on the environmental sensing data. Large language models (LLMs) have shown remarkable capabilities in data reasoning, making us wonder if they can conduct causal reasoning and answer questions like "What is the most important socioeconomic factor that impacts regional air quality?" In this paper, we present a new evaluation framework named "Order-of-Thought" based on Bloom's Taxonomy pedagogical framework to quantify the LLMs' ability for causal reasoning. We apply this evaluation framework with both natural language-based and program-based prompting strategies. Our evaluation uncovers the exceptional potentials of LLMs in causal reasoning for sensing data analysis, offering valuable insights regarding their capabilities and limitations, and providing useful directions to further achieve a higher-order thought.
- 3 June 2024
- Published by Association for Computing Machinery (ACM)
- p. 24-29
Open AccessAre Large Language Models Capable of Causal Reasoning for Sensing Data Analysis?
The correlation analysis between socioeconomic factors and environmental impact is essential for policy making to ensure sustainability and economic development simultaneously. With the development of Internet of Things (IoT), citizen science IoT monitoring provides valuable environmental measurements, such as PM 2.5 for air quality monitoring. However, socioeconomic factors are usually interconnected and confound each other, making accurate correlation analysis challenging. To isolate this information on an individual socioeconomic factor, we need to mitigate the confounding effect (e.g., propensity score matching) of other factors on the environmental sensing data. Large language models (LLMs) have shown remarkable capabilities in data reasoning, making us wonder if they can conduct causal reasoning and answer questions like "What is the most important socioeconomic factor that impacts regional air quality?" In this paper, we present a new evaluation framework named "Order-of-Thought" based on Bloom's Taxonomy pedagogical framework to quantify the LLMs' ability for causal reasoning. We apply this evaluation framework with both natural language-based and program-based prompting strategies. Our evaluation uncovers the exceptional potentials of LLMs in causal reasoning for sensing data analysis, offering valuable insights regarding their capabilities and limitations, and providing useful directions to further achieve a higher-order thought.
- Open Access
- 3 June 2024
- Published by Association for Computing Machinery (ACM)
- p. 694-695
Poster: PrivaSee: Augmented Reality-Enabled Privacy Perception Visualization for Internet of Things
Internet of Things (IoT) provides a wide range of services to improve convenience and comfort in our daily lives. However, various sensors equipped on IoT devices often raise privacy concerns. Prior works on privacy focus on passive protection from the data and device perspective, such as data encryption and communication protocol design. In this work, we introduce PrivaSee, an augmented reality (AR)-enabled privacy visualization platform to empower users with proactive privacy protection by enhancing their understanding of privacy perception for multimodal sensors.
- 3 June 2024
- Published by Association for Computing Machinery (ACM)
- p. 694-695
Open AccessPoster: PrivaSee: Augmented Reality-Enabled Privacy Perception Visualization for Internet of Things
Internet of Things (IoT) provides a wide range of services to improve convenience and comfort in our daily lives. However, various sensors equipped on IoT devices often raise privacy concerns. Prior works on privacy focus on passive protection from the data and device perspective, such as data encryption and communication protocol design. In this work, we introduce PrivaSee, an augmented reality (AR)-enabled privacy visualization platform to empower users with proactive privacy protection by enhancing their understanding of privacy perception for multimodal sensors.
- Open Access
- 6 March 2024
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Vol. 8 (1) , 1-29
IOTeeth
While occlusal diseases - the main cause of tooth loss -- significantly impact patients' teeth and well-being, they are the most underdiagnosed dental diseases nowadays. Experiencing occlusal diseases could result in difficulties in eating, speaking, and chronicle headaches, ultimately impacting patients' quality of life. Although attempts have been made to develop sensing systems for teeth activity monitoring, solutions that support sufficient sensing resolution for occlusal monitoring are missing. To fill that gap, this paper presents IOTeeth, a cost-effective and automated intra-oral sensing system for continuous and fine-grained monitoring of occlusal diseases. The IOTeeth system includes an intra-oral piezoelectric-based sensing array integrated into a dental retainer platform to support reliable occlusal disease recognition. IOTeeth focuses on biting and grinding activities from the canines and front teeth, which contain essential information of occlusion. IOTeeth's intra-oral wearable collects signals from the sensors and fetches them into a lightweight and robust deep learning model called Physioaware Attention Network (PAN Net) for occlusal disease recognition. We evaluate IOTeeth with 12 articulator teeth models from dental clinic patients. Evaluation results show an F1 score of 0.97 for activity recognition with leave-one-out validation and an average F1 score of 0.92 for dental disease recognition for different activities with leave-one-out validation.
- 6 March 2024
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Vol. 8 (1) , 1-29
Open AccessIOTeeth
While occlusal diseases - the main cause of tooth loss -- significantly impact patients' teeth and well-being, they are the most underdiagnosed dental diseases nowadays. Experiencing occlusal diseases could result in difficulties in eating, speaking, and chronicle headaches, ultimately impacting patients' quality of life. Although attempts have been made to develop sensing systems for teeth activity monitoring, solutions that support sufficient sensing resolution for occlusal monitoring are missing. To fill that gap, this paper presents IOTeeth, a cost-effective and automated intra-oral sensing system for continuous and fine-grained monitoring of occlusal diseases. The IOTeeth system includes an intra-oral piezoelectric-based sensing array integrated into a dental retainer platform to support reliable occlusal disease recognition. IOTeeth focuses on biting and grinding activities from the canines and front teeth, which contain essential information of occlusion. IOTeeth's intra-oral wearable collects signals from the sensors and fetches them into a lightweight and robust deep learning model called Physioaware Attention Network (PAN Net) for occlusal disease recognition. We evaluate IOTeeth with 12 articulator teeth models from dental clinic patients. Evaluation results show an F1 score of 0.97 for activity recognition with leave-one-out validation and an average F1 score of 0.92 for dental disease recognition for different activities with leave-one-out validation.
- Open Access
- 28 February 2024
- Published by Association for Computing Machinery (ACM)
- p. 27-32
KAP: Kinetic Augmented Pill Bottle for Vibration-Based Medication Interaction Recognition
This paper introduces KAP, an in-home medication interaction recognition system targeting unintentional medication non-adherence. KAP leverages kinetic-augmented pill bottles that have unique IDs embedded into their acoustic signatures for vibrations generated when they are put down. This enhanced pill bottle can be seamlessly integrated with existing vibration-based in-home occupant monitoring systems to detect and recognize medication interactions. Compared to prior work on medication non-adherence monitoring, such as using specially designed pill bottles, pill dispensers, or scanners, our system is battery-free and non-intrusive, while preserving the original usage of the standardized pill bottles from pharmacies. We design and implement KAP with a 3D-printed kinetic-augmentation attachment and a kinetic model-guided feature extraction and recognition algorithm. We conduct real-world data collection and depict preliminary results to demonstrate the feasibility of KAP.
- 28 February 2024
- Published by Association for Computing Machinery (ACM)
- p. 27-32
Open AccessKAP: Kinetic Augmented Pill Bottle for Vibration-Based Medication Interaction Recognition
This paper introduces KAP, an in-home medication interaction recognition system targeting unintentional medication non-adherence. KAP leverages kinetic-augmented pill bottles that have unique IDs embedded into their acoustic signatures for vibrations generated when they are put down. This enhanced pill bottle can be seamlessly integrated with existing vibration-based in-home occupant monitoring systems to detect and recognize medication interactions. Compared to prior work on medication non-adherence monitoring, such as using specially designed pill bottles, pill dispensers, or scanners, our system is battery-free and non-intrusive, while preserving the original usage of the standardized pill bottles from pharmacies. We design and implement KAP with a 3D-printed kinetic-augmentation attachment and a kinetic model-guided feature extraction and recognition algorithm. We conduct real-world data collection and depict preliminary results to demonstrate the feasibility of KAP.
- Open Access
- 12 November 2023
- Published by Association for Computing Machinery (ACM)
- p. 516-517
Poster Abstract: LEVO: LEGO® Bricks Enhanced Single-Point Vibration Sensing for Occupant Monitoring | Proceedings of the 21st ACM Conference on Embedded Networked Sensor Systems
The rising older adults population has led to an increased demand for in-home health monitoring to support their well-being in daily life. For instance, localization and tracking are essential applications in elderly monitoring since they can provide various information on health, mobility and can detect falls. The required power and computational resources of traditional acoustic sensor-array solutions make them unavailable on power- and computation- constrained embedded devices. In this paper, we present LEVO, a single-point directional acoustic sensing system that leverages simple LEGO® bricks to build up a physical structure that can embed directional information into a signal waveform. Our preliminary results verifies the feasibility of adopting LEVO for signal direction recognition from signal-point sensing data.
- 12 November 2023
- Published by Association for Computing Machinery (ACM)
- p. 516-517
Open AccessPoster Abstract: LEVO: LEGO® Bricks Enhanced Single-Point Vibration Sensing for Occupant Monitoring | Proceedings of the 21st ACM Conference on Embedded Networked Sensor Systems
The rising older adults population has led to an increased demand for in-home health monitoring to support their well-being in daily life. For instance, localization and tracking are essential applications in elderly monitoring since they can provide various information on health, mobility and can detect falls. The required power and computational resources of traditional acoustic sensor-array solutions make them unavailable on power- and computation- constrained embedded devices. In this paper, we present LEVO, a single-point directional acoustic sensing system that leverages simple LEGO® bricks to build up a physical structure that can embed directional information into a signal waveform. Our preliminary results verifies the feasibility of adopting LEVO for signal direction recognition from signal-point sensing data.
- Open Access
- 12 November 2023
- Published by Association for Computing Machinery (ACM)
- p. 478-479
Demo Abstract: PA-Pill : Physical Augmented Pill bottle for Vibration-Based Medication Intake Monitoring | Proceedings of the 21st ACM Conference on Embedded Networked Sensor Systems
Medication intake monitoring is pivotal for healthcare management. While several medication management technologies have been developed, limitations include cost, adoption and adherence, and learning how to use a new system, which is particularly challenging for older adults with cognitive impairment. In this work, we present PA-Pill, a vibration-based sensing system for medication intake monitoring. PA-Pill leverages a physical structure with a unique vibration signature for medication intake event detection and recognition. We implement a prototype and depict preliminary results to verify the feasibility.
- 12 November 2023
- Published by Association for Computing Machinery (ACM)
- p. 478-479
Open AccessDemo Abstract: PA-Pill : Physical Augmented Pill bottle for Vibration-Based Medication Intake Monitoring | Proceedings of the 21st ACM Conference on Embedded Networked Sensor Systems
Medication intake monitoring is pivotal for healthcare management. While several medication management technologies have been developed, limitations include cost, adoption and adherence, and learning how to use a new system, which is particularly challenging for older adults with cognitive impairment. In this work, we present PA-Pill, a vibration-based sensing system for medication intake monitoring. PA-Pill leverages a physical structure with a unique vibration signature for medication intake event detection and recognition. We implement a prototype and depict preliminary results to verify the feasibility.
- Open Access
- 9 May 2023
- Published by Association for Computing Machinery (ACM)
- p. 266-271
Augmenting Vibration-Based Customer-Product Interaction Recognition with Sparse Load Sensing
This paper introduces a multimodal solution for autonomous retail customer-product interaction recognition using a combination of vibration and load sensing. Scalable and robust customer-product interaction recognition is important for autonomous retail. Current efforts focus on computer vision-based approaches, which are prone to occlusion from both customers and shelves. A densely deployed load cell array can mitigate this issue, however, the high cost and maintenance effort make it difficult to scale up for large deployments. Vibration-based approaches are also explored to detect such interaction, however, the robustness over noisy environments is limited. We propose a multimodal solution with sparse vibration and load sensing on the shelves. These two modalities are complementary in terms of information – load sensing can effectively detect the weight changes on the shelf while vibration sensing can recognize detailed interaction. Preliminary results are presented to demonstrate the complementarity of these two modalities. Our system is able to augment product recognition performance with the combination of sparse vibration and load sensing.
- 9 May 2023
- Published by Association for Computing Machinery (ACM)
- p. 266-271
Open AccessAugmenting Vibration-Based Customer-Product Interaction Recognition with Sparse Load Sensing
This paper introduces a multimodal solution for autonomous retail customer-product interaction recognition using a combination of vibration and load sensing. Scalable and robust customer-product interaction recognition is important for autonomous retail. Current efforts focus on computer vision-based approaches, which are prone to occlusion from both customers and shelves. A densely deployed load cell array can mitigate this issue, however, the high cost and maintenance effort make it difficult to scale up for large deployments. Vibration-based approaches are also explored to detect such interaction, however, the robustness over noisy environments is limited. We propose a multimodal solution with sparse vibration and load sensing on the shelves. These two modalities are complementary in terms of information – load sensing can effectively detect the weight changes on the shelf while vibration sensing can recognize detailed interaction. Preliminary results are presented to demonstrate the complementarity of these two modalities. Our system is able to augment product recognition performance with the combination of sparse vibration and load sensing.
- Open Access
- 9 May 2023
- Published by Association for Computing Machinery (ACM)
- p. 96-109
CMA: Cross-Modal Association Between Wearable and Structural Vibration Signal Segments for Indoor Occupant Sensing
Indoor occupant sensing enables many smart home applications, and various sensing systems have been explored. Based on their installation requirements, we consider two categories of sensors – on- and off-body – and we look into the combination of them for occupant sensing due to their spatial and temporal complementarity. We focus on an example modality pair of wearable IMU and structural vibration that demonstrate modality complementarity in prior work. However, current efforts are built upon the assumption that the knowledge of the signal segments from two modalities are known, which is challenged in a multiple occupants co-living scenario. Therefore, establishing accurate cross-modal signal segment associations is essential to ensure that a correct complementary relationship is learned. We present CMA, a cross-modal signal segment association scheme between structural vibration and wearable sensors. We propose AD-TCN, a framework built upon a temporal convolutional network that calculates the amount of shared context between an structural vibration sensor and associated wearable sensor candidates from the parameters of the trained model. We evaluate CMA via a public multimodal dataset for systematic evaluation, and we collect a continuous uncontrolled dataset for robustness evaluation. CMA achieves up to AUC value, F1 score, and accuracy improvement compared to baselines.
- 9 May 2023
- Published by Association for Computing Machinery (ACM)
- p. 96-109
Open AccessCMA: Cross-Modal Association Between Wearable and Structural Vibration Signal Segments for Indoor Occupant Sensing
Indoor occupant sensing enables many smart home applications, and various sensing systems have been explored. Based on their installation requirements, we consider two categories of sensors – on- and off-body – and we look into the combination of them for occupant sensing due to their spatial and temporal complementarity. We focus on an example modality pair of wearable IMU and structural vibration that demonstrate modality complementarity in prior work. However, current efforts are built upon the assumption that the knowledge of the signal segments from two modalities are known, which is challenged in a multiple occupants co-living scenario. Therefore, establishing accurate cross-modal signal segment associations is essential to ensure that a correct complementary relationship is learned. We present CMA, a cross-modal signal segment association scheme between structural vibration and wearable sensors. We propose AD-TCN, a framework built upon a temporal convolutional network that calculates the amount of shared context between an structural vibration sensor and associated wearable sensor candidates from the parameters of the trained model. We evaluate CMA via a public multimodal dataset for systematic evaluation, and we collect a continuous uncontrolled dataset for robustness evaluation. CMA achieves up to AUC value, F1 score, and accuracy improvement compared to baselines.
- Open Access
Supervised Off-Policy Ranking
Off-policy evaluation (OPE) is to evaluate a target policy with data generated by other policies. Most previous OPE methods focus on precisely estimating the true performance of a policy. We observe that in many applications, (1) the end goal of OPE is to compare two or multiple candidate policies and choose a good one, which is a much simpler task than precisely evaluating their true performance; and (2) there are usually multiple policies that have been deployed to serve users in real-world systems and thus the true performance of these policies can be known. Inspired by the two observations, in this work, we study a new problem, supervised off-policy ranking (SOPR), which aims to rank a set of target policies based on supervised learning by leveraging off-policy data and policies with known performance. We propose a method to solve SOPR, which learns a policy scoring model by minimizing a ranking loss of the training policies rather than estimating the precise policy performance. The scoring model in our method, a hierarchical Transformer based model, maps a set of state-action pairs to a score, where the state of each pair comes from the off-policy data and the action is taken by a target policy on the state in an offline manner. Extensive experiments on public datasets show that our method outperforms baseline methods in terms of rank correlation, regret value, and stability. Our code is publicly available at GitHub.
Open AccessSupervised Off-Policy Ranking
Off-policy evaluation (OPE) is to evaluate a target policy with data generated by other policies. Most previous OPE methods focus on precisely estimating the true performance of a policy. We observe that in many applications, (1) the end goal of OPE is to compare two or multiple candidate policies and choose a good one, which is a much simpler task than precisely evaluating their true performance; and (2) there are usually multiple policies that have been deployed to serve users in real-world systems and thus the true performance of these policies can be known. Inspired by the two observations, in this work, we study a new problem, supervised off-policy ranking (SOPR), which aims to rank a set of target policies based on supervised learning by leveraging off-policy data and policies with known performance. We propose a method to solve SOPR, which learns a policy scoring model by minimizing a ranking loss of the training policies rather than estimating the precise policy performance. The scoring model in our method, a hierarchical Transformer based model, maps a set of state-action pairs to a score, where the state of each pair comes from the off-policy data and the action is taken by a target policy on the state in an offline manner. Extensive experiments on public datasets show that our method outperforms baseline methods in terms of rank correlation, regret value, and stability. Our code is publicly available at GitHub.