Mu Yuan

Biosignals collected from wearable devices are widely utilized in healthcare applications. Machine learning models used in these applications often rely on features extracted from biosignals due to their effectiveness, lower data dimensionality, and wide compatibility across various model architectures. However, existing feature extraction methods often lack task-specific contextual knowledge, struggle to identify optimal features in high-dimensional combinatorial feature space, and are prone to automated code generation and execution errors. In this paper, we propose DeepFeature, the first LLM-empowered, context-aware feature generation framework for wearable biosignals. DeepFeature introduces a multi-source feature generation mechanism that integrates the inherent ability of LLMs, expert knowledge and inter-feature interactions. It also employs an iterative feature refinement process that uses feature assessment-based feedback for feature re-selection. Additionally, DeepFeature utilizes a robust multi-layer filtering and verification approach for feature description-to-code translation to ensure that the feature extraction functions run without crashing. Experimental evaluation results show that DeepFeature achieves the highest average AUROC across eight tasks under both sample-level and subject-level settings, outperforming the best baselines by 4.60% and 4.61%, respectively. DeepFeature achieves the most pronounced gains on the PPG-BP tasks, while remaining competitive with the best-performing baselines on Epilepsy, WESAD, and our self-collected SEN dataset.

Multimodal human action recognition (HAR) utilizes complementary data for activity classification. Built on traditional HAR tasks, recent advances in Large Language Models (LLMs) enable detailed descriptions and causal reasoning of human actions, advancing new tasks of human action understanding (HAU) and human action reasoning (HARn). However, most LLMs, especially multimodal Large Vision-Language Models (LVLMs), struggle with modalities other than RGB images, like depth, IMU, ormmWave, due to a lack of large-scale datasets in these task domains. Existing HAR datasets provide only coarse-grained annotations, in-sufficient for depicting the detailed action dynamics required in HAU and HARn tasks. Simply combining annotations and generating captions with LLMs often lacks necessary logical and spatiotemporal consistency. In this paper, we introduce CUHK-X, a large-scale multi-modal dataset and benchmarks for HAR, HAU, and HARn. It includes 64,267 samples of 40 actions performed by 30 participants across two indoor environments, covering diverse daily scenarios. To address the challenge of spatiotemporal inconsistencies in captions, we propose a prompt-based scene creation method that leverages LLMs to generate logically connected activity sequences. CUHK-X also includes three benchmarks with six tasks to evaluate state-of-the-art models. Experimental results show average accuracies of 76.52% for HAR, 40.76% for HAU, and 70.25% for HARn. This large-scale multimodal dataset aims to empower the research community to apply, develop, and adapt data-intensive learning techniques for a wide range of human activity-related tasks.

The large language model (LLM) based on the Transformer architecture and its derived various applications have greatly changed people's lives. Considering some concerns such as privacy and network conditions, deploying LLM on smart devices has gradually become a research focus. In order to reduce the huge computation and storage overhead of LLMs, many works have studied model compression technology to reduce the model computation and parameter amount, thereby reducing the inference latency. This paper analyzes the characteristics of the on-device LLM service, including small batch size and latency focus, etc. We find the inefficiency of existing model compression technologies and new optimization opportunities, i.e., allocating different layers for different input tokens based on the task QoS requirements. Then we propose CALSI, a context-aware layer skipping LLM inference system for on-device serving. In the offline profiling phase, we analyze the importance of different layers of the model to different tokens and train a lightweight gated predictor. Then, we map the latency QoS requirements of different tasks with the predictor threshold. During the online inference phase, we adaptively allocate the layers that need to be computed to the specific token and design a KV cache delayed computation management mechanism to solve the KV cache missing problem caused by layer skipping. Experiments on real devices show that CALSI can achieve up to 24.1% latency reduction. CALSI has good generalization ability on various models and datasets and is compatible with other existing complementary optimization techniques.

Federated graph learning (FGL) enables multiple participants with distributed but connected graph data to collaboratively train a model in a privacy-preserving way. However, the high communication cost hinders the adoption of FGL in many resource-limited or delay-sensitive applications. In this work, we focus on reducing the communication cost incurred by the transmission of neighborhood information in FGL. We propose to search for local proxies that can play a substitute role as the external neighbors and develop a novel federated graph learning framework named Gproxy. Gproxy utilizes representation similarity and class correlation to select local proxies for external neighbors. Additionally, we propose to dynamically adjust the proxy strategy according to the changing representation of nodes during the iterative training process. We also design a proxy cache to accelerate the search process by reusing proxy search outcomes for similar external neighbors. Furthermore, we provide a theoretical analysis and show that using a proxy node has a similar influence on training when it is sufficiently similar to the external one. Extensive evaluations show that Gproxy significantly reduces communication cost while maintaining model performance compared to strong baselines.