Menghan Wu

Wearable Human Activity Recognition (WHAR) is an important research field of ubiquitous and mobile computing. Deep WHAR models suffer from the overfitting problem caused by the lack of a large amount and variety of labeled data, which is usually addressed by generating data to enlarge the training set, i.e., Data Augmentation (DA). Generative Adversarial Networks (GANs) have shown their excellent data generation ability, and the generalization ability of a classification model can be improved by GAN-based DA. However, existing GANs cannot make full use of the important modality information and fail to balance modality details and global consistency, which cannot meet the requirements of deep multi-modal WHAR. In this paper, a hierarchical multi-modal GAN model (HMGAN) is proposed for WHAR. HMGAN consists of multiple modal generators, one hierarchical discriminator, and one auxiliary classifier. Multiple modal generators can learn the complex multi-modal data distributions of sensor data. Hierarchical discriminator can provide discrimination outputs for both low-level modal discrimination losses and high-level overall discrimination loss to draw a balance between modality details and global consistency. Experiments on five public WHAR datasets demonstrate that HMGAN achieves the state-of-the-art performance for WHAR, outperforming the best baseline by an average of 3.4%, 3.8%, and 3.5% in accuracy, macro F1 score, and weighted F1 score, respectively.

Complex activity recognition is a valuable issue in mobile and wearable computing. Since complex activities are strongly relevant to users' locations, location data can be used in complex activity recognition. Current activity recognition methods utilizing location data either do not make the most of location data or have a “cold-start” problem. In this paper, we propose CAROLINA (complex activity recognition using acceleration, vital sign, and location data) to recognize complex activities from wearable sensors. CAROLINA divides a map into grid cells and builds a grid cell-activity matrix by integrating check-in and ATUS (American Time Use Survey) datasets. Since some grid cells do not have check-in records, there are some null entries in the matrix, and all the entries are reconstructed by matrix factorization. Meanwhile, a POI (point of interest) category dataset is exploited to build a grid cell-grid cell similarity matrix, which is used to reduce the loss of matrix factorization. The results of experiments show that CAROLINA can effectively utilize location data to improve complex activity recognition performance.