Jiyoun Lim

This study proposes a geometry‐ and color‐aware 3D data‐augmentation framework to enhance fashion asset segmentation for digital twin applications. This study focuses on converting 2D fashion videos into 3D point‐cloud data and augmenting the minority classes. The constructed dataset comprised 502 mannequin‐wearing scenes and additional single‐asset captures, totaling 628 instances across 16 categories and 104 items, all recorded using an iPhone 14 Pro. To address class imbalance, three augmentation strategies are introduced: SMOTE3D (normal‐aware coordinate interpolation with red–green–blue jitter), scene reconfiguration, and class‐consistent color shifts. Using OctFormer, OA‐CNNs, and PTv3, the augmented data consistently improved the accuracy, intersection over union (IoU), and mean average precision (mAP), yielding significant gains over a Mix3D plug‐in baseline, as confirmed by a paired Wilcoxon test.

Sleep is one of the most important factors in maintaining both physical and mental health. There are many causes of sleep problems, it is generally necessary to maintain a healthy lifestyle to avoid them. In the medical field, information related to sleep problems including lifestyle information is obtained through interviews, but this approach is limited because it is dependent on the patient’s memory. Thus, there are many studies adopting ecological momentary assessments (EMAs) to collect patient’s lifestyles. Some of them also use smart devices to collect data effectively. However, these studies focused on specific factors such as smoking, exercising so that they have limits to reflect complex narrative of lifestyle patterns. Therefore, we proposed indicators consist of EMAs data for assessing everyday sleep quality and these indicators contain the complex lifestyle contexts in a quantitative manner. First, we collected real-life data using a smartphone through a 4-week data collection experiment. Second, we develop a method of generating daily indexes reflecting geospatial and social habits, social condition, activity level, and emotional condition using self-report data. Third, we evaluated daily indexes whether could use to supplement indicators comprising features using EMAs from conventional sleep questionnaires. The goal of analysis consists of five metrics of sleep quality that explain perceived sleep quality. The result of analysis indicates that features using both daily indexes and sleep questionnaires lead to better prediction of sleep quality. Additionally, it also shows the potential to generate indicators identifying complex human behaviors with the help of mobile devices and EMAs. Further research on user-friendly data acquisition methods and more diverse lifestyle information should be useful to support behavior decisions for better sleep in well-being services and in specialized medical fields.

Speech emotion recognition (SER) is a natural method of recognizing individual emotions in everyday life. To distribute SER models to real-world applications, some key challenges must be overcome, such as the lack of datasets tagged with emotion labels and the weak generalization of the SER model for an unseen target domain. This study proposes a multi-path and group-loss-based network (MPGLN) for SER to support multi-domain adaptation. The proposed model includes a bidirectional long short-term memory-based temporal feature generator and a transferred feature extractor from the pre-trained VGG-like audio classification model (VGGish), and it learns simultaneously based on multiple losses according to the association of emotion labels in the discrete and dimensional models. For the evaluation of the MPGLN SER as applied to multi-cultural domain datasets, the Korean Emotional Speech Database (KESD), including KESDy18 and KESDy19, is constructed, and the English-speaking Interactive Emotional Dyadic Motion Capture database (IEMOCAP) is used. The evaluation of multi-domain adaptation and domain generalization showed 3.7% and 3.5% improvements, respectively, of the F1 score when comparing the performance of MPGLN SER with a baseline SER model that uses a temporal feature generator. We show that the MPGLN SER efficiently supports multi-domain adaptation and reinforces model generalization.

In this paper, we perform a systematic study about the on-body sensor positioning and data acquisition details for Human Activity Recognition (HAR) systems. We build a testbed that consists of eight body-worn Inertial Measurement Units (IMU) sensors and an Android mobile device for activity data collection. We develop a Long Short-Term Memory (LSTM) network framework to support training of a deep learning model on human activity data, which is acquired in both real-world and controlled environments. From the experiment results, we identify that activity data with sampling rate as low as 10 Hz from four sensors at both sides of wrists, right ankle, and waist is sufficient in recognizing Activities of Daily Living (ADLs) including eating and driving activity. We adopt a two-level ensemble model to combine class-probabilities of multiple sensor modalities, and demonstrate that a classifier-level sensor fusion technique can improve the classification performance. By analyzing the accuracy of each sensor on different types of activity, we elaborate custom weights for multimodal sensor fusion that reflect the characteristic of individual activities.