Guodong Guo

We introduce a novel framework with strong generalization capabilities for enabling humanoid robots to perform semantically grounded, context-aware, and physically realizable expressive behaviors, with the goal of enhancing end-user engagement in open-world human–robot interaction (HRI). To achieve this goal, we develop a new large language model–based human cognition module that interprets user dialogs to infer latent intent and generates high-level multi-modal behavior descriptions. These semantic representations are mapped to speech and motion outputs through a dual-stage embodied behavior generation pipeline. The pipeline consists of a shape adaptation module that maps human body motions into the robot’s kinematic space, followed by a motion retargeting module that generates executable joint trajectories under physical constraints. Additionally, the modular architecture enables seamless integration of state-of-the-art generative models and serves as a practical testbed for evaluating expressive behavior generation in real-world settings. We validate this system on a 58-DoF humanoid platform through both controlled video-based studies and live HRI experiments. The results show significant improvements over rule-based and handcrafted baselines in terms of expressiveness, behavioral appeal, and user engagement. This work helps bridging the gap between high-level language understanding and low-level robot control, thereby enabling scalable and human-aligned expressive behavior generation for embodied agents.

Electric motors are key power components in industrial applications and motor-driven household appliances and power tools. Rapid health diagnosis based on short-term operational signals before delivery is critical for quality control and maintenance cost reduction. In practical manufacturing environments, however, this task remains challenging due to fault sample scarcity, class imbalance, and non-stationary operating conditions, which hinder reliable modeling and generalization. This paper presents a large-scale empirical study on the applicability of time-series foundation models (TSFMs), including Chronos and MOMENT, for industrial motor fault diagnosis using real-world factory end-of-line inspection data collected from 46 production motors. A comprehensive experimental framework is constructed, covering multiple adaptation strategies, patch sizes, aggregation methods, and downstream classifiers. The results consistently show that, under small-sample and non-stationary conditions, traditional approaches based on physically motivated features achieve more stable and often superior performance compared to TSFM-based representations in terms of Accuracy, F1-score, and AUROC. To interpret this observation, we introduce the notion of Data-Layer Mismatch, which describes the discrepancy between the statistical characteristics of pretraining data used by TSFMs and those of raw industrial sensor signals. While TSFMs are typically pretrained on relatively smooth and structured time-series data, industrial vibration and acoustic signals are characterized by strong noise, transient impulses, and pronounced non-stationarity. We use this concept as an empirical interpretation of the observed transfer difficulty, rather than as a universal theoretical claim. Further analysis indicates that diagnostic performance strongly depends on the alignment between model inductive biases and task-specific physical properties, including temporal scale, feature aggregation, and representation stability. These observations remain consistent across vibration signals, acoustic signals, and their feature-level fusion. The results are obtained under a specific industrial motor diagnosis setting and should be interpreted as context-dependent empirical evidence, highlighting the need for further investigation into domain adaptation strategies for foundation models in industrial applications.

This work is motivated by the complexity of conventional robot programming methods, which require a deep understanding of both robotics and software development. This situation presents significant challenges for nontechnical end-users who need to program a new robotic task. To address this issue, we propose a new robot programming method that enables humans to specify robot motion tasks using simple and intuitive body-language commands. To this end, we propose a triadic task decomposition framework to decompose complex robot motion tasks into three basic task units, i.e., location, operation, and motion. These units are paired with a set of body-language cues that enable end-users to specify a variety of actions and motions, providing an intuitive alternative to traditional robot programming methods. To program these units, we develop a multimodal fusion-based instruction-parsing algorithm with a dual-phase visual architecture, enabling temporally aware body-language recognition and high-accuracy cross-modal object localization. Furthermore, a semantics-adaptive task execution system is proposed, which incorporates an intent reasoning module that dynamically handles end-users' nonstandard programming sequences and incomplete task logic, enhancing programming success rates and system usability. We evaluate the proposed method on a dual-arm robotic platform and a humanoid robot through various pick-and-place tasks, and compare its performance with that of several baselines from the literature.

Cross-modality object detection aims to fuse complementary information from different modalities to improve model performance, which achieves a wider range of applications. However, traditional cross-modality fusion methods, based on CNN or Transformer, inadequately address the issue of pseudo-target information, which causes model attention dispersion to degrade object detection performance. In this paper, we investigate a novel cross-modality fusion approach by associating cross-modal features in a hidden state space based on an improved Mamba with a gating attention mechanism. We propose the Fusion-Mamba Block (FMB), designed to map cross-modal features into a hidden state space for interaction, thereby refining the model's attention on true target areas and enhancing overall performance. The FMB comprises two key modules: State Space Channel Swapping (SSCS) module, which facilitates the fusion of shallow features, and Dual State Space Fusion (DSSF) module, which enables deep fusion and effectively suppresses pseudo-target information within the hidden state space. Our proposed method outperforms state-of-the-art approaches, achieving improvements of 5.9%, 3.5% and 2.1% mAP on $M^{3}$FD, DroneVehicle and FLIR-Aligned, respectively. To the best of our knowledge, this work establishes a new baseline for cross-modality object detection, providing a robust foundation for future research in this area.

The challenges inherent in long-horizon tasks in robotics persist due to the typical inefficient exploration and sparse rewards in traditional reinforcement learning approaches. To address these challenges, we have developed a novel algorithm, termed hlexplicit-implicit subgoal planning (EISP), designed to tackle long-horizon tasks through a divide-and-conquer approach. We utilize two primary criteria, feasibility and optimality, to ensure the quality of the generated subgoals. EISP consists of three components: a hybrid subgoal generator, a hindsight sampler, and a value selector. The hybrid subgoal generator uses an explicit model to infer subgoals and an implicit model to predict the final goal, inspired by way of human thinking that infers subgoals by using the current state and final goal as well as reason about the final goal conditioned on the current state and given subgoals. Additionally, the hindsight sampler selects valid subgoals from an offline dataset to enhance the feasibility of the generated subgoals. While the value selector utilizes the value function in reinforcement learning to filter the optimal subgoals from subgoal candidates. To validate our method, we conduct four long-horizon tasks in both simulation and the real world. The obtained quantitative and qualitative data indicate that our approach achieves promising performance compared to other baseline methods. These experimental results can be seen on the website https://sites.google.com/view/vaesi.

Sharpness-Aware Minimization (SAM) has emerged as a promising approach for effectively reducing the generalization error. However, SAM incurs twice the computational cost compared to the base optimizer (e.g., SGD). We propose Asymptotic Unbiased data sampling to accelerate SAM (AUSAM), which maintains the model's generalization capacity while significantly enhancing computational efficiency. Concretely, we probabilistically sample a subset of data points beneficial for SAM optimization based on a theoretically guaranteed criterion, i.e., the Gradient Norm of each Sample (GNS). We further approximate the GNS by evaluating the difference in loss values before and after perturbation in SAM. As a plug-and-play, architecture-agnostic method, our approach consistently accelerates SAM across various tasks and networks, i.e., classification, human pose estimation, and network quantization. On CIFAR-10/100 and Tiny-ImageNet, AUSAM achieves results comparable to SAM while providing a speedup of over 70%. By adjusting hyperparameters, AUSAM can match the speed of the base optimizer while significantly surpassing the base optimizer's performance. Compared to recent dynamic data pruning methods, AUSAM is better suited for SAM and excels in maintaining performance. Additionally, AUSAM accelerates optimization in human pose estimation and model quantization without sacrificing performance, demonstrating its broad practicality.

Graph Neural Networks (GNNs) have recently gained widespread attention as a successful tool for analyzing graph-structured data. However, imperfect graph structure with noisy links lacks enough robustness and may damage graph representations, therefore limiting the GNNs' performance in practical tasks. Moreover, existing generative architectures fail to fit discriminative graph-related tasks. To tackle these issues, we introduce an unsupervised method based on a joint of generative training and discriminative training to learn graph structure and representation, aiming to improve the discriminative performance of generative models. We propose an Energy-based Contrastive Learning (ECL) guided Graph Structure Refinement (GSR) framework, denoted as ECL-GSR. To our knowledge, this is the first work to combine energy-based models with contrastive learning for GSR. Specifically, we leverage ECL to approximate the joint distribution of sample pairs, which increases the similarity between representations of positive pairs while reducing the similarity between negative ones. Refined structure is produced by augmenting and removing edges according to the similarity metrics among node representations. Extensive experiments demonstrate that ECL-GSR outperforms the state-of-the-art on eight benchmark datasets in node classification. ECL-GSR achieves faster training with fewer samples and memories against the leading baseline, highlighting its simplicity and efficiency in downstream tasks.

Deep learning-based methods have shown substantial promise in visual depression severity estimation. Nonetheless, their effectiveness is limited by the scarce availability of labeled depression data, potentially leading to overfitting during representation learning. One feasible approach to address this issue is to incorporate, in the training objective, regularization that considers the unique characteristics of depression data. Typical regularization includes the similarity ranking through ordered consistency between visual features and their target scores. However, previous ranking methods are limited to using only samples within a mini-batch, resulting in a decreased regularization effect in depression representation learning. To address this limitation, we propose MemRank, a global similarity ranking method that operates not only on mini-batch samples but also on a well-designed feature memory, which stores smoothed and dynamically updated feature prototypes at diverse levels of depression during training. Furthermore, we show that incorporating the feature memory in the regression loss enhances the stability of training a deep regressor, leading to improved depression predictions. Empirically and analytically, we show that our MemRank outperforms alternative ranking methods and achieves state-of-the-art results on two benchmark datasets. We have released the model weights at https://github.com/PushineLee/MemRank.