Wei-Yen Wang

Task-conditioned robotic manipulation requires grasps that are geometrically plausible and compatible with downstream function. Although vision–language models (VLMs) have shown promise for semantic reasoning in robotic manipulation, VLM-selected grasps may not consistently satisfy both requirements. To address this issue, we propose a two-stage generate-then-reason framework for task-conditioned 6-DoF grasp selection. In Stage 1, a VLM localizes task-relevant object regions, followed by segmentation and GraspNet-based generation of plausible grasp candidates. In Stage 2, another VLM ranks grounded grasp candidates according to their compatibility with the specified downstream task. The evaluation in this journal includes a 60-trial full-system physical study and four controlled replay experiments that examine Stage 1 localization, Stage 2 cross-model ranking, repeated-inference reliability, and cross-stage candidate-pool sensitivity. The physical system achieved an overall task success rate of 63.3%. In the replay analysis, Gemini 3 Flash Preview achieved the highest Stage 1 localization accuracy of 95.0% and the lowest median Stage 1 latency of 11.09 s. For Stage 2 ranking, GPT-5.6 Sol achieved the highest Top-1 task-valid rate of 92.9% on identical frozen candidate pools and showed the strongest repeated-inference behavior among the evaluated critics. The cross-stage sensitivity experiment yielded task-valid selection rates ranging from 70.8% to 87.5% across six model combinations, indicating sensitivity of downstream selection to upstream candidate evidence within the evaluated conditions. These results indicate that physical task completion and stage-specific VLM behavior capture different aspects of system performance and should be evaluated separately when assessing VLM-based grasp-selection systems.

The growing demand for smart surveillance systems, particularly in elderly and patient care, has driven significant advancements in human action recognition (HAR) technologies. Despite these advancements, real-time HAR deployment in Internet of Things (IoT) environments remains limited by latency, bandwidth, and the tradeoff between model size and performance. To address these limitations, we propose Tiny-HAR, a lightweight HAR framework tailored to edge devices. Built on DSConVBlock_N, Tiny-HAR comprises two key modules: a high-level feature extraction module and a temporal deep feature extraction module. Using skeletal data and diverse spatiotemporal information, Tiny-HAR achieves state-of-the-art accuracy while remaining compact. Experimental analysis indicates its high performance across various benchmark datasets: 82.1% on joint-annotated human motion database (JHMDB); 96.7% and 93.2% on SHREC2017 for 14 and 28 classes, respectively; 90.27% on MPOSE2021; and 97.62% on University of Rzeszów (UR) Falling. Tiny-HAR also achieves up to nine times faster inference and 11.5 times fewer giga floating-point operations (GFLOPs) compared with existing models. Deployment on a Raspberry Pi 4 Model B confirmed its real-time capability with an inference time of 2.8 ms. Overall, these results position Tiny-HAR as a promising solution for efficient and reliable HAR in IoT-based health monitoring applications.

Recent advances in embedded computing have enabled edge devices to run AI models more efficiently, sparking interest in deploying video anomaly detection (VAD) systems for smart surveillance. However, practical implementation requires a careful balance between detection accuracy and computational efficiency. This letter proposes a novel and lightweight anomaly scoring model that integrates a normalizing flow with a multi-scale spatial temporal graph convolutional network (stGCN). The proposed model supports both unsupervised and supervised modes. To evaluate its deployment feasibility, we implement the full VAD pipeline—including YOLOv8n-Pose, BoT-SORT, and the proposed scoring model—on a Raspberry Pi 5. Experimental results demonstrate that our method achieves AUC scores of 86.2% and 72.2% on the ShanghaiTech and UBnormal datasets for unsupervised VAD, respectively, and an AUC score of 82.4% for supervised VAD on the UBnormal dataset, outperforming state-of-the-art methods.

With the advance of deep learning, improving the understanding and cognition of artificial intelligence (AI) systems has become an increasingly crucial research trend. Although most AI studies have focused on improving the efficiency and reach of deep learning technologies for the next wave of nascent AI solutions, they have also highlighted the real-world challenges and limitations of current deep learning approaches. In view of this, this paper proposes a novel cognitive system based on deep learning. To mathematically analyze and solve the critical problem of unstable convergence existing in general cognitive systems, we propose a system framework consisting of three models: a perception model, a hypothesis model, and a memory model. In contrast to conventional reinforcement learning systems, the online learning of our proposed cognitive system can be carried out by only comparing the current outputs with the expected inputs. Then, the memory model (as an evaluation model) can estimate the learning results more accurately so that the hypothesis model is capable of generating improved hypotheses. The contribution of our method is to refer to the memory theory in cognitive psychology to improve the stability of the image-to-robot motor end-to-end learning system. Moreover, an auto-encoder, as the perception model, can encode an observed image into a perception code as the features to easily find an optimal solution. To validate the effectiveness of the proposed cognitive system, Chinese calligraphy writing tasks are used to evaluate its performance. Experimental results show that the proposed cognitive system significantly enhances the online learning process with stable convergence and improves the writing performance of the calligraphy work.

The demand for smart surveillance systems has been driven by the ubiquity of cameras in modern society. Among the crucial tasks in such systems, person re-identification (re-ID) and multiple object tracking (MOT) are paramount. Despite the common photographic challenges they share, these tasks serve distinct objectives, complicating their integration into a unified system. To be specific, most existing work lacks a comprehensive study on effectively integrating re-ID models with object trackers to achieve optimal MOT performance. A decrease in MOT performance may occur without proper calibration for the integration of both components despite using an enhanced re-ID model for the tracker. To address these issues, we propose a straightforward and effective solution that integrates an improved re-ID model into a MOT framework, the BoT-SORT tracker, ensuring enhanced MOT performance on the well-known benchmarks MOT17 and MOT20 with fewer parameters for tuning. Recognizing the sub-optimal performance of existing re-ID models with their original loss functions, we introduce a novel loss function that incorporates a polynomial cross-entropy component to enhance training efficiency on closed-world datasets. As a result, the re-ID model trained with the proposed method achieves state-of-the-art performance on Market1501 and DukeMTMC datasets, and is subsequently applied to a BoT-SORT tracker with a post-processing re-ranking module for MOT. Experimental results show that the proposed method achieves 81.2% and 77.8% MOTA scores on MOT17 and MOT20 datasets, respectively, outperforming the state-of-the-art MOT methods.

As the Industry 4.0 landscape unfolds, collaborative robots (cobots) play an important role in intelligent manufacturing. Compared with industrial robots, cobots are more flexible and intuitive in programming, especially for industrial and home service applications; However, there are still issues to be solved, including understanding human intention in a natural way, adaptability to execute tasks, and robot mobility in a working environment. As an attempt to solve the problems aforementioned, in this paper, we propose a modularized solution for mobile cobot systems, where the cobot equipped with a multi-camera localization scheme for self-localization can understand the human intention via human voice commands to execute the tasks in an unseen scenario in a small-area working environment. As far as intention understanding is concerned, we devise a natural language processing approach to establish an action base to describe human commands. According to the action base, the robot can then execute the tasks by planning a trajectory with the help of an object localization module, which integrates the point cloud and the object detected by YOLOv4 to locate the object’s position in three-dimension space. Depending on where the cobot interacts with the object, the cobot might need to navigate around the working environment. Thus, we also establish a low-cost and high-efficiency multi-camera localization system with ArUco markers to locate the mobile cobot in a larger sensing area. Experimental results show that the proposed vision-based mobile cobot can successfully interact with a human operator to assemble a wooden chair in a small workshop.

This paper attempts to use a delta robot’s structure and reliable coordinates to develop a self-learning Chinese calligraphy-writing system that requires precise control. Ideally, to achieve human-like behavior, a delta robot can learn stroke trajectories autonomously and present the stroke beauty of calligraphy characters. Unfortunately, state-of-the-art approaches have not yet considered the presentation of stroke beauty resulting from angles of rotation and tilt of the brush. This paper presents an integrated system consisting of a stroke processing module, a hypothesis generation net (HGN) learning model, a delta robot, and an image capture module. Our approach utilizes both the stroke trajectories from the stroke processing module and angles information from the HGN learning model to automatically produce five degrees of freedom action instructions. Based on the instructions, the delta robot completes calligraphy writing. Then, the image capture module provides feedback to the writing system for error calculation and coordinate correction. We utilize the mean absolute percentage error to verify the performance of the writing results. A correction algorithm and linear regression were used to improve the error correction results (less than 2% error). After several cycles, the written results approached the target sample finally. Consequently, the written results produced by the delta robot prove that our proposed system is capable of self-learning and correction.

Person re-identification (re-ID) is one of the essential tasks for modern visual intelligent systems to identify a person from images or videos captured at different times, viewpoints, and spatial positions. In fact, it is easy to make an incorrect estimate for person re-ID in the presence of illumination change, low resolution, and pose differences. To provide a robust and accurate prediction, machine learning techniques are extensively used nowadays. However, learning-based approaches often face difficulties in data imbalance and distinguishing a person from others having strong appearance similarity. To improve the overall re-ID performance, false positives and false negatives should be part of the integral factors in the design of the loss function. In this work, we refine the well-known AGW baseline by incorporating a focal Tversky loss to address the data imbalance issue and facilitate the model to learn effectively from the hard examples. Experimental results show that the proposed re-ID method reaches rank-1 accuracy of 96.2% (with mAP: 94.5) and rank-1 accuracy of 93% (with mAP: 91.4) on Market1501 and DukeMTMC datasets, respectively, outperforming the state-of-the-art approaches.

The development of autonomous mobile robots (AMR) has brought with its requirements for intelligence and safety. Human action recognition (HAR) within AMR has become increasingly important because it provides interactive cognition between human and AMR. This study presents a full architecture for Edge-AI human action recognition (Edge-AI HAR) to allow AMR to detect human actions in real time. The architecture consists of three parts: a human detection & tracking network, a key frame extraction function, and a human action recognition network. The human action recognition network is a cascade of a DenseNet121 and a two-layer bi-directional LSTM (DLBiLSTM), in which the DenseNet121 is a pre-trained model to extract spatial features from action key frames and the DLBiLSTM provides a deep two-directional LSTM inference to classify complicated time-series human actions. Edge-AI HAR undergoes two optimizations - ROS distributed computation and TensorRT structure optimization - to give a small model structure and high computational efficiency. Edge-AI HAR is demonstrated in two experiments using an autonomous mobile robot and is demonstrated to give an average precision of 97.58% for single action recognition and around 86% for continuous action recognition.