Honglin Sun

The Segment Anything Model (SAM) provides a versatile promptable segmentation interface, but adverse weather and image degradation can obscure visual cues and substantially reduce its accuracy. Existing approaches mainly follow two directions to mitigate this issue. The first adapts SAM through fine-tuning or additional trainable modules, which requires model-side optimization. The second preprocesses degraded images before SAM inference, but such methods are generally optimized for perceptual restoration rather than downstream segmentation. We introduce WeatherAgent-SAM, an agentic framework that improves degraded-image segmentation through response-conditioned preprocessing selection while keeping SAM frozen. WeatherAgent-SAM takes SAM’s initial response and a spatial prompt as inputs and uses response-conditioned knowledge to determine which preprocessing method should be applied next. To realize this process, the framework comprises two key modules. First, Fine-Grained Response-Conditioned Knowledge Construction adopts a progressive coarse-to-fine strategy: it first categorizes SAM responses into recurring states and then explores how individual preprocessing methods affect segmentation within each state. This design enables the agent to distinguish methods that pursue similar enhancement objectives but produce different segmentation outcomes. Second, a learnable Mask Verifier estimates the quality of each resulting SAM mask without ground truth and provides feedback to the agent, allowing tool selection to proceed iteratively and stop when a satisfactory prediction is identified. Experiments demonstrate that WeatherAgent-SAM achieves results comparable to those of the evaluated baselines on the Robust-Seg dataset and outperforms SAM, RobustSAM, and GaRA-SAM-Syn on the ACDC dataset.

Vision-language-action (VLA) models perform well on short-horizon manipulation but degrade on long-horizon tasks, where accumulated multimodal context introduces irrelevant signals that interfere with action prediction. We propose Gate$^2$-VLA, a lightweight framework that improves long-horizon performance by regulating attention routing within a VLA model. Gate$^2$-VLA introduces a query-dependent gate on each attention head, applies group-relative policy optimization with rollout filtering for post-training under sparse rewards, and couples the two through a gate-aware extension that adds routing statistics to both the filtering step and the advantage estimation. We evaluate Gate$^2$-VLA on the LIBERO-Long benchmark under full fine-tuning and few-shot setting. Across both settings, Gate$^2$-VLA outperforms representative baselines despite using a backbone with substantially fewer parameters. Ablation studies show that the gain comes from the joint operation of gating, filtering, and gate-aware advantage estimation rather than from any single component.

Robotic manipulators in warehousing and logistics often face complex tasks that involve multiple steps, frequent task switching, and long-term dependencies. Inspired by the hierarchical structure of human motor control, this paper proposes a Hierarchical Reinforcement Learning (HRL) framework utilizing a multi-task goal-conditioned Decision Transformer (MTGC-DT). The high-level policy treats the Markov decision process as a sequence modeling task, allowing the agent to manage temporal dependencies. The low-level policy is made up of parameterized action primitives that handle physical execution. This design improves long-term reasoning and generalization. This method is evaluated on two common logistics manipulation tasks: sequential stacking and spatial sorting with sparse reward and low-quality dataset. The main contributions include introducing a HRL framework that integrates Decision Transformer (DT) with task and goal embeddings, along with a path-efficiency loss (PEL) correction and designing a parameterized, learnable primitive skill library for low-level control to enhance generalization and reusability. Experimental results demonstrate that the proposed Decision Transformer-based Hierarchical Reinforcement Learning (DT-HRL) achieves over a 10% higher success rate and over 8% average reward compared with the baseline, and a normalized score increase of over 2% in the ablation experiments.

Walking robots have been widely applied in complex terrains due to their good terrain adaptability and trafficability. However, in some environments (such as disaster relief, field navigation, etc.), although a single strategy can adapt to various environments, it is unable to strike a balance between speed and stability. Existing control schemes like model predictive control (MPC) and traditional incremental control can manage certain environments. However, they often cannot balance speed and stability well. These methods usually rely on a single strategy and lack adaptability for dynamic adjustment to different terrains. To address this limitation, this paper proposes an innovative double-layer reinforcement learning algorithm. This algorithm combines Deep Double Q-Network (DDQN) and Proximal Policy Optimization (PPO), leveraging their complementary strengths to achieve both fast adaptation and high stability in complex terrains. This algorithm utilizes terrain information and the robot’s state as observations, determines the walking speed command of the quadruped robot Unitree Go1 through DDQN, and dynamically adjusts the current walking speed in complex terrains based on the robot action control system of PPO. The speed command serves as a crucial link between the robot’s perception and movement, guiding how fast the robot should walk depending on the environment and its internal state. By using DDQN, the algorithm ensures that the robot can set an appropriate speed based on what it observes, such as changes in terrain or obstacles. PPO then executes this speed, allowing the robot to navigate in real time over difficult or uneven surfaces, ensuring smooth and stable movement. Then, the proposed model is verified in detail in Isaac Gym. Wecompare the distances walked by the robot using six different control methods within 10 s. The experimental results indicate that the method proposed in this paper demonstrates excellent speed adjustment ability in complex terrains. On the designed test route, the quadruped robot Unitree Go1 can not only maintain a high walking speed but also maintain a high degree of stability when switching between different terrains. Ouralgorithm helps the robot walk 25.5 m in 10 s, outperforming other methods.

Estimating the 6D pose and size of objects is crucial in the task of visual grasping for robotic arms. Most current algorithms still require the 3D CAD model of the target object to match with the detected points, and they are unable to predict the object’s size, which significantly limits the generalizability of these methods. In this paper, we introduce category priors and extract high-dimensional abstract features from both the observed point cloud and the prior to predict the deformation matrix of the reconstructed point cloud and the dense correspondence between the reconstructed and observed point clouds. Furthermore, we propose a staged geometric correction and dense correspondence refinement mechanism to enhance the accuracy of regression. In addition, a novel lightweight attention module is introduced to further integrate the extracted features and identify potential correlations between the observed point cloud and the category prior. Ultimately, the object’s translation, rotation, and size are obtained by mapping the reconstructed point cloud to a normalized canonical coordinate system. Through extensive experiments, we demonstrate that our algorithm outperforms existing methods in terms of performance and accuracy on commonly used benchmarks for this type of problem. Additionally, we implement the algorithm in robotic arm-grasping simulations, further validating its effectiveness.

As technology rapidly evolves, the application of bipedal robots in various environments has widely expanded. These robots, compared to their wheeled counterparts, exhibit a greater degree of freedom and a higher complexity in control, making the challenge of maintaining balance and stability under changing wind speeds particularly intricate. Overcoming this challenge is critical as it enables bipedal robots to sustain more stable gaits during outdoor tasks, thereby increasing safety and enhancing operational efficiency in outdoor settings. To transcend the constraints of existing methodologies, this research introduces an adaptive bio-inspired exploration framework for bipedal robots facing wind disturbances, which is based on the Deep Deterministic Policy Gradient (DDPG) approach. This framework allows the robots to perceive their bodily states through wind force inputs and adaptively modify their exploration coefficients. Additionally, to address the convergence challenges posed by sparse rewards, this study incorporates Hindsight Experience Replay (HER) and a reward-reshaping strategy to provide safer and more effective training guidance for the agents. Simulation outcomes reveal that robots utilizing this advanced method can more swiftly explore behaviors that contribute to stability in complex conditions, and demonstrate improvements in training speed and walking distance over traditional DDPG algorithms.

We propose a motion forecasting model called BANet, which means Boundary-Aware Network, and it is a variant of LaneGCN. We believe that it is not enough to use only the lane centerline as input to obtain the embedding features of the vector map nodes. The lane centerline can only provide the topology of the lanes, and other elements of the vector map also contain rich information. For example, the lane boundary can provide traffic rule constraint information such as whether it is possible to change lanes which is very important. Therefore, we achieved better performance by encoding more vector map elements in the motion forecasting model.We report our results on the 2022 Argoverse2 Motion Forecasting challenge and rank 1st on the test leaderboard.