Josh Qixuan Sun

Vision-Language Navigation in Continuous Environments (VLNCE), where an agent follows instructions and moves freely to reach a destination, is a key research problem in embodied AI. However, most existing approaches are sensitive to viewpoint changes, i.e. variations in camera height and viewing angle. Here we introduce a more general scenario, V$^{2}$-VLNCE (VLNCE with Varied Viewpoints) and propose a view-invariant post-training framework, called VIL (View Invariant Learning), that makes existing navigation policies more robust to changes in camera viewpoint. VIL employs a contrastive learning framework to learn sparse and view-invariant features. We also introduce a teacher-student framework for the Waypoint Predictor Module, a standard part of VLNCE baselines, where a view-dependent teacher model distills knowledge into a view-invariant student model. We employ an end-to-end training paradigm to jointly optimize these components. Empirical results show that our method outperforms state-of-the-art approaches on V$^{2}$-VLNCE by 8-15% measured on Success Rate for two standard benchmark datasets R2R-CE and RxR-CE. Evaluation of VIL in standard VLNCE settings shows that despite being trained for varied viewpoints, VIL often still improves performance. On the harder RxR-CE dataset, our method also achieved state-of-the-art performance across all metrics. This suggests that adding VIL does not diminish the standard viewpoint performance and can serve as a plug-and-play post-training method. We further evaluate VIL for simulated camera placements derived from real robot configurations (e.g. Stretch RE-1, LoCoBot), showing consistent improvements of performance. Finally, we present a proof-of-concept real-robot evaluation in two physical environments using a panoramic RGB sensor combined with LiDAR. These results show that VIL improves robustness not only in simulation but also in real-world navigation scenarios, making it a practical strategy for embodied agents. The code is available at https://github.com/realjoshqsun/V2-VLNCE.

Kolmogorov–Arnold Network (KAN) is an emerging interpretable neural network compared to fully black-box MLPs. Recently, emerging works focus on comprehensive and fair comparisons between KAN and MLP in various tasks. However, these works didn't focus on the strongest advantage of KAN: generating symbolic outputs. The ability of KAN to provide scientific insights or even discover new science is under-examined. In this work, we propose several novel metrics to measure how well a KAN performs on symbolic function fitting: R^2-Mean, weighted R^2-complexity loss, and ranking metrics. We also propose a metric to determine mathematical complexity of a target function and evaluate KAN with several functions of different mathematical complexity. Additionally, we also tried inputs with different ranges to find the effect of normalization.

Kolmogorov–Arnold Network (KAN) is an emerging interpretable neural network compared to fully black-box MLPs. Recently, emerging works focus on comprehensive and fair comparisons between KAN and MLP in various tasks. However, these works didn't focus on the strongest advantage of KAN: generating symbolic outputs. The ability of KAN to provide scientific insights or even discover new science is under-examined.In this work, we propose several novel metrics to measure how well a KAN performs on symbolic function fitting: R^2-Mean, weighted R^2-complexity loss, and ranking metrics. We also propose a metric to determine mathematical complexity of a target function and evaluate KAN with several functions of different mathematical complexity.Additionally, we also tried inputs with different ranges to find the effect of normalization.

Multimodal medical image segmentation is always a critical problem in medical image segmentation. Traditional deep learning methods utilize fully CNNs for encoding given images, thus leading to deficiency of long-range dependencies and bad generalization performance. Recently, a sequence of Transformer-based methodologies emerges in the field of image processing, which brings great generalization and performance in various tasks. On the other hand, traditional CNNs have their own advantages, such as rapid convergence and local representations. Therefore, we analyze a hybrid multimodal segmentation method based on Transformers and CNNs and propose a novel architecture, HybridCTrm network. We conduct experiments using HybridCTrm on two benchmark datasets and compare with HyperDenseNet, a network based on fully CNNs. Results show that our HybridCTrm outperforms HyperDenseNet on most of the evaluation metrics. Furthermore, we analyze the influence of the depth of Transformer on the performance. Besides, we visualize the results and carefully explore how our hybrid methods improve on segmentations.