Peng-Yuan Kao
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Peng-Yuan Kao has a total of 18 co-authors. Here are the top co-authors by number of shared publications:
- I-Ju Hsieh
Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan
2 shared publications - Kuan-Wei Tseng
Department of Computer Science, Tokyo Institute of Technology, Tokyo, Japan
2 shared publications - Timothy Chen
Graduate Institute of Computer Science and Information Engineering, National Taiwan University,Taipei,Taiwan
2 shared publications - Yi-Ping Hung
Graduate Institute of Networking and Multimedia, National Taiwan University, Taipei, Taiwan
2 shared publications - Yi-Ping Hung
National Taiwan University, Taipei, Taiwan
2 shared publications - Yo-Chung Lau
National Taiwan University
2 shared publications
Latest Publications
- Open Access
- 11 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Indoor and Seamless Positioning and Navigation
- Vol. 1, 211-220
Real-Time Object Pose Tracking System With Low Computational Cost for Mobile Devices
Real-time object pose estimation and tracking is challenging but essential for some emerging applications such as augmented reality (AR). In general, state-of-the-art methods address this problem using deep neural networks which indeed yield satisfactory results. Nevertheless, the high computational cost of these methods makes them unsuitable for mobile devices where real-world applications usually take place. We propose real-time object pose tracking system with low computational cost for mobile devices. It is a monocular inertial-assisted-visual system with a client-server architecture connected by high-speed networking. Inertial measurement unit (IMU) pose propagation is performed on the client side for fast pose tracking, and RGB image-based 3D object pose estimation is performed on the server side to obtain accurate poses, after which the pose is sent to the client side for refinement, where we propose a bias self-correction mechanism to reduce the drift. We also propose a fast and effective pose inspection algorithm to detect tracking failures and incorrect pose estimation. In this way, the pose updates rapidly even within 5 ms on low-level devices, making it possible to support real-time tracking for applications. Additionally, an object pose dataset with RGB images and IMU measurements is delivered for evaluation. Experiments also show our method performs well with both accuracy and robustness.
- 11 December 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Indoor and Seamless Positioning and Navigation
- Vol. 1, 211-220
Open AccessReal-Time Object Pose Tracking System With Low Computational Cost for Mobile Devices
Real-time object pose estimation and tracking is challenging but essential for some emerging applications such as augmented reality (AR). In general, state-of-the-art methods address this problem using deep neural networks which indeed yield satisfactory results. Nevertheless, the high computational cost of these methods makes them unsuitable for mobile devices where real-world applications usually take place. We propose real-time object pose tracking system with low computational cost for mobile devices. It is a monocular inertial-assisted-visual system with a client-server architecture connected by high-speed networking. Inertial measurement unit (IMU) pose propagation is performed on the client side for fast pose tracking, and RGB image-based 3D object pose estimation is performed on the server side to obtain accurate poses, after which the pose is sent to the client side for refinement, where we propose a bias self-correction mechanism to reduce the drift. We also propose a fast and effective pose inspection algorithm to detect tracking failures and incorrect pose estimation. In this way, the pose updates rapidly even within 5 ms on low-level devices, making it possible to support real-time tracking for applications. Additionally, an object pose dataset with RGB images and IMU measurements is delivered for evaluation. Experiments also show our method performs well with both accuracy and robustness.
- Open Access
- 6 December 2023
- Published by Association for Computing Machinery (ACM)
- p. 1-8
Domain-Adaptive Mean Teacher for Category-Level Object Pose Estimation
Category-level object pose estimation aims at predicting 6-DoF object poses for previously unseen objects. Current methods mostly rely on ground-truth labels such as object poses and CAD models. However, annotating these labels manually is time-consuming and error-prone in the real-world scenario. Hence, we propose a novel method to solve unsupervised domain adaptation (UDA) for category-level object pose estimation. We adopt a teacher-student framework to utilize both labeled synthetic data and unlabeled real-world data. The student and the teacher are trained to make consistent predictions under different perturbations. Furthermore, we introduce domain adversarial training to bridge the domain gap between synthetic and real-world data. To prevent false feature alignment between domains, we adopt multiple discriminators instead of a single one and perform category-aware alignments. Extensive experiments show that our method achieves state-of-the-art performance on the REAL275 dataset. Through ablation studies, we also demonstrate that our method is not restricted to certain network architecture and can serve as a general UDA method for category-level object pose estimation.
- 6 December 2023
- Published by Association for Computing Machinery (ACM)
- p. 1-8
Open AccessDomain-Adaptive Mean Teacher for Category-Level Object Pose Estimation
Category-level object pose estimation aims at predicting 6-DoF object poses for previously unseen objects. Current methods mostly rely on ground-truth labels such as object poses and CAD models. However, annotating these labels manually is time-consuming and error-prone in the real-world scenario. Hence, we propose a novel method to solve unsupervised domain adaptation (UDA) for category-level object pose estimation. We adopt a teacher-student framework to utilize both labeled synthetic data and unlabeled real-world data. The student and the teacher are trained to make consistent predictions under different perturbations. Furthermore, we introduce domain adversarial training to bridge the domain gap between synthetic and real-world data. To prevent false feature alignment between domains, we adopt multiple discriminators instead of a single one and perform category-aware alignments. Extensive experiments show that our method achieves state-of-the-art performance on the REAL275 dataset. Through ablation studies, we also demonstrate that our method is not restricted to certain network architecture and can serve as a general UDA method for category-level object pose estimation.
- Open Access
- 23 May 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 61525-61534
VIUNet: Deep Visual–Inertial–UWB Fusion for Indoor UAV Localization
Camera, inertial measurement unit (IMU), and ultra-wideband (UWB) sensors are commonplace solutions to unmanned aerial vehicle (UAV) localization problems. The performance of a localization system can be improved by integrating observations from different sensors. In this paper, we propose a learning-based UAV localization method using the fusion of vision, IMU, and UWB sensors. Our model consists of visual–inertial (VI) and UWB branches. We combine the estimation results of both branches to predict global poses. To evaluate our method, we augment a public VI dataset with UWB simulations and conduct a real-world experiment. The experimental results show that our method provides more robust and accurate results than VI/UWB-only localization.
- 23 May 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 11, 61525-61534
Open AccessVIUNet: Deep Visual–Inertial–UWB Fusion for Indoor UAV Localization
Camera, inertial measurement unit (IMU), and ultra-wideband (UWB) sensors are commonplace solutions to unmanned aerial vehicle (UAV) localization problems. The performance of a localization system can be improved by integrating observations from different sensors. In this paper, we propose a learning-based UAV localization method using the fusion of vision, IMU, and UWB sensors. Our model consists of visual–inertial (VI) and UWB branches. We combine the estimation results of both branches to predict global poses. To evaluate our method, we augment a public VI dataset with UWB simulations and conduct a real-world experiment. The experimental results show that our method provides more robust and accurate results than VI/UWB-only localization.
- Open Access
Absolute Camera Pose Regression Using an RGB-D Dual-Stream Network and Handcrafted Base Poses
Absolute pose regression (APR) for camera localization is a single-shot approach that encodes the information of a 3D scene in an end-to-end neural network. The camera pose result of APR methods can be observed as the linear combination of the base poses. Previous APR methods’ base poses are learned from training data. However, the training data can limit the performance of the methods, which cannot be generalized to cover the entire scene. To solve this issue, we use handcrafted base poses instead of learning-based base poses, which prevents overfitting the camera poses of the training data. Moreover, we use a dual-stream network architecture to process color and depth images separately to get more accurate localization. On the 7 Scenes dataset, the proposed method is among the best in median rotation error, and in median translation error, it outperforms previous APR methods. On a more difficult dataset—Oxford RobotCar dataset, the proposed method achieves notable improvements in median translation and rotation errors compared to the state-of-the-art APR methods.
Open AccessAbsolute Camera Pose Regression Using an RGB-D Dual-Stream Network and Handcrafted Base Poses
Absolute pose regression (APR) for camera localization is a single-shot approach that encodes the information of a 3D scene in an end-to-end neural network. The camera pose result of APR methods can be observed as the linear combination of the base poses. Previous APR methods’ base poses are learned from training data. However, the training data can limit the performance of the methods, which cannot be generalized to cover the entire scene. To solve this issue, we use handcrafted base poses instead of learning-based base poses, which prevents overfitting the camera poses of the training data. Moreover, we use a dual-stream network architecture to process color and depth images separately to get more accurate localization. On the 7 Scenes dataset, the proposed method is among the best in median rotation error, and in median translation error, it outperforms previous APR methods. On a more difficult dataset—Oxford RobotCar dataset, the proposed method achieves notable improvements in median translation and rotation errors compared to the state-of-the-art APR methods.
- 21 February 2021
- Published by Springer Nature
- p. 291-304
Camera Ego-Positioning Using Sensor Fusion and Complementary Method
- 21 February 2021
- Published by Springer Nature
- p. 291-304
Camera Ego-Positioning Using Sensor Fusion and Complementary Method