Xiaochun Mai
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Xiaochun Mai has a total of 26 co-authors. Here are the top co-authors by number of shared publications:
- Max Qing-Hu Meng
Southern University of Science and Technology, Shenzhen, China
5 shared publications - Chaoqun Wang
School of Control Science and Engineering Shandong University Jinan China
3 shared publications - Yixuan Yuan
The Chinese University of Hong Kong
3 shared publications - Meilu Zhu
Department of Mechanical Engineering, City University of Hong Kong, Kowloon Tong, Hong Kong
2 shared publications - Xiao Jia
School of Control Science and Engineering, Shandong University, Jinan, China
2 shared publications Chenming Lie
Department of Electronic Engineering, The Chinese University of Hong Kong. N.T., Hong Kong SAR, China
1 shared publication
Latest Publications
- 22 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 27 (15209210) , 9304-9315
Geometric Continuity and Consistency Learning for Self-Supervised Point Cloud Completion
Point cloud completion aims to infer the complete point clouds from incomplete ones. In real-world scenarios, where the paired data is absent, self-supervised methods have emerged as a promising solution. Although existing self-supervised methods perform well at relatively low resolutions, they suffer significant performance degradation at higher resolution primarily because they focus on point cloud reconstruction at patch-level or point-level. In this paper, we propose a self-supervised method based on Geometric Continuity and Consistency Learning (GCCL) at multi-scale level to improve the accuracy of predicting local details and global shapes of point clouds. Specifically, to capture local details, we employ a patch-topoint strategy and a coarse-fine manner for geometric continuity learning. To constrain the global shapes, we construct multiple branches for mutual supervision and utilize class priors to build a memory queue for contrasting current features, enhancing the network focus on geometric consistency learning. We evaluate GCCL on multiple datasets, and the results show that our method outperforms existing self-supervised methods by a 4.4 improvement in CD-$\ell_2$ on the synthetic PCN dataset and can generate more uniformly distributed completion results on realworld datasets
- 22 September 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 27 (15209210) , 9304-9315
Geometric Continuity and Consistency Learning for Self-Supervised Point Cloud Completion
Point cloud completion aims to infer the complete point clouds from incomplete ones. In real-world scenarios, where the paired data is absent, self-supervised methods have emerged as a promising solution. Although existing self-supervised methods perform well at relatively low resolutions, they suffer significant performance degradation at higher resolution primarily because they focus on point cloud reconstruction at patch-level or point-level. In this paper, we propose a self-supervised method based on Geometric Continuity and Consistency Learning (GCCL) at multi-scale level to improve the accuracy of predicting local details and global shapes of point clouds. Specifically, to capture local details, we employ a patch-topoint strategy and a coarse-fine manner for geometric continuity learning. To constrain the global shapes, we construct multiple branches for mutual supervision and utilize class priors to build a memory queue for contrasting current features, enhancing the network focus on geometric consistency learning. We evaluate GCCL on multiple datasets, and the results show that our method outperforms existing self-supervised methods by a 4.4 improvement in CD-$\ell_2$ on the synthetic PCN dataset and can generate more uniformly distributed completion results on realworld datasets
- 4 November 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 22 (15455955) , 8554-8566
ParetoSSL: Pareto Semi-Supervised Learning With Bias-Aware Gradient Preferences for Fruit Yield Estimation
Fruit counting is a fundamental task for fruit yield estimation. Though semi-supervised counting methods have received increased attention in recent years, due to the high data utilization of unlabeled data, they suffer from two limitations. Firstly, difficult weight selection is a limitation, as these methods rely on manually selected fixed weights for both the supervised learning loss and the consistency learning loss, resulting in limited performance. Secondly, biased pseudo-labeling is another limitation, as they may predict biased pseudo-labels that result in small consistency learning losses, leading to training being dominated by supervised learning with large losses. To tackle these two limitations, in this paper, we propose a novel method named ParetoSSL to automatically derive weights of losses from the perspective of multi-task learning. Specifically, ParetoSSL formulates a multi-objective optimization problem for weight derivation by maximizing the similarity between weighted gradients of losses and a customized gradient preference vector, in which, the vector can guide weight derivation. Moreover, to relieve the effect of pseudo-label biases on consistency learning, we propose a bias-aware gradient preference vector. This vector considers gradient biases brought by the pseudo-label biases, which will down-weight the supervised learning loss while high-weighting the consistency learning loss. Meanwhile, to improve the robustness of ParetoSSL, an inequality equation regarding the norm of the gradients of the consistency learning loss is designed to control the range of gradient biases. Extensive experiments are conducted on the Clustered-Fruit dataset and Fruit-2019 dataset to evaluate the effectiveness of ParetoSSL on semi-supervised counting. Experimental results show that our ParetoSSL is superior to state-of-the-art methods. Note to Practitioners —This work is motivated by the emerging need for semi-supervised counting methods in fruit yield estimation. The difficulty of selecting loss weights for training semi-supervised counting algorithms is exacerbated by the pseudo-label bias issue that pseudo-label biases mislead the weight derivation while maximizing the similarity between the weight gradient and gradient preference vectors. The proposed bias-aware gradient preference vectors help users derive loss weights automatically and save the time of choosing loss weights for model fine-tuning. The proposed ParetoSSL is generic as it can be employed as a fruit yield estimation component of crop management support systems, while at the same time being applied to counting frameworks of other objects.
- 4 November 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 22 (15455955) , 8554-8566
ParetoSSL: Pareto Semi-Supervised Learning With Bias-Aware Gradient Preferences for Fruit Yield Estimation
Fruit counting is a fundamental task for fruit yield estimation. Though semi-supervised counting methods have received increased attention in recent years, due to the high data utilization of unlabeled data, they suffer from two limitations. Firstly, difficult weight selection is a limitation, as these methods rely on manually selected fixed weights for both the supervised learning loss and the consistency learning loss, resulting in limited performance. Secondly, biased pseudo-labeling is another limitation, as they may predict biased pseudo-labels that result in small consistency learning losses, leading to training being dominated by supervised learning with large losses. To tackle these two limitations, in this paper, we propose a novel method named ParetoSSL to automatically derive weights of losses from the perspective of multi-task learning. Specifically, ParetoSSL formulates a multi-objective optimization problem for weight derivation by maximizing the similarity between weighted gradients of losses and a customized gradient preference vector, in which, the vector can guide weight derivation. Moreover, to relieve the effect of pseudo-label biases on consistency learning, we propose a bias-aware gradient preference vector. This vector considers gradient biases brought by the pseudo-label biases, which will down-weight the supervised learning loss while high-weighting the consistency learning loss. Meanwhile, to improve the robustness of ParetoSSL, an inequality equation regarding the norm of the gradients of the consistency learning loss is designed to control the range of gradient biases. Extensive experiments are conducted on the Clustered-Fruit dataset and Fruit-2019 dataset to evaluate the effectiveness of ParetoSSL on semi-supervised counting. Experimental results show that our ParetoSSL is superior to state-of-the-art methods. Note to Practitioners —This work is motivated by the emerging need for semi-supervised counting methods in fruit yield estimation. The difficulty of selecting loss weights for training semi-supervised counting algorithms is exacerbated by the pseudo-label bias issue that pseudo-label biases mislead the weight derivation while maximizing the similarity between the weight gradient and gradient preference vectors. The proposed bias-aware gradient preference vectors help users derive loss weights automatically and save the time of choosing loss weights for model fine-tuning. The proposed ParetoSSL is generic as it can be employed as a fruit yield estimation component of crop management support systems, while at the same time being applied to counting frameworks of other objects.
- 29 September 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 21 (4) , 5766-5778
CMCNet: Colorization-Aware Mix-Uncertainty-Adaptive Consistency Network for Semi-Supervised Fruit Counting
Fruit counting is a fundamental and challenging task of automatic fruit yield estimation in the field of intelligent agriculture. In recent years, to relieve the burden of data annotation, semi-supervised counting methods have been studied. Though significant progress has been achieved, the state-of-the-art method estimates the uncertainty of binary segmentation to guide the consistency training of density maps, being prone to deficient uncertainty estimation. Moreover, the method treats pixels with different difficulty equally in each training iteration, being troubled by inflexible consistency training which results in high supervision loss at the beginning of training and even causes network collapse. To alleviate the above limitations, in this paper, we propose a novel semi-supervised counting method CMCNet for fruit counting. CMCNet designs image colorization as an auxiliary task to estimate the uncertainty for density map consistency. Note that this work is the first effort to utilize image colorization for uncertainty estimation in semi-supervised counting. To obtain accurate uncertainty estimation for density map consistency, CMCNet estimates density uncertainty on density maps to depict the difficulty of fruit pixels from the semantic perspective, while using image colorization for constructing colorization uncertainty to measure the difficulty of part of fruit pixels and background pixels from the visual perspective. Then we obtain a comprehensive uncertainty by mixing density uncertainty and colorization uncertainty. Further, we propose a mix-uncertainty-adaptive consistency (MUAC) module for consistency training of density maps. With mix-uncertainty, uncertainty distribution is estimated. By adaptively adjusting the uncertainty threshold, harder pixels will be selected first and easier ones will be added into consistency training gradually. To evaluate the effectiveness of CMCNet, extensive experiments are conducted on two fruit datasets. Experimental results show that our CMCNet is superior to state-of-the-art semi-supervised counting methods. Note to Practitioners —This work is motivated by the emerging need for semi-supervised counting methods in fruit yield estimation. The difficulty of training semi-supervised counting methods with unlabeled images is exacerbated by the noisy supervision issue that pseudo-labels of unlabeled images are noisy. The proposed colorization-aware uncertainty estimation strategy and mix-uncertainty-adaptive consistency approach help the fruit planter sufficiently utilize the information of a large amount of unlabeled data and save the annotation cost in fruit quantity estimation. The proposed method is generic as it can be employed as a fruit yield estimation component of crop management support systems, while at the same time being applied to counting frameworks of other objects.
- 29 September 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 21 (4) , 5766-5778
CMCNet: Colorization-Aware Mix-Uncertainty-Adaptive Consistency Network for Semi-Supervised Fruit Counting
Fruit counting is a fundamental and challenging task of automatic fruit yield estimation in the field of intelligent agriculture. In recent years, to relieve the burden of data annotation, semi-supervised counting methods have been studied. Though significant progress has been achieved, the state-of-the-art method estimates the uncertainty of binary segmentation to guide the consistency training of density maps, being prone to deficient uncertainty estimation. Moreover, the method treats pixels with different difficulty equally in each training iteration, being troubled by inflexible consistency training which results in high supervision loss at the beginning of training and even causes network collapse. To alleviate the above limitations, in this paper, we propose a novel semi-supervised counting method CMCNet for fruit counting. CMCNet designs image colorization as an auxiliary task to estimate the uncertainty for density map consistency. Note that this work is the first effort to utilize image colorization for uncertainty estimation in semi-supervised counting. To obtain accurate uncertainty estimation for density map consistency, CMCNet estimates density uncertainty on density maps to depict the difficulty of fruit pixels from the semantic perspective, while using image colorization for constructing colorization uncertainty to measure the difficulty of part of fruit pixels and background pixels from the visual perspective. Then we obtain a comprehensive uncertainty by mixing density uncertainty and colorization uncertainty. Further, we propose a mix-uncertainty-adaptive consistency (MUAC) module for consistency training of density maps. With mix-uncertainty, uncertainty distribution is estimated. By adaptively adjusting the uncertainty threshold, harder pixels will be selected first and easier ones will be added into consistency training gradually. To evaluate the effectiveness of CMCNet, extensive experiments are conducted on two fruit datasets. Experimental results show that our CMCNet is superior to state-of-the-art semi-supervised counting methods. Note to Practitioners —This work is motivated by the emerging need for semi-supervised counting methods in fruit yield estimation. The difficulty of training semi-supervised counting methods with unlabeled images is exacerbated by the noisy supervision issue that pseudo-labels of unlabeled images are noisy. The proposed colorization-aware uncertainty estimation strategy and mix-uncertainty-adaptive consistency approach help the fruit planter sufficiently utilize the information of a large amount of unlabeled data and save the annotation cost in fruit quantity estimation. The proposed method is generic as it can be employed as a fruit yield estimation component of crop management support systems, while at the same time being applied to counting frameworks of other objects.
- 4 December 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 21 (10) , 11946-11960
A Searching Space Constrained Partial to Full Registration Approach With Applications in Airport Trolley Deployment Robot
For airports with high passenger and luggage flows, a large number of staff members have to be hired to deploy the scattered passenger luggage trolleys. To release humans from the repetitive and laborious job, we develop an autonomous trolley deployment robot to detect, transport and collect the scattered idle trolleys to recycling points. This paper will firstly illustrate the entire collection pipeline of the deployment robot system and then address the key challenge: partial to full point set registration. With the perception framework, the robot can detect the idle trolleys and acquire the pose of the trolleys on the ground, and then capture the trolley from behind, along the same direction for subsequent grasping and manipulation. With RGB-D camera and a segmentation Convolutional Neural Network, the robot can generate a partial surface point cloud of the detected trolley. The resulting point cloud, data and a pre-scanned full trolley point cloud, model, are matched by an implicit pose. To tackle the low accuracy and long computation time issues, a novel searching space-constrained point set registration algorithm is proposed to register the two overlapping point sets. Based on Branch-and-Bound (BnB) mechanism, the error between data and model is iteratively optimized. The constraint of searching space speeds up the global searching of the optimal pose, by pruning the candidate spaces which is impossible to contain the optimal result. To evaluate the performance, an airport trolley segmentation dataset and a point cloud dataset for registration are constructed. Experimental results on the datasets and synthetic dataset show that our method achieves higher accuracy and success rate than the previous methods. The experiments demonstrated in video clips validate the developed system works in real-world applications.
- 4 December 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 21 (10) , 11946-11960
A Searching Space Constrained Partial to Full Registration Approach With Applications in Airport Trolley Deployment Robot
For airports with high passenger and luggage flows, a large number of staff members have to be hired to deploy the scattered passenger luggage trolleys. To release humans from the repetitive and laborious job, we develop an autonomous trolley deployment robot to detect, transport and collect the scattered idle trolleys to recycling points. This paper will firstly illustrate the entire collection pipeline of the deployment robot system and then address the key challenge: partial to full point set registration. With the perception framework, the robot can detect the idle trolleys and acquire the pose of the trolleys on the ground, and then capture the trolley from behind, along the same direction for subsequent grasping and manipulation. With RGB-D camera and a segmentation Convolutional Neural Network, the robot can generate a partial surface point cloud of the detected trolley. The resulting point cloud, data and a pre-scanned full trolley point cloud, model, are matched by an implicit pose. To tackle the low accuracy and long computation time issues, a novel searching space-constrained point set registration algorithm is proposed to register the two overlapping point sets. Based on Branch-and-Bound (BnB) mechanism, the error between data and model is iteratively optimized. The constraint of searching space speeds up the global searching of the optimal pose, by pruning the candidate spaces which is impossible to contain the optimal result. To evaluate the performance, an airport trolley segmentation dataset and a point cloud dataset for registration are constructed. Experimental results on the datasets and synthetic dataset show that our method achieves higher accuracy and success rate than the previous methods. The experiments demonstrated in video clips validate the developed system works in real-world applications.
- 14 September 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 21 (10) , 11740-11747
Improving Dense Mapping for Mobile Robots in Dynamic Environments Based on Semantic Information
In recent decades, semantic mapping has become a hot topic benefited from the maturity of visual simultaneous localization and mapping (visual SLAM) and the success of deep learning. Despite the impressive performance of the current state-of-the-art systems, semantic mapping in dynamic environments is still a challenging task. To address this problem, we propose a framework that fuses geometric information, semantic information, and human activity into a 3D dense map. The accuracy of the map is guaranteed by the reliable camera trajectory estimation and the static pixels used for 3D reconstruction. With the proposed framework, we achieve two objectives. On the one hand, we accurately reconstruct the environment from both geometric and semantic perspectives. On the other hand, we record human activity by tracking the human trajectory during the mapping period. We conduct both qualitative and quantitative experiments on the public TUM dataset. The experimental results demonstrate the feasibility and effectiveness of the proposed framework.
- 14 September 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 21 (10) , 11740-11747
Improving Dense Mapping for Mobile Robots in Dynamic Environments Based on Semantic Information
In recent decades, semantic mapping has become a hot topic benefited from the maturity of visual simultaneous localization and mapping (visual SLAM) and the success of deep learning. Despite the impressive performance of the current state-of-the-art systems, semantic mapping in dynamic environments is still a challenging task. To address this problem, we propose a framework that fuses geometric information, semantic information, and human activity into a 3D dense map. The accuracy of the map is guaranteed by the reliable camera trajectory estimation and the static pixels used for 3D reconstruction. With the proposed framework, we achieve two objectives. On the one hand, we accurately reconstruct the environment from both geometric and semantic perspectives. On the other hand, we record human activity by tracking the human trajectory during the mapping period. We conduct both qualitative and quantitative experiments on the public TUM dataset. The experimental results demonstrate the feasibility and effectiveness of the proposed framework.
- 7 September 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 21 (10) , 11844-11857
Coarse-to-Fine Visual Object Catching Strategy Applied in Autonomous Airport Baggage Trolley Collection
In this study, a systematic solution for mobile manipulation with the application of autonomous trolley collection in the airport is presented. The developed system features a novel integrated hardware and software system, incorporating a specially designed manipulator capable of tightly catching the trolley. Particularly, we innovatively present a supervisory control guided coarse-to-fine visual trolley catching strategy in large-scale airport environments. At the coarsest level, an image-based visual servoing method is utilized to approach the trolley from far to near, where a learning-based object detection method is adopted to identify trolleys in the environment in real-time. At the finest level, we present an accurate point cloud registration based object pose estimation method to promote the object manipulation with a position-based visual servoing controller. The tightly integrated system and the subsystems are evaluated through conducting experiments in the trolley collection task and comparing the key module with the state of the art. Extensive experiments demonstrate that the proposed framework can find and fetch the trolley robustly. Notably, the proposed object detection and pose estimation methods exhibit higher accuracy, providing credible perceptual approaches for mobile manipulation. Moreover, the proposed framework can be easily extended to other mobile manipulation tasks for object search and retrieval.
- 7 September 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Sensors Journal
- Vol. 21 (10) , 11844-11857
Coarse-to-Fine Visual Object Catching Strategy Applied in Autonomous Airport Baggage Trolley Collection
In this study, a systematic solution for mobile manipulation with the application of autonomous trolley collection in the airport is presented. The developed system features a novel integrated hardware and software system, incorporating a specially designed manipulator capable of tightly catching the trolley. Particularly, we innovatively present a supervisory control guided coarse-to-fine visual trolley catching strategy in large-scale airport environments. At the coarsest level, an image-based visual servoing method is utilized to approach the trolley from far to near, where a learning-based object detection method is adopted to identify trolleys in the environment in real-time. At the finest level, we present an accurate point cloud registration based object pose estimation method to promote the object manipulation with a position-based visual servoing controller. The tightly integrated system and the subsystems are evaluated through conducting experiments in the trolley collection task and comparing the key module with the state of the art. Extensive experiments demonstrate that the proposed framework can find and fetch the trolley robustly. Notably, the proposed object detection and pose estimation methods exhibit higher accuracy, providing credible perceptual approaches for mobile manipulation. Moreover, the proposed framework can be easily extended to other mobile manipulation tasks for object search and retrieval.
- 30 January 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 17 (15455955) , 1-15
Faster R-CNN With Classifier Fusion for Automatic Detection of Small Fruits
Fruit detection is a fundamental task for automatic yield estimation. The goal is to detect all the fruits in images. The-state of the art of fruit detection algorithm, Faster R-CNN, shows a lack of detection advantage on small fruits. One of the reasons is only that single-level features and a classifier are used for localization of proposal candidates. In this article, we propose to incorporate a multiple classifier fusion strategy into a Faster R-CNN network for small fruit detection. We utilize features from three different levels to learn three classifiers for objectness classification in the stage of proposal localization. Probabilities from classifiers are combined by a simple convolutional layer to generate final objectness classification for proposal candidates. During training, in order to train a model with strong generalization capability, we propose to use correlation coefficients to measure the diversity of multiple classifiers. A novel loss function with classifier correlation is introduced to train the region proposal network. We evaluate the proposed model on two data sets of small fruits. Extensive experiments show that the proposed model outperforms the state-of-the-art detectors for fruit detection.
- 30 January 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. 17 (15455955) , 1-15
Faster R-CNN With Classifier Fusion for Automatic Detection of Small Fruits
Fruit detection is a fundamental task for automatic yield estimation. The goal is to detect all the fruits in images. The-state of the art of fruit detection algorithm, Faster R-CNN, shows a lack of detection advantage on small fruits. One of the reasons is only that single-level features and a classifier are used for localization of proposal candidates. In this article, we propose to incorporate a multiple classifier fusion strategy into a Faster R-CNN network for small fruit detection. We utilize features from three different levels to learn three classifiers for objectness classification in the stage of proposal localization. Probabilities from classifiers are combined by a simple convolutional layer to generate final objectness classification for proposal candidates. During training, in order to train a model with strong generalization capability, we propose to use correlation coefficients to measure the diversity of multiple classifiers. A novel loss function with classifier correlation is introduced to train the region proposal network. We evaluate the proposed model on two data sets of small fruits. Extensive experiments show that the proposed model outperforms the state-of-the-art detectors for fruit detection.
- 30 January 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. PP (15455955) , 1-15
Automatic Polyp Recognition in Colonoscopy Images Using Deep Learning and Two-Stage Pyramidal Feature Prediction
Polyp recognition in colonoscopy images is crucial for early colorectal cancer detection and treatment. However, the current manual review requires undivided concentration of the gastroenterologist and is prone to diagnostic errors. In this article, we present an effective, two-stage approach called
- 30 January 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automation Science and Engineering
- Vol. PP (15455955) , 1-15
Automatic Polyp Recognition in Colonoscopy Images Using Deep Learning and Two-Stage Pyramidal Feature Prediction
Polyp recognition in colonoscopy images is crucial for early colorectal cancer detection and treatment. However, the current manual review requires undivided concentration of the gastroenterologist and is prone to diagnostic errors. In this article, we present an effective, two-stage approach called