Xuguang Wang
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Xuguang Wang has a total of 14 co-authors. Here are the top co-authors by number of shared publications:
- Kaizhu Huang
Duke Kunshan University, Kunshan, China
4 shared publications - Xin Liu
Suzhou Institute of Nano-tech and Nano-bionics, Chinese Academy of Sciences, Suzhou, China
4 shared publications - Yangfan Zhou
Suzhou Institute of Nano-tech and Nano-bionics, Chinese Academy of Sciences, Suzhou, China
4 shared publications - Cheng Cheng
Suzhou Institute of Nano-Tech and Nano-Bionics (SINANO), Chinese Academy of Sciences, Suzhou City, Jiangsu Province, China
3 shared publications - Amir Hussain
School of Computing, Edinburgh Napier University, Scotland, UK
2 shared publications Amir Hussain
School of Computing, Edinburgh Napier University, Edinburgh, U.K.
1 shared publication
Latest Publications
- 6 July 2022
- Published by Springer Nature
- p. 763-773
A Non-Local Means Based Multiplicative Denoising Method for Image Processing
- 6 July 2022
- Published by Springer Nature
- p. 763-773
A Non-Local Means Based Multiplicative Denoising Method for Image Processing
- Open Access
FastAdaBelief: Improving Convergence Rate for Belief-based Adaptive Optimizers by Exploiting Strong Convexity
AdaBelief, one of the current best optimizers, demonstrates superior generalization ability compared to the popular Adam algorithm by viewing the exponential moving average of observed gradients. AdaBelief is theoretically appealing in that it has a data-dependent $O(\sqrt{T})$ regret bound when objective functions are convex, where $T$ is a time horizon. It remains however an open problem whether the convergence rate can be further improved without sacrificing its generalization ability. %on how to exploit strong convexity to further improve the convergence rate of AdaBelief. To this end, we make a first attempt in this work and design a novel optimization algorithm called FastAdaBelief that aims to exploit its strong convexity in order to achieve an even faster convergence rate. In particular, by adjusting the step size that better considers strong convexity and prevents fluctuation, our proposed FastAdaBelief demonstrates excellent generalization ability as well as superior convergence. As an important theoretical contribution, we prove that FastAdaBelief attains a data-dependant $O(\log T)$ regret bound, which is substantially lower than AdaBelief. On the empirical side, we validate our theoretical analysis with extensive experiments in both scenarios of strong and non-strong convexity on three popular baseline models. Experimental results are very encouraging: FastAdaBelief converges the quickest in comparison to all mainstream algorithms while maintaining an excellent generalization ability, in cases of both strong or non-strong convexity. FastAdaBelief is thus posited as a new benchmark model for the research community.Open AccessFastAdaBelief: Improving Convergence Rate for Belief-based Adaptive Optimizers by Exploiting Strong Convexity
AdaBelief, one of the current best optimizers, demonstrates superior generalization ability compared to the popular Adam algorithm by viewing the exponential moving average of observed gradients. AdaBelief is theoretically appealing in that it has a data-dependent $O(\sqrt{T})$ regret bound when objective functions are convex, where $T$ is a time horizon. It remains however an open problem whether the convergence rate can be further improved without sacrificing its generalization ability. %on how to exploit strong convexity to further improve the convergence rate of AdaBelief. To this end, we make a first attempt in this work and design a novel optimization algorithm called FastAdaBelief that aims to exploit its strong convexity in order to achieve an even faster convergence rate. In particular, by adjusting the step size that better considers strong convexity and prevents fluctuation, our proposed FastAdaBelief demonstrates excellent generalization ability as well as superior convergence. As an important theoretical contribution, we prove that FastAdaBelief attains a data-dependant $O(\log T)$ regret bound, which is substantially lower than AdaBelief. On the empirical side, we validate our theoretical analysis with extensive experiments in both scenarios of strong and non-strong convexity on three popular baseline models. Experimental results are very encouraging: FastAdaBelief converges the quickest in comparison to all mainstream algorithms while maintaining an excellent generalization ability, in cases of both strong or non-strong convexity. FastAdaBelief is thus posited as a new benchmark model for the research community. - 23 May 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Emerging Topics in Computational Intelligence
- Vol. 7 (2) , 565-577
Towards Faster Training Algorithms Exploiting Bandit Sampling From Convex to Strongly Convex Conditions
The training process for deep learning and pattern recognition normally involves the use of convex and strongly convex optimization algorithms such as AdaBelief and SAdam to handle lots of “uninformative” samples that should be ignored, thus incurring extra calculations. To solve this open problem, we propose to design bandit sampling method to make these algorithms focus on “informative” samples during training process. Our contribution is twofold: first, we propose a convex optimization algorithm with bandit sampling, termed AdaBeliefBS, and prove that it converges faster than its original version; second, we prove that bandit sampling works well for strongly convex algorithms, and propose a generalized SAdam, called SAdamBS, that converges faster than SAdam. Finally, we conduct a series of experiments on various benchmark datasets to verify the fast convergence rate of our proposed algorithms.
- 23 May 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Emerging Topics in Computational Intelligence
- Vol. 7 (2) , 565-577
Towards Faster Training Algorithms Exploiting Bandit Sampling From Convex to Strongly Convex Conditions
The training process for deep learning and pattern recognition normally involves the use of convex and strongly convex optimization algorithms such as AdaBelief and SAdam to handle lots of “uninformative” samples that should be ignored, thus incurring extra calculations. To solve this open problem, we propose to design bandit sampling method to make these algorithms focus on “informative” samples during training process. Our contribution is twofold: first, we propose a convex optimization algorithm with bandit sampling, termed AdaBeliefBS, and prove that it converges faster than its original version; second, we prove that bandit sampling works well for strongly convex algorithms, and propose a generalized SAdam, called SAdamBS, that converges faster than SAdam. Finally, we conduct a series of experiments on various benchmark datasets to verify the fast convergence rate of our proposed algorithms.
- 10 March 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 34 (9) , 6515-6529
FastAdaBelief: Improving Convergence Rate for Belief-Based Adaptive Optimizers by Exploiting Strong Convexity
AdaBelief, one of the current best optimizers, demonstrates superior generalization ability over the popular Adam algorithm by viewing the exponential moving average of observed gradients. AdaBelief is theoretically appealing in which it has a data-dependent O(√T) regret bound when objective functions are convex, where T is a time horizon. It remains, however, an open problem whether the convergence rate can be further improved without sacrificing its generalization ability. To this end, we make the first attempt in this work and design a novel optimization algorithm called FastAdaBelief that aims to exploit its strong convexity in order to achieve an even faster convergence rate. In particular, by adjusting the step size that better considers strong convexity and prevents fluctuation, our proposed FastAdaBelief demonstrates excellent generalization ability and superior convergence. As an important theoretical contribution, we prove that FastAdaBelief attains a data-dependent O(log T) regret bound, which is substantially lower than AdaBelief in strongly convex cases. On the empirical side, we validate our theoretical analysis with extensive experiments in scenarios of strong convexity and nonconvexity using three popular baseline models. Experimental results are very encouraging: FastAdaBelief converges the quickest in comparison to all mainstream algorithms while maintaining an excellent generalization ability, in cases of both strong convexity or nonconvexity. FastAdaBelief is, thus, posited as a new benchmark model for the research community.
- 10 March 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 34 (9) , 6515-6529
FastAdaBelief: Improving Convergence Rate for Belief-Based Adaptive Optimizers by Exploiting Strong Convexity
AdaBelief, one of the current best optimizers, demonstrates superior generalization ability over the popular Adam algorithm by viewing the exponential moving average of observed gradients. AdaBelief is theoretically appealing in which it has a data-dependent O(√T) regret bound when objective functions are convex, where T is a time horizon. It remains, however, an open problem whether the convergence rate can be further improved without sacrificing its generalization ability. To this end, we make the first attempt in this work and design a novel optimization algorithm called FastAdaBelief that aims to exploit its strong convexity in order to achieve an even faster convergence rate. In particular, by adjusting the step size that better considers strong convexity and prevents fluctuation, our proposed FastAdaBelief demonstrates excellent generalization ability and superior convergence. As an important theoretical contribution, we prove that FastAdaBelief attains a data-dependent O(log T) regret bound, which is substantially lower than AdaBelief in strongly convex cases. On the empirical side, we validate our theoretical analysis with extensive experiments in scenarios of strong convexity and nonconvexity using three popular baseline models. Experimental results are very encouraging: FastAdaBelief converges the quickest in comparison to all mainstream algorithms while maintaining an excellent generalization ability, in cases of both strong convexity or nonconvexity. FastAdaBelief is, thus, posited as a new benchmark model for the research community.
- 11 January 2022
- Published by Springer Nature in Cognitive Computation
- Vol. 14 (2) , 764-779
LightAdam: Towards a Fast and Accurate Adaptive Momentum Online Algorithm
- 11 January 2022
- Published by Springer Nature in Cognitive Computation
- Vol. 14 (2) , 764-779
LightAdam: Towards a Fast and Accurate Adaptive Momentum Online Algorithm
- 1 May 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 3944, 1-4
AFD: A feature detection method for outdoor real-time video stitching system
This paper proposes an efficient feature detection solution called AFD (Auto-adapted Feature Detection) based on FAST and LDB applied to outdoor real-time video stitching system. As FAST features are sensitive to illumination and blur, AFD comes out to optimize it. AFD detects features with local and global auto-adapted FAST threshold. Global score compensation is adopted to get good performance under different outdoor situation as well. We use a multiple grid test method to generate more distinctive descriptors. In order to reduce the computing time and increase accuracy, features are extracted in the region of interest (ROI). The feature detection with low complexity and robustness performs well in the real-time stitching system.
- 1 May 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 3944, 1-4
AFD: A feature detection method for outdoor real-time video stitching system
This paper proposes an efficient feature detection solution called AFD (Auto-adapted Feature Detection) based on FAST and LDB applied to outdoor real-time video stitching system. As FAST features are sensitive to illumination and blur, AFD comes out to optimize it. AFD detects features with local and global auto-adapted FAST threshold. Global score compensation is adopted to get good performance under different outdoor situation as well. We use a multiple grid test method to generate more distinctive descriptors. In order to reduce the computing time and increase accuracy, features are extracted in the region of interest (ROI). The feature detection with low complexity and robustness performs well in the real-time stitching system.