Nhan Huu Pham
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Nhan Huu Pham has a total of 19 co-authors. Here are the top co-authors by number of shared publications:
- Lam M. Nguyen
IBM Research Thomas J. Watson Research Center Yorktown Heights NY USA
6 shared publications - Dzung T. Phan
IBM Research, Thomas J. Watson Research Center, Yorktown Heights, USA
5 shared publications - Quoc Tran-Dinh
Department of Statistics and Operations Research, The University of North Carolina at Chapel Hill, Chapel Hill, USA
5 shared publications Hung M. La
Advanced Robotics and Automation (ARA) Lab, Department of Computer Science and Engineering, University of Nevada, Reno, 89557, USA
2 shared publicationsAnh H. Vo
1 shared publication- Anh Quyen Pham
University of Nevada
1 shared publication
Latest Publications
- Open Access
FedDR -- Randomized Douglas-Rachford Splitting Algorithms for Nonconvex Federated Composite Optimization
We develop two new algorithms, called, FedDR and asyncFedDR, for solving a fundamental nonconvex composite optimization problem in federated learning. Our algorithms rely on a novel combination between a nonconvex Douglas-Rachford splitting method, randomized block-coordinate strategies, and asynchronous implementation. They can also handle convex regularizers. Unlike recent methods in the literature, e.g., FedSplit and FedPD, our algorithms update only a subset of users at each communication round, and possibly in an asynchronous manner, making them more practical. These new algorithms can handle statistical and system heterogeneity, which are the two main challenges in federated learning, while achieving the best known communication complexity. In fact, our new algorithms match the communication complexity lower bound up to a constant factor under standard assumptions. Our numerical experiments illustrate the advantages of our methods over existing algorithms on synthetic and real datasets.
Open AccessFedDR -- Randomized Douglas-Rachford Splitting Algorithms for Nonconvex Federated Composite Optimization
We develop two new algorithms, called, FedDR and asyncFedDR, for solving a fundamental nonconvex composite optimization problem in federated learning. Our algorithms rely on a novel combination between a nonconvex Douglas-Rachford splitting method, randomized block-coordinate strategies, and asynchronous implementation. They can also handle convex regularizers. Unlike recent methods in the literature, e.g., FedSplit and FedPD, our algorithms update only a subset of users at each communication round, and possibly in an asynchronous manner, making them more practical. These new algorithms can handle statistical and system heterogeneity, which are the two main challenges in federated learning, while achieving the best known communication complexity. In fact, our new algorithms match the communication complexity lower bound up to a constant factor under standard assumptions. Our numerical experiments illustrate the advantages of our methods over existing algorithms on synthetic and real datasets.
- 4 January 2021
- Published by Springer Nature in Mathematical Programming
- Vol. 191 (2) , 1005-1071
A hybrid stochastic optimization framework for composite nonconvex optimization
- 4 January 2021
- Published by Springer Nature in Mathematical Programming
- Vol. 191 (2) , 1005-1071
A hybrid stochastic optimization framework for composite nonconvex optimization
- Open Access
A Hybrid Stochastic Optimization Framework for Stochastic Composite Nonconvex Optimization
We introduce a new approach to develop stochastic optimization algorithms for a class of stochastic composite and possibly nonconvex optimization problems. The main idea is to combine two stochastic estimators to create a new hybrid one. We first introduce our hybrid estimator and then investigate its fundamental properties to form a foundational theory for algorithmic development. Next, we apply our theory to develop several variants of stochastic gradient methods to solve both expectation and finite-sum composite optimization problems. Our first algorithm can be viewed as a variant of proximal stochastic gradient methods with a single-loop, but can achieve $\mathcal{O}(\sigma^3\varepsilon^{-1} + \sigma \varepsilon^{-3})$-oracle complexity bound, matching the best-known ones from state-of-the-art double-loop algorithms in the literature, where $\sigma > 0$ is the variance and $\varepsilon$ is a desired accuracy. Then, we consider two different variants of our method: adaptive step-size and restarting schemes that have similar theoretical guarantees as in our first algorithm. We also study two mini-batch variants of the proposed methods. In all cases, we achieve the best-known complexity bounds under standard assumptions. We test our methods on several numerical examples with real datasets and compare them with state-of-the-arts. Our numerical experiments show that the new methods are comparable and, in many cases, outperform their competitors.Open AccessA Hybrid Stochastic Optimization Framework for Stochastic Composite Nonconvex Optimization
We introduce a new approach to develop stochastic optimization algorithms for a class of stochastic composite and possibly nonconvex optimization problems. The main idea is to combine two stochastic estimators to create a new hybrid one. We first introduce our hybrid estimator and then investigate its fundamental properties to form a foundational theory for algorithmic development. Next, we apply our theory to develop several variants of stochastic gradient methods to solve both expectation and finite-sum composite optimization problems. Our first algorithm can be viewed as a variant of proximal stochastic gradient methods with a single-loop, but can achieve $\mathcal{O}(\sigma^3\varepsilon^{-1} + \sigma \varepsilon^{-3})$-oracle complexity bound, matching the best-known ones from state-of-the-art double-loop algorithms in the literature, where $\sigma > 0$ is the variance and $\varepsilon$ is a desired accuracy. Then, we consider two different variants of our method: adaptive step-size and restarting schemes that have similar theoretical guarantees as in our first algorithm. We also study two mini-batch variants of the proposed methods. In all cases, we achieve the best-known complexity bounds under standard assumptions. We test our methods on several numerical examples with real datasets and compare them with state-of-the-arts. Our numerical experiments show that the new methods are comparable and, in many cases, outperform their competitors. - Open Access
Finite-Time Analysis of Stochastic Gradient Descent under Markov Randomness
Motivated by broad applications in reinforcement learning and machine learning, this paper considers the popular stochastic gradient descent (SGD) when the gradients of the underlying objective function are sampled from Markov processes. This Markov sampling leads to the gradient samples being biased and not independent. The existing results for the convergence of SGD under Markov randomness are often established under the assumptions on the boundedness of either the iterates or the gradient samples. Our main focus is to study the finite-time convergence of SGD for different types of objective functions, without requiring these assumptions. We show that SGD converges nearly at the same rate with Markovian gradient samples as with independent gradient samples. The only difference is a logarithmic factor that accounts for the mixing time of the Markov chain.
Open AccessFinite-Time Analysis of Stochastic Gradient Descent under Markov Randomness
Motivated by broad applications in reinforcement learning and machine learning, this paper considers the popular stochastic gradient descent (SGD) when the gradients of the underlying objective function are sampled from Markov processes. This Markov sampling leads to the gradient samples being biased and not independent. The existing results for the convergence of SGD under Markov randomness are often established under the assumptions on the boundedness of either the iterates or the gradient samples. Our main focus is to study the finite-time convergence of SGD for different types of objective functions, without requiring these assumptions. We show that SGD converges nearly at the same rate with Markovian gradient samples as with independent gradient samples. The only difference is a logarithmic factor that accounts for the mixing time of the Markov chain.
- Open Access
Hybrid Stochastic Gradient Descent Algorithms for Stochastic Nonconvex Optimization
We introduce a hybrid stochastic estimator to design stochastic gradient algorithms for solving stochastic optimization problems. Such a hybrid estimator is a convex combination of two existing biased and unbiased estimators and leads to some useful property on its variance. We limit our consideration to a hybrid SARAH-SGD for nonconvex expectation problems. However, our idea can be extended to handle a broader class of estimators in both convex and nonconvex settings. We propose a new single-loop stochastic gradient descent algorithm that can achieve $O(\max\{\sigma^3\varepsilon^{-1},\sigma\varepsilon^{-3}\})$-complexity bound to obtain an $\varepsilon$-stationary point under smoothness and $\sigma^2$-bounded variance assumptions. This complexity is better than $O(\sigma^2\varepsilon^{-4})$ often obtained in state-of-the-art SGDs when $\sigma < O(\varepsilon^{-3})$. We also consider different extensions of our method, including constant and adaptive step-size with single-loop, double-loop, and mini-batch variants. We compare our algorithms with existing methods on several datasets using two nonconvex models.Open AccessHybrid Stochastic Gradient Descent Algorithms for Stochastic Nonconvex Optimization
We introduce a hybrid stochastic estimator to design stochastic gradient algorithms for solving stochastic optimization problems. Such a hybrid estimator is a convex combination of two existing biased and unbiased estimators and leads to some useful property on its variance. We limit our consideration to a hybrid SARAH-SGD for nonconvex expectation problems. However, our idea can be extended to handle a broader class of estimators in both convex and nonconvex settings. We propose a new single-loop stochastic gradient descent algorithm that can achieve $O(\max\{\sigma^3\varepsilon^{-1},\sigma\varepsilon^{-3}\})$-complexity bound to obtain an $\varepsilon$-stationary point under smoothness and $\sigma^2$-bounded variance assumptions. This complexity is better than $O(\sigma^2\varepsilon^{-4})$ often obtained in state-of-the-art SGDs when $\sigma < O(\varepsilon^{-3})$. We also consider different extensions of our method, including constant and adaptive step-size with single-loop, double-loop, and mini-batch variants. We compare our algorithms with existing methods on several datasets using two nonconvex models. - Open Access
ProxSARAH: An Efficient Algorithmic Framework for Stochastic Composite Nonconvex Optimization
We propose a new stochastic first-order algorithmic framework to solve stochastic composite nonconvex optimization problems that covers both finite-sum and expectation settings. Our algorithms rely on the SARAH estimator introduced in (Nguyen et al, 2017) and consist of two steps: a proximal gradient and an averaging step making them different from existing nonconvex proximal-type algorithms. The algorithms only require an average smoothness assumption of the nonconvex objective term and additional bounded variance assumption if applied to expectation problems. They work with both constant and adaptive step-sizes, while allowing single sample and mini-batches. In all these cases, we prove that our algorithms can achieve the best-known complexity bounds. One key step of our methods is new constant and adaptive step-sizes that help to achieve desired complexity bounds while improving practical performance. Our constant step-size is much larger than existing methods including proximal SVRG schemes in the single sample case. We also specify the algorithm to the non-composite case that covers existing state-of-the-arts in terms of complexity bounds. Our update also allows one to trade-off between step-sizes and mini-batch sizes to improve performance. We test the proposed algorithms on two composite nonconvex problems and neural networks using several well-known datasets.
Open AccessProxSARAH: An Efficient Algorithmic Framework for Stochastic Composite Nonconvex Optimization
We propose a new stochastic first-order algorithmic framework to solve stochastic composite nonconvex optimization problems that covers both finite-sum and expectation settings. Our algorithms rely on the SARAH estimator introduced in (Nguyen et al, 2017) and consist of two steps: a proximal gradient and an averaging step making them different from existing nonconvex proximal-type algorithms. The algorithms only require an average smoothness assumption of the nonconvex objective term and additional bounded variance assumption if applied to expectation problems. They work with both constant and adaptive step-sizes, while allowing single sample and mini-batches. In all these cases, we prove that our algorithms can achieve the best-known complexity bounds. One key step of our methods is new constant and adaptive step-sizes that help to achieve desired complexity bounds while improving practical performance. Our constant step-size is much larger than existing methods including proximal SVRG schemes in the single sample case. We also specify the algorithm to the non-composite case that covers existing state-of-the-arts in terms of complexity bounds. Our update also allows one to trade-off between step-sizes and mini-batch sizes to improve performance. We test the proposed algorithms on two composite nonconvex problems and neural networks using several well-known datasets.
- 11 January 2018
- Published by Cambridge University Press (CUP) in Robotica
- Vol. 37 (5) , 947-967
Automated robotic monitoring and inspection of steel structures and bridges
Summary: This paper presents visual and 3D structure inspection for steel structures and bridges using a developed climbing robot. The robot can move freely on a steel surface, carry sensors, collect data and then send to the ground station in real-time for monitoring as well as further processing. Steel surface image stitching and 3D map building are conducted to provide a current condition of the structure. Also, a computer vision-based method is implemented to detect surface defects on stitched images. The effectiveness of the climbing robot's inspection is tested in multiple circumstances to ensure strong steel adhesion and successful data collection. The detection method was also successfully evaluated on various test images, where steel cracks could be automatically identified, without the requirement of some heuristic reasoning.
- 11 January 2018
- Published by Cambridge University Press (CUP) in Robotica
- Vol. 37 (5) , 947-967
Automated robotic monitoring and inspection of steel structures and bridges
Summary: This paper presents visual and 3D structure inspection for steel structures and bridges using a developed climbing robot. The robot can move freely on a steel surface, carry sensors, collect data and then send to the ground station in real-time for monitoring as well as further processing. Steel surface image stitching and 3D map building are conducted to provide a current condition of the structure. Also, a computer vision-based method is implemented to detect surface defects on stitched images. The effectiveness of the climbing robot's inspection is tested in multiple circumstances to ensure strong steel adhesion and successful data collection. The detection method was also successfully evaluated on various test images, where steel cracks could be automatically identified, without the requirement of some heuristic reasoning.
- 1 May 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 3672-3677
Autonomous robotic system using non-destructive evaluation methods for bridge deck inspection
Bridge condition assessment is important to maintain the quality of highway roads for public transport. Bridge deterioration with time is inevitable due to aging material, environmental wear and in some cases, inadequate maintenance. Non-destructive evaluation (NDE) methods are preferred for condition assessment for bridges, concrete buildings, and other civil structures. Some examples of NDE methods are ground penetrating radar (GPR), acoustic emission, and electrical resistivity (ER). NDE methods provide the ability to inspect a structure without causing any damage to the structure in the process. In addition, NDE methods typically cost less than other methods, since they do not require inspection sites to be evacuated prior to inspection, which greatly reduces the cost of safety related issues during the inspection process. In this paper, an autonomous robotic system equipped with three different NDE sensors is presented. The system employs GPR, ER, and a camera for data collection. The system is capable of performing real-time, cost-effective bridge deck inspection, and is comprised of a mechanical robot design and machine learning and pattern recognition methods for automated steel rebar picking to provide realtime condition maps of the corrosive deck environments.
- 1 May 2017
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 3672-3677
Autonomous robotic system using non-destructive evaluation methods for bridge deck inspection
Bridge condition assessment is important to maintain the quality of highway roads for public transport. Bridge deterioration with time is inevitable due to aging material, environmental wear and in some cases, inadequate maintenance. Non-destructive evaluation (NDE) methods are preferred for condition assessment for bridges, concrete buildings, and other civil structures. Some examples of NDE methods are ground penetrating radar (GPR), acoustic emission, and electrical resistivity (ER). NDE methods provide the ability to inspect a structure without causing any damage to the structure in the process. In addition, NDE methods typically cost less than other methods, since they do not require inspection sites to be evacuated prior to inspection, which greatly reduces the cost of safety related issues during the inspection process. In this paper, an autonomous robotic system equipped with three different NDE sensors is presented. The system employs GPR, ER, and a camera for data collection. The system is capable of performing real-time, cost-effective bridge deck inspection, and is comprised of a mechanical robot design and machine learning and pattern recognition methods for automated steel rebar picking to provide realtime condition maps of the corrosive deck environments.
- 1 September 2016
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 556-562
Design and implementation of an autonomous robot for steel bridge inspection
Steel bridges constitute the second most bridges in the U.S. while the number of deficient bridges are growing. Currently, the majority number of current bridge inspections are done manually by inspectors which require significant amount of human resources along with expensive and specialized equipment. Moreover, it is difficult and dangerous for inspectors to inspect large bridges with high structures. In this paper, we propose a new design and implementation of a robotic system that can be used for steel bridge inspection. The robot consists of four magnetic wheels which create adhesion to steel surface. It is able to carry multiple sensors for navigation and mapping. Collected data are sent to the ground station for live monitoring as well as further processing. In addition, magnetic field and range sensors are also integrated to enable robot to move safely on steel surfaces. Results with physical tests on real steel structures are shown to validate the feasibility of robot design.
- 1 September 2016
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 556-562
Design and implementation of an autonomous robot for steel bridge inspection
Steel bridges constitute the second most bridges in the U.S. while the number of deficient bridges are growing. Currently, the majority number of current bridge inspections are done manually by inspectors which require significant amount of human resources along with expensive and specialized equipment. Moreover, it is difficult and dangerous for inspectors to inspect large bridges with high structures. In this paper, we propose a new design and implementation of a robotic system that can be used for steel bridge inspection. The robot consists of four magnetic wheels which create adhesion to steel surface. It is able to carry multiple sensors for navigation and mapping. Collected data are sent to the ground station for live monitoring as well as further processing. In addition, magnetic field and range sensors are also integrated to enable robot to move safely on steel surfaces. Results with physical tests on real steel structures are shown to validate the feasibility of robot design.
- 21 July 2016
- Published by International Association for Automation and Robotics in Construction (IAARC) in Proceedings of the 37th International Symposium on Automation and Robotics in Construction (isarc)
- Vol. 2016 Proce, 141-149
Visual and 3D Mapping for Steel Bridge Inspection Using a Climbing Robot
I.A.A.R.C. - International Association for Automation and Robotics in Construction Civil Engineering
- 21 July 2016
- Published by International Association for Automation and Robotics in Construction (IAARC) in Proceedings of the 37th International Symposium on Automation and Robotics in Construction (isarc)
- Vol. 2016 Proce, 141-149
Visual and 3D Mapping for Steel Bridge Inspection Using a Climbing Robot
I.A.A.R.C. - International Association for Automation and Robotics in Construction Civil Engineering