Ali Heydari

Optimal control of nonlinear impulsive systems with free impulse instants and the number of impulses is investigated in this study. A scheme based on adaptive dynamic programming is developed, which leads to a feedback (approximate) solution to the defined optimal impulsive control problem. This is done by proposing a learning algorithm for tuning parameters of a function approximator, which, once tuned offline, provides feedback solution on-the-fly. The scheme is shown to handle single and multiple impulsive actuators with a small online computational burden. Afterward, the controller is applied to a challenging problem, namely, the orbital maneuver of spacecraft with the fixed final time using impulsive actuators. The objective is triggering the actuators in a fuel-optimal manner such that the spacecraft transfers to the desired orbit at a prescribed time. It was shown that the proposed scheme leads to simultaneous and feedback path planning and control for the maneuver. The potentials of the scheme are analyzed in different scenarios, including enforcing a shorter final time, selecting different initial states, and incorporating actuator faults.

The problem of codesign or simultaneous synthesis of triggering and input policies is investigated. The policies are synthesized to minimize a cost function incorporating a data transmission cost. This is unlike the common practice of utilizing an a priori -designed stabilizing control policy and designing a triggering policy, which retains stability. Initially, a finite-horizon cost function that penalizes the state, the control input, and each data transmission is selected. The optimal actions comprised of the control input and the triggering decision, under the operation choice of zero-order hold (ZOH), are approximately obtained in a feedback form, using adaptive dynamic programming. Afterward, the design is extended to the case of generalized ZOH, where the last transmitted state information is updated using a possibly imperfect model. Extension to infinite-horizon cost functions is presented. After analysis of convergence and optimality, the performance of the method is demonstrated through some numerical examples.

Adaptive optimal control using value iteration initiated from a stabilizing control policy is theoretically analyzed in terms of stability of the system during the learning stage without ignoring the effects of approximation errors. This analysis includes the system operated using any single/constant resulting control policy and also using an evolving/time-varying control policy. A feature of the presented results is providing estimations of the region of attraction so that if the initial condition is within the region, the whole trajectory will remain inside it and hence, the function approximation results remain valid.

Two approximate solutions for optimal control of switched systems with autonomous subsystems and continuous-time dynamics are presented. The first solution formulates a policy iteration (PI) algorithm for the switched systems with recursive least squares. To reduce the computational burden imposed by the PI algorithm, a second solution, called single loop PI, is presented. Online and concurrent training algorithms are discussed for implementing each solution. At last, effectiveness of the presented algorithms is evaluated through numerical simulations.

Adaptive optimal control using value iteration initiated from a stabilizing control policy is theoretically analyzed in terms of stability of the system during the learning stage without ignoring the effects of approximation errors. This analysis includes the system operated using any single/constant resulting control policy and also using an evolving/time-varying control policy. A feature of the presented results is providing estimations of the \textit{region of attraction} so that if the initial condition is within the region, the whole trajectory will remain inside it and hence, the function approximation results remain valid.

Adaptive optimal control using value iteration initiated from a stabilizing control policy is theoretically analyzed. The analysis is in terms of stability of the system during the learning stage and includes the system controlled by any fixed control policy and also by an evolving policy. A feature of the presented results is finding subsets of the region of attraction. This is done so that if the initial condition belongs to this region, the entire state trajectory remains within the training region. Therefore, the function approximation results remain reliable, as no extrapolation will be conducted.

This paper is focused on bandwidth allocation in nonlinear networked control systems. The objective is optimal triggering/scheduling for transmitting sensor measurements to the controller through a communication network. An algorithm based on approximate dynamic programming is developed for problems with fixed final times and then the result is extended to problems with infinite horizon. Zero-order-hold (ZOH), generalized ZOH, and networks with packet dropouts are the investigated cases. Problems with unknown models are also addressed and a model-free scheme is established for learning the (approximate) optimal solution. The convergence, optimality, and stability of the results are analyzed and, afterwards, some numerical analyses are presented for demonstrating the potentials of the algorithms in practice.

Optimal switching between different topologies in step-down dc-dc voltage converters, with nonideal inductors and capacitors, is investigated in this paper. Challenges including constraint on the inductor current and voltage leakages across the capacitor (due to switching) are incorporated. The objective is generating the desired voltage with low ripples and high robustness toward line and load disturbances. A previously developed tool, which is based on approximate dynamic programming, is adapted for this application. The scheme leads to tuning a parametric function approximator to provide optimal switching in a feedback form. No fixed cycle time is assumed, as the cycle time and the duty ratio will be adjusted on the fly in an optimal fashion. The controller demonstrates good capabilities in controlling the system even under parameter uncertainties. Finally, some modifications on the scheme are conducted to handle optimal switching problems with state jumps at the switching times.