What Is Optimal Control?

Optimal control is a branch of mathematical control theory that seeks control laws or input trajectories for a dynamic system that minimize (or maximize) a specified performance criterion while satisfying the system's equations of motion and any constraints on states or inputs. Rather than designing a controller that merely stabilizes a system, optimal control engineers a controller that does so in the best possible way according to a quantitative objective, such as minimum energy, minimum time, or minimum deviation from a reference trajectory.

The field emerged from the calculus of variations and was formalized in the late 1950s and 1960s through contributions including Bellman's dynamic programming and Pontryagin's minimum principle. It now forms a cornerstone of aerospace guidance, robotics, process control, and economics.

Pontryagin Minimum Principle and Bang-Bang Control

The Pontryagin minimum principle provides necessary conditions for optimality in continuous-time optimal control problems. It introduces a costate (adjoint) vector that evolves according to an auxiliary differential equation; the optimal control at each instant minimizes the Hamiltonian, a scalar function of state, costate, and control. For problems where the control enters the Hamiltonian linearly, the minimizing control takes values only at the boundaries of the allowable control set, a phenomenon called bang-bang control. Time-optimal problems for linear systems are canonical examples: the optimal thrust or steering input switches between maximum and minimum values a finite number of times. Pontryagin's original development is presented in SIAM publications that remain the standard reference.

Linear Quadratic Regulator

The linear quadratic regulator (LQR) is the most widely applied result in optimal control. For a linear time-invariant system, minimizing an infinite-horizon cost that penalizes both state deviation and control effort yields a time-invariant state-feedback gain matrix. This gain is computed from the solution to the algebraic Riccati equation, which has well-established numerical algorithms. LQR balances performance against actuator usage in a principled way: increasing the weight on the state penalty drives more aggressive correction; increasing the control penalty forces gentler inputs.

The dual problem, optimal state estimation, yields the Kalman filter. Together, LQR and the Kalman filter form the linear quadratic Gaussian (LQG) regulator, the optimal controller for linear systems with Gaussian noise. IEEE Transactions on Automatic Control has published the foundational and applied work on LQR, Riccati equations, and their robustness properties over decades.

Model Predictive Control

Model predictive control (MPC) solves a finite-horizon optimal control problem at each sampling instant, applies the first step of the resulting input sequence, and then re-solves the problem at the next step using updated state measurements. This receding-horizon approach handles constraints on states and inputs explicitly, something that LQR cannot do directly, and can accommodate nonlinear dynamics by solving nonlinear programs online.

MPC is widespread in chemical process control, building energy management, and automotive powertrain systems. Modern hardware and solver advances have extended its applicability to fast systems including automotive active suspensions and power converters. Research on robust and stochastic MPC addresses how to guarantee constraint satisfaction despite model uncertainty and disturbances.

Infinite Horizon Problems

Infinite-horizon optimal control considers performance over an unbounded time interval. When the cost is discounted or the system reaches a steady state, the optimal policy becomes time-invariant, simplifying both analysis and implementation. Dynamic programming characterizes the value function through the Hamilton-Jacobi-Bellman (HJB) equation. Solving the HJB equation exactly is feasible only for low-dimensional systems; approximate dynamic programming and reinforcement learning are used for higher-dimensional problems.

Applications

• Aerospace trajectory optimization, including fuel-optimal orbital maneuvers and entry guidance for reusable launch vehicles
• Autonomous vehicle motion planning, where MPC generates collision-free trajectories satisfying speed and steering constraints
• Industrial process control in chemical plants, where MPC manages multi-variable interactions and enforces safety limits on temperature and pressure
• Robotic arm control using LQR-based regulators for smooth, minimum-energy manipulation trajectories
• Power grid frequency regulation, where optimal dispatch of generating units balances cost against stability requirements
• Medical device control, including closed-loop insulin delivery systems that regulate blood glucose using model-based prediction