Recursive estimation

What Is Recursive Estimation?

Recursive estimation is a class of statistical inference methods that update an estimate of an unknown quantity each time a new observation becomes available, without reprocessing previous data. Rather than accumulating all measurements and solving a batch problem, a recursive estimator maintains a compact summary of past information and incorporates each new measurement through a prescribed update rule. This approach is computationally efficient, well-suited to real-time operation, and capable of tracking quantities that change over time as more sensor data arrives.

The discipline draws on probability theory, linear algebra, and control theory. Recursive estimators appeared in practical form in the 1960s alongside the development of digital computers capable of real-time arithmetic, with Kalman's 1960 paper on optimal linear filtering providing a rigorous probabilistic foundation. Applications span virtually every field that involves sensors, feedback, or dynamic systems, and the underlying mathematics unifies methods that were historically developed independently in statistics, signal processing, and automatic control.

Recursive Least Squares

The recursive least squares (RLS) algorithm is the sequential counterpart of the ordinary least-squares criterion. Given a linear model relating parameters to observations, the batch solution minimizes the sum of squared residuals over all available data. The RLS update rule computes this same solution incrementally: when a new observation arrives, it adjusts the previous parameter estimate and updates an associated gain matrix without visiting earlier data. A forgetting factor, typically a scalar slightly less than one, exponentially de-emphasizes older measurements, enabling the algorithm to track slowly drifting parameters. The UC Berkeley lecture notes on recursive least squares give a self-contained derivation of the update equations and their relationship to the matrix inversion lemma.

Kalman Filtering

The Kalman filter extends recursive estimation to state-space models in which the unknown quantity evolves according to a known dynamic equation and is observed through a noise-corrupted measurement equation. The filter alternates between a prediction step, which propagates the state estimate forward using the dynamic model, and an update step, which corrects the prediction using the latest measurement weighted by the Kalman gain. The gain balances the uncertainty in the prediction against the uncertainty in the measurement: a noisy sensor produces a small gain and the prediction dominates; a precise sensor produces a large gain and the measurement dominates. Work compiled at IEEE Xplore on Kalman filtering and recursive least squares demonstrates that the Kalman filter is the minimum-variance linear unbiased estimator for linear Gaussian models, and that the RLS algorithm is a special case of the Kalman filter applied to a static parameter with a random-walk prior.

Adaptive and Nonlinear Extensions

Standard Kalman filtering assumes linearity, but practical systems are often nonlinear. The extended Kalman filter (EKF) linearizes the dynamic and measurement models around the current estimate using first-order Taylor expansions, applying the standard Kalman update to the approximated linear model. The unscented Kalman filter (UKF) instead propagates a set of carefully chosen sigma points through the true nonlinear functions, capturing mean and covariance to second-order accuracy without explicit Jacobian computation. Particle filters represent the state distribution as a weighted set of samples and are applicable to highly nonlinear, non-Gaussian problems. A detailed treatment of adaptive Kalman filtering developed from RLS principles shows how the two families of methods share a unified theoretical core despite their different origins.

Applications

Recursive estimation has applications in a wide range of disciplines, including: