What Is Estimation?

Estimation is the mathematical discipline of inferring unknown quantities from noisy, incomplete, or indirect observations. In engineering and signal processing, estimation theory provides the tools to extract parameters, states, or signals from measurements corrupted by uncertainty. Whether tracking a missile trajectory, reconstructing a medical image, or calibrating a sensor network, virtually every system that observes the physical world depends on principled estimation.

Parameter Estimation

Parameter estimation seeks fixed but unknown values, such as the frequency of a sinusoid buried in noise or the resistance of a circuit element measured with a noisy voltmeter. Maximum likelihood estimation (MLE) finds the parameter value that makes the observed data most probable under the assumed statistical model. Under regularity conditions, MLE estimates are asymptotically unbiased and achieve the lowest possible variance.

That minimum variance is quantified by the Cramer-Rao lower bound (CRLB), derived from the Fisher information matrix. An estimator that achieves the CRLB is called efficient. The bound is indispensable for system designers: it tells engineers whether their estimator still has room to improve or has already reached the theoretical limit. Methods such as least-squares, method-of-moments, and Bayesian MAP estimation each make different assumptions about the statistical model and the prior information available.

State Estimation and Kalman Filtering

When the unknown quantity evolves over time according to a dynamic model, parameter estimation gives way to state estimation. The Kalman filter, developed by Rudolf Kalman in 1960, is the optimal linear estimator for systems with Gaussian noise. It propagates a probability distribution over the hidden state by alternating between a predict step, which advances the distribution through the system dynamics, and an update step, which fuses new measurements to sharpen the estimate.

The Kalman filter's theoretical foundation and recursive structure made it tractable for real-time computation, and it became foundational in aerospace navigation. For nonlinear systems, the extended Kalman filter (EKF) linearizes around the current estimate, while the unscented Kalman filter (UKF) propagates a set of deterministically chosen sigma points to capture higher-order statistics without explicit Jacobians.

Particle filters handle fully nonlinear, non-Gaussian problems by representing the posterior distribution as a weighted ensemble of random samples. They are computationally intensive but applicable to problems where EKF and UKF diverge.

Bayesian and Robust Estimation

Bayesian estimation treats unknown parameters as random variables with prior distributions encoding prior knowledge. The posterior distribution, obtained via Bayes' theorem, captures all available information after observing data. The minimum mean-squared-error (MMSE) estimator, the posterior mean, minimizes expected squared error but requires integration over the posterior, which is analytically tractable only in special cases.

Robust estimation addresses scenarios where the assumed noise model is incorrect. Estimators such as the M-estimator, median-based methods, and total least-squares reduce sensitivity to outliers and model mismatch. NIST guidance on measurement uncertainty formalizes how uncertainty propagates through estimation chains in metrology.

Spectral estimation, a specialized branch, infers the power distribution of a signal across frequencies from finite, noisy time-series data. Techniques range from the classical periodogram to parametric autoregressive models and modern subspace methods such as MUSIC and ESPRIT, which exploit the eigenstructure of the data covariance matrix to resolve closely spaced spectral lines. Research on compressed sensing extended spectral estimation to scenarios where far fewer samples than the Nyquist rate are available.

Applications

• Inertial navigation systems fuse accelerometer and gyroscope data with GPS corrections using Kalman filtering to maintain position and attitude estimates.
• Radar and sonar systems use maximum likelihood and subspace methods to estimate target range, velocity, and bearing.
• Medical imaging techniques including MRI reconstruction and PET apply regularized estimation to recover anatomical structure from limited projections.
• Power grid monitoring uses state estimation to infer voltages and currents at unobserved buses from available meter readings.
• Wireless communications rely on channel estimation to measure multipath propagation coefficients and enable coherent equalization and MIMO precoding.
• Robotics and autonomous vehicles combine LIDAR, camera, and wheel odometry through probabilistic state estimators for simultaneous localization and mapping (SLAM).