What Is Maximum Likelihood Estimation?

Maximum likelihood estimation is a method of point estimation that selects the parameter values making the observed data most probable under a specified probability model. Given a set of observations and a parametric family of distributions, the maximum likelihood estimator (MLE) is the parameter vector that maximizes the likelihood function, which expresses the probability of the data as a function of the parameters. The method originates in the work of Ronald A. Fisher, who formalized the likelihood principle in the 1920s and established its asymptotic optimality properties. Maximum likelihood estimation draws on mathematical statistics, probability theory, and numerical optimization, and it serves as the default estimation method across engineering, physics, econometrics, and the life sciences.

The likelihood function is typically expressed as the product of individual observation probabilities, and maximization is carried out on its logarithm, the log-likelihood, for computational convenience. Closed-form solutions exist for many standard models, including the Gaussian, exponential, and Poisson families. For models without closed-form solutions, iterative algorithms such as gradient ascent, Newton-Raphson, and the expectation-maximization (EM) algorithm are used. The EM algorithm is especially important for models with latent variables or missing data, where it alternates between computing the expected complete-data log-likelihood and maximizing it over the parameters.

The Likelihood Function and Fisher Information

The likelihood function encodes all the information the observed sample provides about the unknown parameters. Its curvature near the maximum, measured by the Fisher information matrix, determines the asymptotic precision of the MLE: steeper curvature corresponds to more informative data and tighter estimates. The Cramér-Rao lower bound, derived from the Fisher information, states that no unbiased estimator can have a covariance matrix smaller than the inverse of the Fisher information, and the MLE achieves this bound asymptotically as the sample size grows. The Fisher information is also central to optimal experimental design, guiding the choice of observation schedules that maximize the precision of the resulting estimates. IEEE publications on maximum likelihood parameter estimation of noisy data, including work on parameter estimation of superimposed exponential signals, illustrate how the Fisher information matrix is used to predict estimation accuracy before and after data collection.

Estimation in Tracking and Dynamic Systems

In target tracking and state estimation, maximum likelihood estimation is applied to identify the parameters of motion models and sensor noise covariances from recorded trajectory data. Given a state-space model with Gaussian noise, the log-likelihood of the observation sequence can be computed recursively using the Kalman filter, and the parameter estimates are obtained by maximizing this expression over the model parameters. This approach is known as prediction-error minimization or the innovations likelihood method. For nonlinear systems, extended or unscented Kalman filters provide approximate likelihood evaluations. Parameter identification in continuous-time stochastic systems using this framework is addressed in IEEE work on maximum likelihood estimation from incomplete data via sensitivity equations, which derives recursive filters for computing the likelihood gradient with respect to system parameters.

Asymptotic Properties and Model Selection

Under regularity conditions, the MLE is consistent, asymptotically normal, and asymptotically efficient: as the sample size increases, the estimates converge to the true parameter values and their distribution approaches a Gaussian centered at the truth with covariance equal to the inverse Fisher information. These properties make the MLE a reliable workhorse when data are plentiful, but the estimates can be badly behaved in small samples or when the model is misspecified. The Akaike information criterion (AIC) and the Bayesian information criterion (BIC) use the maximized log-likelihood to compare competing models, penalizing complexity to avoid overfitting. An introduction to maximum likelihood and its connections to information theory is presented in lectures hosted on the IEEE Technology Navigator page for maximum likelihood estimation.

Applications

Maximum likelihood estimation has applications in a wide range of quantitative fields, including:

• Radar and sonar target localization, where MLE of range, bearing, and Doppler parameters is derived from matched-filter outputs
• Econometrics and financial modeling, where MLE fits GARCH and ARIMA models to time series of returns and volatility
• Biostatistics and clinical trials, where MLE of survival distributions and hazard ratios informs treatment effect estimates
• Communications channel estimation, where MLE recovers channel coefficients from pilot symbol observations
• Genome-wide association studies, where MLE identifies allele frequency parameters associated with disease phenotypes