What Is Logistic Regression?

Logistic regression, also called the logit model, is a regression technique for predicting the probability of a categorical outcome from a set of explanatory variables. In its binary form it models the log-odds of an event as a linear function of the predictors, so that the fitted probability is passed through the logistic function and constrained to the interval between zero and one. This is what separates it from ordinary least squares regression, which can return predicted values outside the range that a probability is allowed to occupy and which assumes an error term that a binary response cannot produce.

The method belongs to the family of generalized linear models formalized by John Nelder and Robert Wedderburn in 1972, where the logit is the canonical link function for a Bernoulli response. Its earlier lineage runs through the logistic growth curve studied by Pierre Verhulst in the 1840s, Joseph Berkson's introduction of the term "logit" in 1944, and David Cox's 1958 treatment of the regression of binary sequences. Because the coefficients have a direct reading as changes in log-odds, and because exponentiating a coefficient yields an odds ratio, the model is used as much for explanation as for prediction.

Model Form and the Logit Link

For a binary response, the model states that the logit of the success probability equals a linear predictor. Each coefficient gives the change in log-odds associated with a one-unit increase in its variable while the others are held fixed, and its exponential gives the multiplicative effect on the odds. Course material such as the Penn State treatment of logistic regression for a binary response works through the link function, the deviance, and the interpretation of fitted odds ratios. The logit is not the only possible link: the probit link, based on the normal cumulative distribution function, gives similar fitted probabilities with differently scaled coefficients, and the complementary log-log link is preferred when the response is asymmetric.

Estimation and Inference

Coefficients are estimated by maximum likelihood rather than in closed form. The log-likelihood is concave for the standard model, and it is maximized numerically by Newton-Raphson, which in this setting reduces to iteratively reweighted least squares, or by gradient methods and quasi-Newton solvers on large problems. Inference relies on Wald tests, likelihood ratio tests, and profile likelihood intervals, while goodness of fit is assessed through deviance residuals and the Hosmer-Lemeshow test. The classical theory assumes the number of observations greatly exceeds the number of predictors, and research on a modern maximum-likelihood theory for high-dimensional logistic regression shows that when the two are comparable the estimator becomes biased and standard significance tests lose their nominal size. Complete separation, where a linear combination of predictors perfectly splits the classes, drives estimates to infinity and is handled by penalized likelihood.

Extensions and Regularization

Multinomial logistic regression generalizes the binary case to unordered categories through a softmax formulation, and ordered logit models, sometimes called proportional odds models, handle ranked outcomes such as severity grades by fitting a common slope across cumulative thresholds. Conditional and mixed-effects variants accommodate matched designs and clustered data. Penalized fitting with ridge or lasso terms controls variance when predictors are many or collinear, and a comparison of lasso and maximum likelihood estimation in logistic regression examines how the two differ in discrimination performance. In machine learning, the same model appears as a single-layer neural network trained with cross-entropy loss, which is the negative log-likelihood under another name.

Applications

Logistic regression is applied across a wide range of fields, including:

• Clinical risk prediction and epidemiological case-control studies
• Credit scoring and insurance underwriting
• Marketing response and churn modeling
• Text classification and spam filtering
• Reliability engineering for pass-fail test data
• Transportation research on discrete travel mode choice