What Is Analysis of Variance?

Analysis of variance (ANOVA) is a statistical method for partitioning the total variability observed in a dataset into components attributable to specific sources, such as experimental treatments, blocking factors, or their interactions, and a residual component attributable to random error. By comparing the variance explained by a factor against the unexplained variance, ANOVA tests whether the means of two or more groups differ by more than would be expected by chance alone. The technique was developed by Ronald A. Fisher in the 1920s and has since become one of the most widely used tools in experimental science and engineering.

ANOVA generalizes the two-sample t-test to situations with three or more groups, addressing the problem that performing multiple pairwise t-tests inflates the probability of a Type I error. Rather than comparing all pairs separately, ANOVA conducts a single omnibus test on the full set of means simultaneously. It assumes that observations within each group are independent, drawn from normally distributed populations with equal variances, an assumption known as homoscedasticity. The NIST/SEMATECH e-Handbook of Statistical Methods provides a detailed treatment of these assumptions and the diagnostics used to verify them.

One-Way and Factorial Designs

One-way ANOVA examines the effect of a single categorical factor on a continuous response variable. The total sum of squares in the data is decomposed into the sum of squares attributable to the factor (the treatment sum of squares) and the sum of squares attributable to random error within groups. With k groups and N total observations, the treatment sum of squares has k - 1 degrees of freedom and the error sum of squares has N - k degrees of freedom.

Factorial ANOVA extends this framework to experiments with two or more factors applied in all combinations. A two-factor factorial design separates main effects for each factor from the interaction effect, which captures whether the influence of one factor depends on the level of the other. Factorial designs are more informative and statistically efficient than a sequence of one-factor experiments because they reveal interactions that one-factor studies cannot detect. The NIST one-way ANOVA overview explains the model structure and assumptions underlying these designs, and the handbook's section on factorial experiments extends the framework to multi-factor studies.

The F-test and Variance Partitioning

The test statistic in ANOVA is the F-ratio, the ratio of the mean square for the factor of interest to the mean square error. The mean square for each source is its sum of squares divided by its degrees of freedom. Under the null hypothesis that all group means are equal, the F-ratio follows an F-distribution with degrees of freedom corresponding to the treatment and error terms. A large F-ratio indicates that between-group variation substantially exceeds within-group variation, providing evidence against the null hypothesis.

The Penn State STAT 462 ANOVA table explanation describes how the F-ratio is computed and interpreted in the context of regression and analysis of variance, connecting the partitioning logic to the broader framework of linear models. The practical computation of ANOVA tables is routinely handled by statistical software packages such as R, SAS, and Python's statsmodels, which also compute p-values and confidence intervals for the estimated effects.

Multiple Comparisons and Post-hoc Tests

A significant F-test confirms that not all group means are equal but does not identify which pairs differ. Post-hoc multiple comparison procedures, including Tukey's honest significant difference, Bonferroni correction, and the Scheffé method, address this by controlling the family-wise error rate across all pairwise comparisons. These procedures adjust individual comparison thresholds to account for the number of tests being performed simultaneously, preventing the accumulation of Type I error that would occur without correction.

Applications

Analysis of variance is used across engineering, natural science, and social science wherever designed experiments compare group means, including:

• Quality control and process engineering, to identify which manufacturing factors most affect product characteristics
• Signal processing research, to compare the performance of competing estimation algorithms across multiple test conditions
• Clinical and biomedical studies, to evaluate treatment effects across patient groups
• Agricultural and environmental science, to assess the effects of soil treatment, climate, or exposure conditions on measured outcomes
• Machine learning evaluation, to determine whether observed performance differences between models exceed chance variation