What Is Numerical Stability?

Numerical stability is the property of a computational algorithm by which small perturbations in the input data or arithmetic rounding errors do not grow unboundedly as the computation proceeds. An algorithm is called numerically stable if the computed result is close to the exact solution of a slightly perturbed version of the original problem; it is called unstable if rounding errors accumulate and amplify in ways that render the output meaningless. The concept is central to scientific computing, numerical linear algebra, and the design of solvers for differential equations, because all practical computers represent numbers in finite-precision floating-point format and therefore introduce rounding at every arithmetic operation.

The field draws on classical numerical analysis, including foundational backward error analysis associated with J.H. Wilkinson, and on the IEEE 754 floating-point standard that governs how modern processors represent and round numbers.

Sources of Numerical Instability

Instability arises from two distinct sources that interact in practice. The first is problem ill-conditioning: the mathematical problem itself may be sensitive to input perturbations, in which case even a perfectly stable algorithm cannot produce accurate output. The condition number of a matrix or function measures this sensitivity. For a linear system Ax = b, the condition number of A (defined as the ratio of the largest to smallest singular value) bounds how much relative error in b can amplify into relative error in x. A condition number of 10^10 implies that roughly ten digits of input precision are consumed by amplification, leaving only six accurate output digits in double precision. As UC Berkeley's EECS lecture notes on floating-point arithmetic and error analysis explain, the total error in a computation equals approximately the backward error (the smallest perturbation to the input that would make the computed result exact) multiplied by the condition number.

The second source is algorithm instability: certain computational procedures, even applied to well-conditioned problems, amplify rounding errors through their internal structure. Gaussian elimination without pivoting is a canonical example, since it can produce growth factors that cause rounding error to overwhelm the true solution. Partial pivoting, which reorders rows to keep multipliers small, converts an unstable algorithm into a stable one for almost all practical inputs. Similarly, the explicit evaluation of a recurrence in the wrong direction can transform a stable mathematical sequence into a diverging numerical one.

Stability in ODE and PDE Solvers

Time-stepping algorithms for ordinary and partial differential equations carry a distinct form of stability constraint: they must not amplify the solution or its errors from one time step to the next. For linear systems, the region of absolute stability of a method defines the set of step size and eigenvalue products for which the numerical solution remains bounded. Explicit schemes such as the forward Euler method have finite stability regions, requiring the product of the time step and the largest system eigenvalue to remain below a threshold. Implicit schemes such as backward Euler have unbounded stability regions (A-stability), making them preferred for stiff problems where eigenvalues span many decades of magnitude. For hyperbolic PDE solvers such as finite difference schemes for the wave equation, the Courant-Friedrichs-Lewy (CFL) condition requires that the numerical domain of dependence contain the physical domain of dependence; as SimScale's CFL condition reference explains, this translates in practice to requiring the Courant number to remain at or below 1 for explicit schemes.

Conditioning and Round-Off Error

The IEEE 754 standard specifies that each floating-point operation be computed with a relative error no larger than one-half unit in the last place (0.5 ulp), guaranteeing that individual operations are as accurate as the binary representation allows. Accumulated round-off across a sequence of operations can still degrade results substantially, particularly when catastrophic cancellation occurs: the subtraction of two nearly equal quantities eliminates significant digits and leaves only the rounding noise in the difference. Numerically stable algorithms avoid or mitigate cancellation by algebraic rearrangement, such as using compensated summation (the Kahan algorithm) in place of naive sequential addition for large float arrays, as documented in foundational references on numerical stability analysis from the University of Saskatchewan CMPT 898 course notes.

Applications

Numerical stability has applications in a range of fields, including:

• Linear algebra solvers for structural analysis, circuit simulation, and data fitting
• Time integration of stiff chemical kinetics and control system differential equations
• Signal processing filters and fast Fourier transform implementations
• Machine learning training algorithms where gradient accumulation must avoid precision loss
• Financial risk modeling where small errors in matrix factorizations can alter hedging decisions