What Are Roundoff Errors?

Roundoff errors are the differences between the exact result of a mathematical operation and the result actually produced when that operation is carried out in finite-precision arithmetic. Because digital computers represent real numbers with a finite number of bits, almost every stored value and almost every arithmetic result is approximated by the nearest representable number, introducing a small residual that can grow as computations proceed. The study of roundoff belongs to numerical analysis and underlies every claim about accuracy, stability, and reliability of scientific and engineering software.

The dominant representation in modern hardware is binary floating-point arithmetic governed by the IEEE Standard 754 for Floating-Point Arithmetic, first issued in 1985 and revised in 2008 and 2019. IEEE 754 specifies storage formats such as binary32 and binary64, rounding modes including round-to-nearest-even, and a requirement that the basic operations of addition, subtraction, multiplication, division, and square root be correctly rounded. These rules make roundoff predictable and analyzable rather than an artifact of a particular processor.

Sources in Floating-Point Representation

A floating-point number is stored as a sign, an exponent, and a significand of fixed length. Any real number that does not fall on the discrete grid of representable values must be rounded, producing a relative error bounded by the unit roundoff, often written u and equal to one half of the machine epsilon for round-to-nearest. For binary64, u is roughly 1.11 times 10 to the negative 16, which sets the best accuracy an unscaled scalar computation can attain. Subnormal numbers, signed zeros, and infinities extend the representable range but do not change this basic grain. A concise treatment of representation and error bounds is given in the Berkeley EECS course notes on floating-point arithmetic and error analysis.

Error Propagation and Cancellation

Individual roundoff errors are small, but they combine in ways that depend on the algorithm and the data. Adding a large number to a much smaller one shifts the smaller operand's significand past the precision of the sum, losing information, a phenomenon called absorption. Subtracting two nearly equal numbers produces catastrophic cancellation, where leading digits vanish and the relative error of the result is amplified. Long sums, recurrences, and iterative solvers can accumulate errors in ways captured by forward and backward error analysis, a framework systematized by James Wilkinson at the UK National Physical Laboratory during the 1960s.

Mitigation and Rigorous Bounds

Numerical analysts mitigate roundoff through algorithm redesign, higher-precision arithmetic, and rigorous bounding. Compensated summation such as the Kahan algorithm recovers most of the accuracy lost in long additions, while pairwise and tree-based reductions limit the growth of rounding with problem size. Interval arithmetic and verified computation produce enclosures that are guaranteed to contain the true answer despite roundoff, and stochastic arithmetic estimates the number of reliable digits at runtime. A survey of rigorous roundoff error analysis techniques published in CAV 2021 reviews tools that certify worst-case error bounds for real code.

Applications

Roundoff error analysis has applications in a wide range of disciplines, including:

• Scientific computing for linear algebra, differential equations, and optimization
• Signal processing and digital filter implementation in fixed and floating-point hardware
• Computer graphics and physically based rendering, where accumulated error shows as visible artifacts
• Control systems and embedded software where stability margins depend on numerical precision
• Finance, where regulations often require specific rounding conventions
• Verification and certification of safety-critical avionics, medical, and automotive software