What Is Approximation Error?

Approximation error is the difference between an exact mathematical quantity and a numerical or analytical approximation used in its place. It arises whenever an infinite, continuous, or otherwise ideal mathematical process is replaced by a finite, discrete, or simplified one, which occurs routinely in numerical computation, digital signal processing, control systems design, and machine learning. Quantifying, bounding, and minimizing approximation error is a central concern of numerical analysis, because uncontrolled error can render computed results meaningless or, in safety-critical systems, dangerous.

Approximation error is not a single phenomenon but a family of related effects categorized by their source. The two primary categories are truncation error, which arises from cutting an infinite process short, and roundoff error, which arises from the finite-precision representation of real numbers in digital hardware. Understanding both and their interaction is essential for designing algorithms that produce reliable results.

Truncation Error

Truncation error results from replacing an exact infinite process with a finite approximation. The most familiar example is Taylor series truncation: the exponential function exp(x) equals the infinite series 1 + x + x²/2! + x³/3! + ..., but a computer program that evaluates only the first four terms introduces a truncation error equal to the discarded tail. Similarly, numerical methods for differential equations, such as Euler's method or Runge-Kutta schemes, replace continuous derivatives with finite differences, and the truncation error per step is proportional to a power of the step size h.

Reducing h decreases truncation error, but as lecture notes on roundoff and truncation errors from Oregon State University describe, reducing h too aggressively increases the number of floating-point operations and thus amplifies roundoff error, creating an irreducible minimum total error at an optimal step size.

Roundoff Error and Floating-Point Representation

Roundoff error arises from the way computers store real numbers. The IEEE 754 standard, established in 1985 and universally adopted in modern processors, specifies a binary floating-point format in which a 64-bit double-precision number represents approximately 15 to 17 significant decimal digits. Any real number that cannot be expressed exactly in this format is rounded to the nearest representable value, introducing an error on the order of machine epsilon (approximately 2.22e-16 for 64-bit double precision).

Individual roundoff errors are small, but they accumulate over long computations. Catastrophic cancellation, in which two nearly equal numbers are subtracted, can eliminate nearly all significant digits in a single step. ScienceDirect's engineering overview of roundoff error surveys these mechanisms and discusses numerically stable algorithm design as the primary mitigation strategy.

Error Propagation and Accumulation

In complex computational pipelines, errors in intermediate results feed forward into subsequent operations. Error propagation analysis characterizes how input errors and per-operation roundoff errors combine to produce output errors. Forward error analysis tracks the error accumulated from the initial inputs forward through each computation; backward error analysis asks what modified input data would make the approximate result exact, and judges an algorithm's stability by whether that modification is small. An overview of numerical validation and assessment of numerical accuracy from CNRS researchers presents interval arithmetic and stochastic arithmetic as formal tools for bounding propagated error in floating-point programs.

Applications

Approximation error analysis is applied in:

• Finite element and finite difference simulations in structural, thermal, and fluid mechanics
• Numerical integration and ordinary differential equation solvers in scientific computing
• Digital signal processing filter design where coefficient quantization introduces frequency response error
• Machine learning, where weight quantization and reduced-precision inference introduce approximation errors relative to full-precision training
• Control system discretization, where sampling a continuous-time controller introduces inter-sample errors