What Is Data Compression?

Data compression is a field of information theory and signal processing concerned with reducing the number of bits required to represent data. It applies mathematical techniques to encode information more efficiently, trading storage or bandwidth for computational effort. Compression is foundational to modern digital communication: without it, high-definition video streaming, wireless telephony, and large-scale cloud storage would be impractical at current infrastructure costs.

The field draws on probability theory, linear algebra, and information theory. Claude Shannon's 1948 source coding theorem established a theoretical lower bound on compression, showing that no lossless code can represent a source with fewer bits per symbol than its entropy. That bound, known as the Shannon entropy limit, guides every coding design that followed.

Entropy Coding

Entropy coding is the branch of data compression concerned with achieving the theoretical minimum code length for a given probability distribution. Huffman coding, introduced in 1952, assigns shorter binary codewords to more probable symbols and longer ones to less probable symbols, producing codes that approach the entropy limit for discrete memoryless sources. Arithmetic coding improves on Huffman coding by encoding an entire message as a single fractional number, allowing it to handle non-integer average bit rates and come within fractions of a bit per symbol of the theoretical optimum. The entropy and information theory framework described by Robert Gray at Stanford provides the mathematical grounding for both approaches, connecting source statistics directly to achievable compression ratios.

Rate-Distortion Theory

Lossy compression accepts some degradation in the reconstructed signal in exchange for a substantially lower bit rate. Rate-distortion theory, a branch of information theory formalized by Shannon, quantifies the fundamental tradeoff: for a given allowable level of distortion, it defines the minimum number of bits per symbol required to represent the source. This bound, the rate-distortion function, depends on the source statistics and the chosen distortion measure. Practical lossy codecs use transform coding, which maps signal blocks into a frequency domain where energy compacts into fewer coefficients. The discrete cosine transform, a relative of the Fourier series, is the transform used in JPEG image compression and the MPEG family of video standards; it concentrates image energy into low-frequency coefficients that can be quantized aggressively without visible loss. Rate-distortion theory as a mathematical basis for data compression remains the framework against which all lossy schemes are evaluated.

Video Compression

Video compression combines spatial redundancy reduction within individual frames with temporal redundancy reduction across frames. Motion estimation algorithms identify regions of a frame that have translated from a previous frame, encoding only the displacement vector rather than the full pixel data. The H.264/AVC and H.265/HEVC standards, developed through joint work between the ITU-T and ISO/IEC, apply this principle at scale and achieve compression ratios that make broadcast-quality video deliverable over broadband connections. The H.265/HEVC standard published by the ITU-T defines the reference architecture used in modern streaming platforms and broadcast infrastructure. Neural network-based codecs, trained end-to-end on rate-distortion objectives, are now competitive with hand-designed standards for still image compression and are an active area of research for video.

Applications

Data compression has applications in a wide range of disciplines, including:

• Streaming media delivery, where video and audio compression enables real-time playback over bandwidth-limited networks
• Transcoding pipelines that convert media between formats and bit rates for multi-device distribution
• Satellite and deep-space telemetry, where link budgets are tightly constrained and every bit saved has operational value
• Medical imaging archives, where lossless compression of DICOM files reduces storage costs without sacrificing diagnostic fidelity
• Database and file system storage, where transparent compression reduces I/O and improves effective throughput