What Is Source Coding?

Source coding is the branch of information theory and communications engineering concerned with representing the output of an information source as efficiently as possible, using the fewest bits that still allow accurate or acceptable reconstruction of the original data. The term distinguishes this compression step from channel coding, which adds redundancy to protect data during transmission. Together, the two steps form the basis of Shannon's separation theorem, which states that near-optimal performance can be achieved by handling source compression and channel protection independently.

Shannon's source coding theorem, established in 1948, provides the fundamental bound: no lossless code can compress a source to fewer bits per symbol than the source's entropy H, measured in bits per symbol. Entropy quantifies the average information content of each symbol, and Nature Research Intelligence's overview of information theory and data compression notes that modern research on entropy analysis continues to shape practical coding design, including adaptive methods that track non-stationary sources.

Lossless Source Coding

Lossless coding compresses data in a way that permits exact reconstruction of the original. Huffman coding, introduced in 1952, assigns shorter codewords to more frequent symbols and longer codewords to less frequent ones, producing an optimal prefix-free code for a given symbol probability distribution. Arithmetic coding represents an entire message as a single fractional number in the unit interval and approaches the entropy bound more closely than Huffman coding for short messages or skewed distributions.

More recent entropy coding techniques, including asymmetric numeral systems (ANS) and range coding, achieve near-entropy compression at high throughput and are used in contemporary compression standards such as Zstandard and LZFSE. The Cambridge University Press chapter on entropy coding techniques describes these methods in the context of digital signal compression, noting that context modeling, where the probability of each symbol is conditioned on preceding symbols, substantially narrows the gap between practical code rates and the theoretical entropy limit.

Lossless source coding is used wherever exact reconstruction is required: executable programs, financial records, and lossless audio formats all depend on it. In these settings, even a single bit error introduced by the compression step is unacceptable.

Lossy Source Coding and Rate-Distortion Theory

Lossy coding accepts some irreversible loss of information in exchange for significantly higher compression ratios. Rate-distortion theory, developed by Shannon in 1959, formalizes this trade-off: the rate-distortion function R(D) gives the minimum bit rate required to represent a source with average distortion no greater than D. For Gaussian sources with mean-squared-error distortion, R(D) has a closed form that shows each doubling of distortion reduces the required rate by one bit per sample.

Practical lossy coding operates on transformed representations of the signal rather than the raw samples. Transform coding converts blocks of samples into decorrelated coefficients using transforms such as the discrete cosine transform (DCT) or wavelet transforms; quantization then discards the least significant information in each coefficient. JPEG image compression and the MPEG family of audio and video codecs all follow this general structure.

Perceptual coding applies psychoacoustic or psychovisual models to allocate bits where distortion will be least noticeable, enabling MP3 and AAC audio formats to achieve high-quality audio at 10 to 20 times compression compared to uncompressed PCM. An IEEE conference paper on combinatorial entropy coding explores algorithmic approaches to closing the gap between practical code lengths and the entropy bound.

Applications

Source coding has applications in a range of fields, including:

• Multimedia streaming and storage in audio, video, and image compression standards
• Wireless communications where bandwidth is limited and power-efficient coding reduces transmission load
• Medical imaging for compressing high-resolution radiological data while maintaining diagnostic fidelity
• Remote sensing and satellite imagery transmission
• Archival and backup systems using lossless compression for data integrity