What Is Multidimensional Signal Processing?

Multidimensional signal processing (MDSP) is the branch of signal processing that analyzes and transforms signals that vary across two or more independent variables. While classical one-dimensional signal processing operates on signals indexed solely by time, MDSP handles signals indexed by spatial coordinates, time, wavelength, or any combination of dimensions. Images indexed by two spatial axes, video sequences indexed by horizontal position, vertical position, and time, and volumetric medical scans indexed by three spatial coordinates are canonical examples. The mathematical frameworks developed for 1D signals, including Fourier analysis, filtering theory, and sampling theorems, extend naturally but non-trivially to higher dimensions.

2D Signal Processing and Image Fundamentals

Two-dimensional signal processing forms the theoretical foundation for digital imaging. A grayscale image is a 2D discrete signal whose sample values represent intensity at each pixel location. The 2D discrete Fourier transform decomposes the image into sinusoidal components oscillating along both spatial axes, enabling frequency-domain analysis of texture, edges, and periodic patterns. 2D filters, implemented as convolution kernels, perform operations such as sharpening, blurring, edge detection, and noise suppression. The IEEE Signal Processing Society's image processing technical resources document decades of fundamental advances in 2D filter design, including separable filters that decompose 2D operations into sequences of 1D passes for computational efficiency.

Sampling theory in two dimensions addresses how finely a scene must be sampled to avoid aliasing. The 2D Nyquist theorem requires the sampling frequency to exceed twice the highest spatial frequency present in the scene along each axis.

Video Coding and Compression

Video is a three-dimensional signal with two spatial dimensions and one temporal dimension. Raw video generates enormous data rates: an uncompressed 4K stream at 60 frames per second requires several gigabits per second. Video compression exploits spatial redundancy within each frame and temporal redundancy across successive frames. Motion estimation algorithms find regions in a previous frame that closely match regions in the current frame, encoding only the displacement vector rather than the full pixel values. The H.265/HEVC standard achieves roughly twice the compression efficiency of H.264 by using flexible coding unit sizes and improved motion compensation, enabling 4K streaming at manageable bitrates. The newer H.266/VVC standard extends these gains further for 8K and 360-degree video.

Spatiotemporal Signals and Higher Dimensions

Many scientific and engineering signals are inherently spatiotemporal, varying across both space and time. Seismic arrays record ground velocity as a function of sensor position and time, requiring 2D processing across the space-time plane to separate wave modes traveling in different directions. Functional magnetic resonance imaging (fMRI) produces 4D data, three spatial dimensions plus time, from which neuroscientists infer brain activity patterns. Hyperspectral imaging adds a spectral dimension to the two spatial dimensions, generating data cubes processed by techniques from both image processing and chemometrics.

Tensor decomposition methods, generalizations of matrix factorization to higher-order arrays, have become important tools for compressing and interpreting multidimensional datasets. Research on tensor methods for signal processing surveys decompositions such as CANDECOMP/PARAFAC and Tucker, which identify low-rank structure in datasets that span many simultaneous dimensions.

Applications

• Medical imaging: CT, MRI, and PET scanners reconstruct volumetric images from projection data using 3D filtered back-projection and iterative algorithms, enabling non-invasive diagnosis.
• Video streaming: Adaptive bitrate systems encode video at multiple quality levels and switch in real time to match available bandwidth, using MDSP-based compression throughout.
• Remote sensing: Satellite hyperspectral imagers classify land cover, detect mineral deposits, and monitor vegetation health by analyzing spectral signatures across thousands of pixels simultaneously.
• Array signal processing: Antenna arrays and microphone arrays exploit spatial signal variations to separate sources, suppress interference, and estimate source directions.
• Consumer photography: Computational photography pipelines on smartphones apply multi-frame super-resolution, HDR fusion, and noise reduction using 2D and 3D filtering algorithms.
• Autonomous vehicles: Lidar and camera sensor fusion combines 3D point clouds with 2D image data, requiring spatiotemporal processing to track objects across frames.