Image reconstruction

What Is Image Reconstruction?

Image reconstruction is the process of recovering a complete, interpretable image from partial, indirect, or degraded measurements. Rather than capturing a scene with a direct pixel-by-pixel sensor, many imaging systems acquire transformed or undersampled data, and a reconstruction algorithm must invert that acquisition process to produce the final image. The field sits at the intersection of signal processing, mathematical optimization, and statistical inference, and its techniques underpin diagnostic medical imaging, computational photography, and remote sensing.

The reconstruction problem is almost always an inverse problem: given a set of measurements related to an unknown image through a known forward model, find the image that best explains the data. Because inverse problems are typically ill-posed, meaning that many images are consistent with a given measurement set, practical reconstruction methods incorporate regularization constraints that favor solutions with certain expected properties, such as smoothness, sparsity in a transform domain, or consistency with learned priors.

Inverse Problems and Compressed Sensing

Classical image reconstruction casts the recovery task as a linear system, where the forward operator maps the unknown image to the measured data. Regularization methods such as Tikhonov (L2) regularization and Total Variation (TV) minimization were among the first to produce useful reconstructions from noisy or incomplete measurements. Compressed sensing extended this framework by showing that images sparse in a suitable basis, such as wavelets, can be exactly or approximately recovered from far fewer measurements than the Nyquist criterion would require, provided the measurement operator satisfies certain incoherence conditions. Compressed sensing theory applied to MRI, published in IEEE Signal Processing Magazine in 2008, demonstrated that clinical-quality MRI scans could be obtained from substantially undersampled k-space data, reducing scan time without sacrificing diagnostic resolution.

Deep learning has since transformed the field. Unrolled networks, which implement learned iterations of classical optimization algorithms, and fully learned networks trained end-to-end on large datasets of image-measurement pairs can exploit statistical structure in natural or medical images far beyond what hand-crafted priors capture. Deep learning for MRI reconstruction, surveyed in IEEE Signal Processing Magazine in 2020, shows that neural network methods consistently reduce reconstruction artifacts compared to classical optimization at matched undersampling rates.

MRI and Tomographic Reconstruction

Magnetic resonance imaging acquires data in the spatial frequency domain (k-space) rather than directly in image space, making reconstruction a mandatory processing step rather than an optional enhancement. A standard MRI scan applies an inverse Fourier transform to a fully sampled k-space grid, but accelerated acquisitions deliberately undersample k-space to shorten scan times, requiring iterative or deep-learning-based reconstruction to fill in missing information. Computed tomography (CT) reconstruction follows an analogous structure: raw data are acquired as line integrals (projections) of attenuation through the body, and filtered backprojection or iterative algebraic methods reconstruct the cross-sectional image. Both modalities share the challenge of balancing reconstruction fidelity against the noise amplification that arises from aggressive acceleration or dose reduction.

Holographic Reconstruction

Holography records the phase and amplitude of a wavefront scattered from an object, rather than the intensity only. Reconstruction from a hologram involves numerical propagation of the recorded wavefront back to the object plane, recovering depth information that intensity-only imaging discards. Computational holographic wavefront reconstruction using deep learning in neural networks, published in Light: Science and Applications, demonstrated that networks trained on synthetic holograms could recover phase and amplitude from noisy holograms faster and with fewer artifacts than iterative phase retrieval algorithms.

Applications

Image reconstruction has applications in a wide range of fields, including: