What Are Videos?

Videos are recorded or synthetically generated sequences of visual frames, typically accompanied by synchronized audio, that represent motion or changing visual information across time. Each frame is a two-dimensional array of pixel values encoded with color and luminance information; displayed in rapid succession at rates typically between 24 and 120 frames per second, they produce the perceptual illusion of continuous motion. Videos are the primary medium for audiovisual communication, documentation, and entertainment in both professional and consumer contexts, and they constitute the dominant share of global Internet traffic.

The engineering of video spans the full signal chain from capture through compression, transmission, storage, retrieval, and display. Each stage has its own set of standards, algorithms, and hardware implementations, and the design choices at each stage affect quality, cost, and compatibility with downstream systems.

Capture and Representation

Video capture converts light from a physical scene into a digital signal through an image sensor, typically a CMOS or CCD array. The sensor samples the scene spatially according to its pixel grid and temporally according to its frame rate and shutter speed. Color is represented using color space models, most commonly the YCbCr format used in broadcast and compression standards, which separates luminance from chrominance to allow differential treatment of spatial frequencies that human vision resolves to different degrees. Resolution, bit depth, dynamic range, and frame rate together determine the fidelity of the captured signal. The IEEE Transactions on Image Processing publishes foundational research on image formation models, sensor noise characteristics, and color processing algorithms that govern how raw sensor data is converted into a usable video signal.

Compression and Delivery

Raw video data rates are impractically high for storage or transmission without compression. A 4K video stream at 60 frames per second with 10-bit color depth requires several gigabits per second uncompressed. The MPEG family of standards, from MPEG-1 through MPEG-4 and its successors H.264/AVC and H.265/HEVC, address this through hybrid predictive and transform coding that removes spatial and temporal redundancy. The MPEG-4 standard introduced object-based representation, allowing independent coding and manipulation of foreground objects and background layers. Content delivery networks cache compressed video files at geographically distributed servers, and adaptive bitrate streaming protocols such as MPEG-DASH select an appropriate quality level for each viewer based on measured network throughput.

Analysis and Understanding

Computational analysis of video content extracts semantic information from raw pixel data, supporting applications from content moderation to scientific measurement. Object detection algorithms identify and locate instances of categories within individual frames; tracking algorithms follow detected objects across frame sequences. Action recognition models classify the activity depicted in a clip, drawing on temporal patterns of appearance change across frames. Video retrieval systems index content for search using automatically extracted features including speech transcripts, detected objects, scene categories, and visual fingerprints. A 2024 survey on perceptual video quality assessment illustrates how quality measurement itself is a form of video analysis, requiring models that predict human perceptual responses from signal-level measurements.

Applications

Videos have applications across an exceptionally wide range of domains, including:

• Broadcast television, streaming media, and on-demand entertainment
• Scientific recording and visualization in biology, physics, and geoscience
• Medical imaging for surgical guidance, diagnostic review, and patient monitoring
• Security and public safety through surveillance and forensic documentation
• Education and training through instructional media and simulation recordings
• Industrial inspection and process monitoring using machine vision systems