What Is Computer Mediated Reality?

Computer mediated reality is a technology area concerned with using computing systems to modify, augment, or replace a person's perception of the physical world through a wearable or handheld display. The defining characteristic is that a computer interposes itself between the user and their environment in some way: adding digital content to a live view, filtering or removing elements from the visual scene, or substituting an entirely synthetic environment for the real one. Computer mediated reality is a superset that encompasses augmented reality (AR), diminished reality, mixed reality (MR), and virtual reality (VR), treating them as points along a continuum defined by the degree to which real-world perception is retained or replaced.

The conceptual framework for this continuum was articulated by Paul Milgram and Fumio Kishino in a widely cited 1994 paper, which introduced the "reality-virtuality continuum" as a tool for classifying display technologies. At one end sits the unmediated physical environment; at the other, fully immersive virtual reality. Between them lie mixed reality environments in which digital and physical objects coexist and interact in real time. Research into computer mediated reality accelerated through the 2000s and 2010s as head-mounted display hardware, computer vision algorithms, and mobile GPU performance reached thresholds that made real-time scene understanding practical. The IEEE International Symposium on Mixed and Augmented Reality (ISMAR) is the primary venue for peer-reviewed research on display architectures, tracking systems, and interaction methods in this domain.

Augmented and Diminished Reality

Augmented reality adds computer-generated content, such as three-dimensional models, labels, or overlaid data, to a live view of the real world, seen either through an optical see-through display or on a camera passthrough. Diminished reality performs the complementary operation, removing or obscuring objects from the perceived scene through real-time inpainting or selective occlusion. Both modalities require precise spatial registration: the added or removed content must be anchored accurately to the physical scene across viewpoint changes and in the presence of moving objects. Computer vision techniques including simultaneous localization and mapping (SLAM) and marker-based tracking are the primary tools for achieving this registration at acceptable latency. The PMC article on augmented and mixed reality for interventional radiology discusses how registration accuracy requirements scale with the precision demands of a given application.

The Mixed Reality Continuum

Mixed reality occupies the region of the reality-virtuality continuum in which real and virtual objects coexist and interact, with neither fully dominant. In a mixed reality system, a virtual object may cast shadows on real surfaces, occlude physical objects correctly from arbitrary viewpoints, or respond to real-world lighting. Achieving perceptually consistent mixed reality requires depth sensing to reconstruct the scene geometry, photometric estimation to match rendering illumination to the real environment, and physics simulation to govern object interactions. Head-mounted displays with onboard depth cameras, such as those used in research and industrial applications, combine these subsystems in a single wearable form factor.

Display and Sensing Technologies

The hardware layer of computer mediated reality spans optical see-through displays, video see-through displays, and projection-based systems. Optical combiners such as waveguides and birdbath designs determine field of view, brightness, and vergence-accommodation conflict properties. Sensing modalities include RGB-D cameras, time-of-flight sensors, inertial measurement units, and eye-tracking systems. The IEEE Xplore collection on virtual, augmented, and mixed reality applications covers recent work on display hardware, sensor fusion, and perceptual quality metrics across these modalities.

Applications

Computer mediated reality has applications in a range of professional and consumer domains, including:

• Surgical training and intraoperative guidance, overlaying anatomical models on a patient view
• Industrial maintenance, displaying repair instructions and diagnostics on physical equipment
• Architecture and construction, visualizing building information models in situ on a site
• Education and simulation, providing immersive training environments without physical risk
• Entertainment and gaming, enabling spatially grounded interactive experiences