Know your realities – The difference between Augmented Reality (AR), Virtual Reality (VR), Mixed Reality (MR), and Extended Reality (XR)

Small differences

Autor: Natascha Stumpp

Science, industry, and media are using a variety of words for development and technology, which extends our reality. Not all terms are agreed upon and consistent in their use. New terms are constantly being added. If you are taking a closer look at the topic you will not only encounter Virtual Reality (VR) and Augmented Reality (AR), but also Mixed Reality (MR), Augmented Virtuality, Extended Reality, or Cross Reality. In the discourse on the subject, this often leads to misunderstanding and confusion.

About the different realities

A lot of people know about Virtual Reality. The same goes for Augmented Reality. But where the exact difference between VR and AR lies is only clear to a few. Thanks to mobile AR Apps and Head-mounted displays (HMD), which are becoming more and more popular, you might get a rough idea of how VR and AR differ. But then you suddenly read about Mixed Reality and, you are reaching a point where it is necessary to do some online research. If you are stumbling upon the term Augmented Virtuality and Extended Reality the confusion is completed. We are showing, how the terms are different in detail and question if it is necessary to have that many varying terms.

Augmented Reality (AR)

Freely translated Augmented Reality means extended or enriched reality, which emphasizes the main theme: our reality gets enriched. We still perceive our real environment, but virtual objects or contextual information are superimposed or visually integrated [1]. The decisive factor is the internal data processing and the visualization of virtual content, that is happening very fast, so the user feels no delay. This guarantees for a real-time interaction with the AR-Software. The possibilities for visual presentations in AR are very divers. The user must be able to still see his environment, either by a transparent display (optical-see through) or an opaque screen that displays the camera image of the viewing direction (video-see through) [1].

The optical-see through variant is often implemented by using smart glasses, where a small display is directly placed in front of the eye. Known devices are the HoloLens by Microsoft or the Epson MOVERIO BT-300. Both smart glasses are representatives for binocular devices and have one display per eye. For AR applications monocular versions can also be used, which are providing AR content for one eye. Those monocular glasses, for example Google Glass, are called peripheral head-mounted displays, because the display is in the periphery of the field of view.

Bigger structures are, for example, used in vehicles or protective screens for robot cells. Since mid-2000 head-up-displays (HUD) are integrated into the windshield of vehicles. They are showing the drivers their speed and navigation information. Newer scientific work is testing HUDs in cars for safety functions like collision warning with pedestrians [2].

Virtual Reality (VR)

Virtual Realities are often described as “virtual environment” or “virtual surrounding”. In the Virtual Reality the surrounding, which you are aware of, is exclusively virtual. The user is sealed off the external world and only sees computer-generated visuals. Often virtual environments are visualized with closed Head-Mounted-Displays. These closed HMDs display a stereoscopic image. With the help of built-in and external sensors, the head movement of the VR-glasses wearer is transmitted, and they can move naturally in the virtual world.

It can be roughly summarized, that within AR the proportion of virtual content in relation to the actual surroundings is smaller and the real world is still being observed. In VR, however, only artificial environments are displayed. But what is behind the word mixed reality?

Mixed Reality (MR)

Professor Paul Milgram from the University of Toronto explains the virtual and real environment (Real Environment = RE) as two big components [3]

“Nothing is more real than reality and nothing is more artificial than virtuality.”

But in between those extrema are gradings in which reality and virtuality are mixed, like in mentioned Augmented Reality. AR is a step of the reality-virtuality-continuum, which Professor Milgram and his colleagues shaped. This step of mixing between reality and virtuality is described as Mixed Reality. All steps of this continuum differ in their proportions of reality and virtuality. AR-applications are rather set in the part of reality, which has a little additional virtual content. Applications that are mainly showing virtual content and only have a few sections of reality in it, are described as Virtual Reality.

Grafik, die das Reality-Virtuality-Continum beschreibt

In industry, the term mixed reality is taken in the literal sense and often describes the physical mixing of realities. The device must master spatial tracking and be able to scan its surroundings. One device which has this is function is the Microsoft HoloLens with its spatial mapping and inside-out tracking. In principle, this type of implementation can be described as Mixed Reality, but the economy views it as its own form within the reality-virtuality-continuum and compares it directly with VR and AR. After scientific definitions, the MR characteristics have their own dimensions of the R-V-continuum and should be named for example spatial augmented reality or spatial AR.

Augmented Virtuality

Within the R-V-continuum, there are many other possibilities of specification. The Augmented Virtuality is also one form of mixed reality and comes, among others, into play for immersive VRgames on HMDs. Here, the elements out of the real surroundings are included in the virtual sphere. This can be the user’s own hands with which he can interact in VR or a video stream of a conversation partner.

A group of scientists worked on a project called SpaceWars [4]. In SpaceWars the whole body of the player is filmed in real-time and transferred as an avatar into the virtual world. The body can be seen by other players and functions as an input unit. The player moves its vehicle by moving his body and gestures and interacts naturally with its surrounding.

Extended Reality, Cross Reality, x Reality, or XR?

The collective term for mixed reality following Professor Milgram’s understanding in which pure VR is extended, is the term extended reality.
In scientific papers, the term extended reality is rarely used. The term is mainly used in specialized blogs and used from companies to cover all different kinds of realities. Extended reality is often shortened with XR, while the ”x”stands for ”eXtended”. Furthermore, the x stands as a variable for augmented, virtual, mixed, or something new. Simultaneously, XR is used as a short form for Cross Reality or X Reality, which is defined more concrete. In cross reality offers a wider spectrum of definitions and includes more than AR and augmented virtuality. Up to today, there are no noteworthy differences between the terms, cross reality and extended reality. Cross reality only involves the use of different hardware and software and aims to be cross-platform and cross-reality.

In order to achieve more uniformity within the professional world, the Khronos™ Group has launched a project to make it easier for developers to get the various technologies and end devices under one roof. The organization has already released some known standards, like OpenGL and WebGL, developed in their workgroup OpenXR™ with a cross-platform standard for VR and AR applications. Developers are not obliged to start a separate project for every device and development environment anymore [5]. The project group Khronos Group takes great care not to confuse its own definition and naming of XR with Extended Reality, Cross, or X reality. Khronos’s X is a combination, of the from Virtual and A from Augmented [6]. OpenXR™ has also the potential technology and hardware to blur the barriers of different realities.

There is still disagreement

If you take a closer look at VR or AR it becomes clear that you should not rely on supposedly fixed terms and shortcuts. MR is not equal with MR and XR not with XR or XR. In their basic message, they mean the same, but they differ in detail and can cause misunderstandings. Unified wording has the advantage, that everybody knows what is talked about and what to search for while researching. Speaking of VR, a long explanation is often unnecessary, and it is easier to refer to it to also describe variations. A certain agreement on technical terms is therefore quite helpful.

Since software and technology are constantly being further developed, new forms of virtuality keep being added. Eventually, they must be explained and named. There should not be any borders for the developers, as long as the aimed target groups understand what it is meant. Especially the term virtual and augmented reality have evolved into trendy buzzwords that are used to generate attention. More frequently, even developments are described with those terms that hardly offer virtual or extending elements. However, if you want the technology to be really understood, you should not misuse the relevant technical vocabulary for advertising purposes, but rather convey the actual meaning.

Quellenverzeichnis

[1] R. Azuma: A Survey of Augmented Reality, In Presence: Teleoperators and Virtual Environments 6, 4 (August 1997), 355-385. URL: http://www.ronaldazuma.com/papers/ARpresence.pdf (Stand: 15.05.2018)

[2] H. Kim, J. L. Gabbard, A. M. Anon and T. Misu: Driver Behavior and Performance with Augmented Reality Pedestrian Collision Warning: An Outdoor User Study, in IEEE Transactions on Visualization and Computer Graphics, vol. 24, no. 4, pp. 1515-1524, April 2018. URL: https://ieeexplore.ieee.org/document/8302393/ (Stand: 15.05.2018)

[3]P. Milgram; H. Takemura; A. Utsumi; F. Kishino: Augmented Reality: A class of displays on the reality-virtuality continuum; SPIE Vol. 2351, Telemanipulator and Telepresence Technologies (1994), 282-292. URL: http://etclab.mie.utoronto.ca/publication/1994/Milgram_Takemura_SPIE1994.pdf (Stand: 15.05.2018)

[4] A. Karakottas; A. Papachristou; A. Doumanoglou; N. Zioulis; D. Zarpalas; P. Daras: Augmented VR – SpaceWars, URL: https://youtu.be/7O_TrhtmP5Q (Stand: 15.05.2018)

[5] The Khronos™ Group Inc.: OpenXR; URL: https://www.khronos.org/openxr (Stand: 15.05.2018)

[6] M. Blumberg: How the term XR was coined […], Silicon Valley Global News, URL: https://medium.com/silicon-valley-global-news/how-the-term-xr-was-coined-its-so-funny-hint-it-does-not-mean-extended-reality-4e7fff539676 (Stand: 15.05.2018)