Our Perspectives on Emerging Opportunities in Mixed Reality
As Virtual Reality (VR) and Augmented Reality (AR) find increasing adoption across multiple industries and consumer applications, it’s only a matter of time before they begin to blend and become a unified experience - Mixed Reality (MR).
But before we look deeper at MR it’s important to be aligned around what V/A/M-Reality mean.
Virtual Reality (VR) headsets immerse the user in a fully artificial digital world. The real world is not visible at all. The digital world can be completely made-up or a simulation of a real world location.
Augmented Reality (AR) displays virtual, digital objects over the real world - but the digital objects don’t interact with the real world. They are merely overlaid on top of the user’s real world environment. AR can be experienced with mobile devices (mobile AR) or with 3D AR glasses such as the MagicLeap headset.
Mixed Reality (MR) is basically Augmented Reality on steroids, where digital virtual world/objects are not just overlaid but anchored to and interact with the real world. When done flawlessly (and in no way feasible today), the virtual objects would be indistinguishable from real world objects.
MR is best experienced with a headset that can show both the real world and digital worlds, mixed and rendered as desired.
The difference between VR, AR and MR is demonstrated in this infographic from Forbes:
Source: Forbes.com
The Spectrum of MR
MR itself is not a single definition but rather a full spectrum of possible blended realities between a full virtual experience (VR) and the undoctored real world. Augmented Reality is a subcategory of MR because you are getting a mix of virtual objects and the real world.
Other possible types of MR are:
Augmented Virtuality (AV) where some real world objects are incorporated into a primarily virtual world
A real world experience but with portions that are “portals” into a full virtual world.
A world that is full virtual in one viewer angle. When viewer turns to another angle they see the real world.
What’s available today?
Magic Leap and Microsoft Hololens are the leading MR headsets available today though the quality of MR they offer is somewhat limited. For example, one way to use the Hololens is for Skype chats. In this case, the user wearing the Hololens headset can see a screen of the Skype chat, floating over the real world. The screen moves around as the user looks around. And, at the other end, the people they are talking to can see whatever the user can see from their point of view.
Source: https://www.youtube.com/watch?v=TylDjQ1udZg&feature=youtu.be
This is one way to enhance communications and improve remote presence, especially if both sides have a MR headset. But it’s a far cry from feeling like the people on the other end are actually present in the room with you, which would require volumetric holograms in real time.
Looking forward to a world without “2D Screens”
We envision an ongoing evolution of Mixed Reality headsets such that they can cover the full spectrum of XR - from the real world (fully transparent) to mixed reality to full VR (fully immersive) all on the same device. This could be similar to a pair of AR glasses a much wider field of view than is available today, with lenses that can be "blacked out" to offer a VR experience that is completely immersive, or that can have see-through lenses to allow an AR (or MR) experience. There may always be optimal VR or AR experiences with separate dedicated devices, but we think the killer device that will be owned by everyone who currently owns a smartphone, will be a well balanced blend.
When this has been achieved, we WILL see a platform shift from mobile to spatial computers. Glasses-based 3D displays will replace traditional 2D screens. Why? The seamlessness, limitless flexibility, and ease of interacting will make our current paradigm of interacting with 2D screens seem archaic. A capable AR device will give you access to unlimited “screens” and virtual animated objects placed anywhere you want them, in any configuration, at any size. And you can interact with them in a more natural way, with gestures and motion controllers in a three dimensional, human, manner.
Use cases of evolved MR
Blending the virtual with the real brings some powerful use cases that are not possible in a purely virtual or real world scenario. Here are some such use cases:
Real Estate
Our real estate customers are already using VR to showcase properties under development and MR to enhance listings through virtual furniture staging.
With MR abilities, prospects can place virtual furniture as they physically stand in an apartment or home they are considering buying/leasing, and see how the virtual furniture interacts with the space or with other pre-existing real furniture. For example, you can place different size dining tables into a space to get a realistic feel for how large of a table feels comfortable in that space. This holds great promise for reducing vacancy for apartment communities, and time on the market for homes, while giving very compelling and memorable experience for prospects. Searching for a home is stressful, and viewings become stale and repetitive. Why not make property tours fun again (and more informative) by injecting a little magic into prospect’s lives?
Communications
One of the holy grails of communication is real-life appearing virtual presence via holograms. If this Holoportation project by Microsoft Research comes to full fruition, we very well might see this become real - having conversations with holographic images of our friends and family talking to us as though they were literally in our living room.
Source: Microsoft.com
Virtual Shopping
Consumers can browse through endless aisles of products in immersive VR mode, and then drag and drop products they like into their real home in MR mode, and then see how they feel about the product. This can be applied to endless types of products, avoids needless shipping and returns, and buyer’s remorse, and is of course, very convenient to the consumer.
Product Design and Engineering
Engineers can delve deep into the heart of an engine in a fully VR environment, manipulate the design, and make changes, and then zoom to an AR/MR environment where they move that engine into the engine bay of an aircraft to see how it fits.
Medical Field
MR has vast applications in the training of medical professionals, in improving interactions between doctors and patients, and in the patient experience itself. Exploring the full scope of such MR applications is beyond the scope of this article but here is one very compelling use case - Mixed Reality overlays of patient imaging scans for surgery planning. A digital overlaid 3D representation of a patient’s scan can be used to assess potential challenges and approaches before and during surgery to ensure a smoother operation.
An Ongoing Evolution
The possibilities are endless. Of course, there are challenges, too, in realizing the full potential of MR. For example, constructing and rendering in real time a virtual space that is exactly in line with the real world and adapts as the real world changes. Often called a digital twin, this is not a trivial task.
Another challenge is also to make the virtual look completely real. Even today, virtual objects, while close to looking like real life objects, are not quite the same. More work has to be done to cross the uncanny valley and get the virtual and real worlds to appear seamless.
In many ways, the parallel strides being made in AI and Machine Learning, will also help Mixed Reality reach its true potential.
None of these challenges are insurmountable and there is considerable R&D in progress to bring this to fruition.
MR is not just hype. It’s merely a logical extension of proven technologies - VR and AR have very compelling business use cases. We are only at the beginning of what’s possible.
MR is not a future tech, it’s available now. If you or someone you know is interested in seeing a demonstration of Composed, our virtual staging solution for unfurnished spaces, please contact us today at david@outerrealmvr.com to learn more.