AR | andy moorer

Spatial Computing is Emerging

Spatial Computing: A New Era in Interaction

Emerging XR technology can completely change the way we entertain ourselves, communicate with others, or even get work done. Spatial computing is opening up exciting possibilities in these areas, merging our physical world with immersive digital experiences. VChatting with a friend in a virtual café, collaborating on a project in a shared 3D space, watching movies with loved ones across the country, playing new types of games and visiting places even if you can’t walk… Whether it’s through work, gaming, creative projects, or enhancing our everyday interactions, spatial computing is starting to reshape our reality in ways we never imagined.

Understanding XR, VR, MR, and AR – Terminology

Extended Reality (XR): This is an umbrella term that encompasses all immersive technologies, including Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). XR represents a continuum of experiences that blend the physical and digital worlds.
Virtual Reality (VR): VR immerses users in a completely digital environment, often using a headset that blocks out the physical world. Users can interact with this virtual space using controllers or their hands.
Augmented Reality (AR): AR overlays digital information onto the real world. This is often achieved through devices like smartphones or AR glasses, allowing users to see and interact with digital elements while remaining aware of their physical surroundings.
Mixed Reality (MR): MR combines elements of VR and AR, allowing digital and physical objects to co-exist and interact in real-time. It utilizes advanced sensors and displays to provide a seamless blend of both worlds.

Use Cases for the Meta Quest 3 Beyond Gaming

The Meta Quest 3 is versatile, inexpensive and powerful mixed reality headset which has significant usage outside of just being a gaming platform:

Education and Training: Institutions can use the Meta Quest 3 for immersive learning experiences, allowing students to explore complex subjects in a virtual environment. For example, medical students can practice surgeries in a simulated environment, and pilots can practice procedures prior to further training. Mixed reality adds a powerful new way to learn, for instance one example that impressed many was a mixed reality application that helps people learn to play the piano.
Shared Spaces: People of all kinds can leverage the device for remote interactions, sharing the same space in 3d much like we communicate with tools like facetime or zoom. Only with a spatial computer people can share activities such as gaming, meetings, watching movies or looking at maps, all while experiencing the presence of the coworkers, friends or family they are interacting with.
Creative Work: Artists and designers of all kinds already use the Quest headsets for many different tasks. Some artists have become widely known for their creations in a 3d space such as Tilt Brush. People can practice oil painting in mixed reality (it’s surprisingly good) and animators have created thousands of excellent experiences in Quill (visit the free app “Theatre Elsewhere to see their work.)
Healthcare: Therapists can use VR therapy for exposure treatments or to create calming environments for patients, medical professionals can practice techniques, surgeons can guide surgical robots or view scan data with colleagues regardless of location.
Fitness and Wellness: This is one area in which the current headsets are booming. Users can engage in guided workouts or meditation sessions in immersive environments that enhance the experience, lose weight while playing physical fitness oriented games, and even participate in shared biking and other workout sessions without having to travel to the gym or worry about the time.
Virtual Screens: One of the most useful aspects of wireless mixed reality headsets is being able to create multiple virtual screens anywhere you like. They can be theatre scale or small windows sitting on your desk. In these screens you can view apps, watch films and video, play xbox games or perhaps most significantly connect to laptops and pcs to work from the comfort of your couch or hotel. Is your environment distracting? Replace your unpleasant surrounding while on a plane, at the hospital or in some skeevy hotel with a comforting virtual space filled with the content you care about. It’s pretty amazing to experience.

Tethered vs. Wireless VR

Tethered VR: This setup connects the VR headset to a computer or console through cables. While tethering can provide high-quality graphics and processing power, it limits mobility and can hinder the immersive experience.
Wireless VR: Wireless headsets, like the Meta Quest 3, provide greater freedom of movement and convenience. They are equipped with their own processing capabilities, allowing users to move around without being restricted by cables. While in wireless mode they sacrifice some of the powerful computing power of a PC with a big graphics card for versatility and freedom. And they can still tether to a PC when the extra power of a high end GPU is desired.

Passthrough Mixed Reality vs. Augmented Reality

Passthrough Mixed Reality: This technology uses external cameras to capture the physical environment, allowing users to see their surroundings with a digital overlay. The integration of real and virtual elements enables interactive experiences where digital content can interact with real-world objects.
Augmented Reality: In contrast, AR typically displays digital content over the physical world viewed through glasses or a visor. While AR enhances real-world views, passthrough mixed reality creates an interactive and immersive experience by merging both environments, with the option to replace or alter the passthru video, for instance dimming your surroundings while watching a movie.

Recent AI Advancements and Their Impact on Spatial Computing

Recent advancements in artificial intelligence have significantly accelerated the timeline for emerging spatial computing technologies. AI is enhancing various aspects of XR, including:

Natural Language Processing: This enables more intuitive interactions with virtual environments, allowing users to communicate with AI-driven avatars or systems using natural speech.
Computer Vision: AI-powered computer vision improves object recognition and tracking, enhancing the realism and interactivity of AR and MR experiences.
Content Creation: AI tools are streamlining the creation of virtual content, making it easier for developers to produce high-quality assets quickly.

These advancements not only expedite the development of spatial computing technologies but also broaden their accessibility, allowing more users to engage with these immersive environments.

The Social Potential of XR Devices

XR devices like the Meta Quest 3 hold substantial social potential. They can foster new forms of social interaction and community building, allowing users to connect in immersive spaces that transcend geographical limitations. For instance:

Virtual Social Spaces: Users can gather in virtual environments for social events, meetups, or collaborative projects, creating a sense of presence and connection that traditional video calls cannot replicate.
Shared Experiences: XR can facilitate shared experiences, such as watching movies or playing games together in a virtual setting, which can enhance social bonds.
Cultural Exchange: XR platforms can serve as venues for cultural exchange, where users from diverse backgrounds can share experiences, ideas, and creativity in a shared virtual space.

The Meta Orion Prototype and Future Use Cases

The Meta Orion prototype showcased a vision for the future of spatial computing, focusing on advanced features like improved field of view, new neural (EMG) interaction, and AI integration similar to the Meta Raybans. It proved that the time for advanced augmented reality is drawing near and how powerful an experience it could be even as a prototype.

Potential future use cases for spatial computing include:

Enhanced Telepresence: Combining virtual avatars and holograms, enabling more immersive remote interactions.
Smart Environments: Integrating spatial computing into smart home systems to allow users to interact with their surroundings through gestures and voice.
Workplace Solutions: Creating virtual workspaces that adapt to individual user needs, enhancing productivity and collaboration.
Personalized Shopping Experiences: AR applications that allow consumers to preview products in their homes before making a purchase.

Final Thoughts

Spatial computing represents a significant shift in how we engage with technology, offering innovative applications that extend far beyond entertainment. With headsets like the Meta Quest 3, we are entering a time where the opportunity to redefine interactions in education, business, and everyday life is emerging. Recent advancements in AI are further accelerating this evolution, making spatial computing technologies more powerful and accessible. As we continue to explore and develop these technologies, the potential for new experiences, social connections, and solutions will only expand.

This point in time is analogous to the early days of the “world wide web.” It was becoming increasingly clear this was a vast new frontier of computing but both the hardware and the applications were still treading new ground. Would the “world wide web” replace services like BBS systems, CompuServe and America Online? Everything was slow to load, and the graphics were not impressive (yet). But the potential was emerging faster and faster, and early adopters blazed new trails and created an entire vast ecosystem that today are used by almost 7 billion people. Spatial computing and XR is the next step in the long story of ever newer forms of media emerging to change the world. Theatre was replaced by radio, then movies emerged, then television, then computers, then portable multimedia in the form of smartphones and tablets, and now here we are at the next evolution.

It’s going to be a fun few decades.

Working in VR

It’s been a while since my last post, mainly because I’ve been busy at Meta. Working in VR on a daily basis has been fascinating. Meetings in VR, working in a virtual office with a Quest Pro, and (now that the pro can be seen in public) traveling with that same virtual office.

There are a lot of things I can’t share but one thing which strikes me is the sheer speed at which VR is becoming a transformative tool for all kinds of business. Interestingly enough, the “standard” business use cases are emerging more rapidly than many of the creative scenarios.

In hindsight this makes sense… You can get a lot of work done with 2d applications on laptops and mobile devices already. While often creative work requires high-end physical setups with resolutions and color clarity not yet available in VR.

What VR is bringing to the table is that you can now use your ”standard” workplace applications tools in virtual spaces… which you can take with you.

By this I mean your workspace is not limited to your portable screen any more. You can take along your whole office (albeit a virtual one), with arrays of huge monitors around you to multitask as you prefer. And already you have increasing control of what the rest of the office around you looks like… a fireplace nearby, or a night sky and aurora above you… a music system, knickknacks on a shelf, photos of a loved one, a working clock on your desk, and so on.

The point is your virtual space can exist wherever you do. In a hotel room, airport terminal, or the cramped confines of airline seating (where the ability to be anywhere other than 9 inches away from the seat in front of you becomes a godsend..) Tired of the current layout or your view? You can change it around whenever you choose. Need an additional monitor? Just spawn another one.

With mixed reality, you can increasingly blend the real world around you with the virtual. You can see your real hands typing on a virtual keyboard. You can remain in VR but confidently move between your couch and your desk. Don’t worry, if your pet walks into your space you can see it, too.

And you can collaborate with your coworkers in your virtual office space, regardless of where you are. I do the majority of my work from my home and meet in VR a lot. I tend to prefer it to zoom meetings.

Sure, you are in a somewhat cartoony place with somewhat cartoony coworkers and self. Surprisingly that really doesn’t matter – you get used to it and even better you don’t have bad hair days or need to worry about what your physical surroundings may look like.

The sense of presence makes a big difference. The nature of discussions is impacted I think, and in a positive way. For better or worse, you are in the meeting. And I find the additional engagement this sense of presence delivers causes me to participate and remember the meeting more.

This sense of additional engagement from a VR presence has been observed in a number of studies which have measured considerable increase in participant’s retention of information in VR versus videoconferencing. And unlike physical real-world meetings participants can be scattered across the planet.

The first time I went to a real-world meeting with my team at meta I was immediately comfortable and productive with my teammates – despite never having actually been in the same room with them before. I knew them already, from VR. Their body language was familiar, their whole presence was known to me already, the fact that we were physically in the same room for the first time was unremarkable.

That’s not to say face-to-face communications are going by the wayside, not at all. Businesses are discovering that it’s critical to make sure closely partnered coworkers get some “real world” time. VR cannot replace face-to-face interactions entirely. But it can make remotely dispersed teams more comfortable and productive.

From any physical location you can all group up. You can read each other’s expressions and body language and feel their proximity. It doesn’t matter if you don’t look your best because your avatar looks fine. Meeting in VR gives you the immediacy of dropping into a videoconference with the familiarity (and retention) of a face to face meeting.

I can tap my headset and see the “real” world around me if I need to. My passthrough image on the Quest Pro is accurate and fully stereoscopic so I can comfortably reach out to grab a cup of hot coffee from a flight attendant, or pet my dog on the head in my home office… before jumping back “into” my VR space.

In a VR workspace I can mute myself but remain in the virtual room. I can see my laptop screen or project it onto the VR whiteboard so everyone can see it. I can draw on the whiteboard from my desk, point at it with a “laser pointer,” or get up in front of it to present to the others. Attendees who call in from “2d” (aka video participants using their cellphones or webcams) are there too.

And like the internet once did, this is going to change everything.

It’s transformative. This is why companies are purchasing headsets designed specifically for this kind of working, and why companies like Microsoft and Accenture are moving to bring their tools and workforces into the VR space.

Like the internet when it was new, you’re going to have to experience it for a while before the benefits of this new kind of media really sinks in. The tools and workflows are going to change and improve. Offices will get rid of the miles of cubicle spaces that exist today and instead emphasize conference spaces and comfortable temp locations for a more mobile and dispersed workforce.

Home workspaces will become more important, while costs once applied towards physical office infrastructure will be reduced to a degree and applied towards equipping a more remote workforce, and supporting periodic meetups.

This isn’t going away: you are going to experience it (if you haven’t already.) And in time literally everyone will. It’s that transformative.

Like the early internet, or smart phones, DVDs, streaming, the web and online shopping this inevitable, and it’s coming soon.

The first years are going to be marked by frustrations and constant change, but that’s part of the fun. Fortunes will be made and lost, new markets will emerge, and surprises are going to happen every step of the way… Enjoy the ride!