In this issue

• Meta copies Apple’s Spatial Apps

• Building a social TV app for Meta Quest

• Listen Now - Designing for Multiple Users in Mixed Reality

Tom Ffiske, author of the Immersive Wire, suggested adding a Table of Contents for our readers on mobile. Thanks, Tom!

Let’s dive in.

Meta copied Apple’s Spatial Apps

Meta wants you to make Apple Vision Pro style Spatial Apps for Meta Quest. In their Meta Connect announcement, they shared Meta Spatial SDK, a set of tools that integrate with Android studio to ship apps on Meta Quest combining traditional Android app functionality with spatial content.

It’s basically Apple’s Spatial App framework for VisionOs but for Android.

The Meta Quest is basically a Chromebook. You can connect a mouse and keyboard to it with USB or Bluetooth and find a way to do most things you use your laptop for. I’ve tried to do this with some success.

It will be interesting to see how this plays out. However, 2D apps have not faired well on Quest. We’ve had a handful for a few years now and they are plagued by bad reviews. Recently there have been questions about Wordle for Meta Quest. If it’s easier to play on your phone, then why port it at all?

I’m excited about the future of app development for Quest. The Godot game engine runs on Quest combining VR gameplay testing with a traditional 2D game engine interface for development.

I’m building an app

Apple’s marketing always shows people using the Vision Pro to watch TV alone.

I don’t want to live in a world where XR devices isolate us. During Covid, I used to use an app called BigscreenVR to watch movies with friends. I could stream a movie from my computer to a virtual movie theatre. It had amazing realtime lighting effects so the walls reflected the light from the movie projector.

Watching movies with friends in VR is an incredible experience. After 5 minutes, your brain is completely convinced you are in a movie theatre. It’s why people are losing their minds over the Apple Vision Pro and bringing them on planes.

Recently I showed Hannah BigscreenVR and we watched a move in a virtual theatre together. She wanted to watch it with me while we both sat on the couch. I was there in person with her and could see her as an avatar inside the headset.

We had problems almost immediately. Our play spaces were never lined up. Sometimes I was physically to her left but my avatar was on her right. If one of us got up to get a drink, then we had to realign everything. It was difficult.

With mixed reality, you can create experiences that align multiple headsets together so virtual content is always anchored relative to each viewer using a technology called Shared Spatial Anchors. I used this tech to produce a demo for an Oil & Gas company where users could pick up virtual tools and navigate a shared experience together.

After Hannah and I had a difficult time using traditional VR, I wondered if we could use Shared Spatial Anchors to create a video player that multiple users could operate together. These users could create massive displays and freely move them about. Both users would see the same display in the same physical location.

After doing a thorough search and realizing this vision hasn’t been realized yet, I put together a design and a mockup. The mockup below uses a yellow and pink humanoid to depict how human users would interact with both the physical hardware, physical environment, and virtual interface.

I wanted to give the user a powerful menu that was easy to use like a TV remote. By attacking a virtual touchscreen to the physical controller, you can use a combination of hand tracking with your free hand to tap and interact with the touch screen while using your controller hand to operate physical buttons that align perfectly with their virtual counterparts.

The display itself is quite magical because it is a 3D display. You can move it around and see around the small buttons and objects inside the screen like a ship in a bottle.

It’s an ambitious project, I have colocation working now. Next up the core video player functionality. I’m sure there will be a colocated video player on the Meta Quest store by the time I’m finished with this project, but so far this is still blue ocean for a Mixed Reality app.

See you next week!

That’s it for this week. I’ve been busy bidding on a freelance 3D design project for a national ad campaign and I’m flying to Boston on Friday to film a wedding.

What did you think about this issue? Hit that reply button and let me know!

- Nate

Want to hang out more often? Hit the button to get this free newsletter delivered to your inbox every week.