In late 2013, in a dark, sweaty room somewhere in Florida, Paul Reynolds saw the future. He was plugged into a machine affectionately called ‘The Beast’ by its creators. Reynolds cannot show us pictures of the device, nor discuss the experience, as he remains under a non-disclosure agreement with Magic Leap, a start-up specializing in VR and mixed reality but his experience with ‘The Beast’ was transformative and left him wanting more.
“The thing about it was, coming from a content development background, I was seeing this amazing technology but I was kind of disappointed by the demo experience,” he explains. “One of the biggest reasons the demos weren’t good was because there was just one guy making content for the device.”
When Reynolds moved to Florida to begin working for Magic Leap, his first act was to encourage the CEO to allow the designers more time with the device. In their first week, Reynolds and his team set created an interactive demo of a tiny dragon flying around the room and then landing on the user’s hand.
“It was like 11:00 at night. The dragon perched on my finger and I began to laugh hysterically. Even though I was the one that actually created this, and I knew all the technology behind it, there was this moment where I just uncontrollably laughed like a child.” It was at this moment that Reynolds had his eureka moment. “I realized that this is what presence in mixed reality is all about. This is the closest we’ve ever been to having our perception system interact with digital systems,” he says.
A scarce two years later Reynolds became Senior Director managing the development of user-facing software on Magic Leap products.
Lessons in Exteroception
In his talk, ‘Mixing Realities’ for With The Best, Paul Reynolds asks, “To think about what is this mixed reality, you have to first take a step back and ask, ‘what is our current reality?’ You have your presence in the moment and this is basically your mind trying to process where you are, what you’re doing.”
‘Presence’ is a term both philosophical and practical. Reynolds, by trying to engineer a simulacrum of the present, is taking a substantive view of your reality. In essence, your reality is a sum of your perceptions and when those perceptions can incorporate virtual elements, designers can begin to realise the promise in the term ‘virtual reality’.
“To go back to the dragon, everything was checking out and I was anticipating this weight hitting my finger,” he says. “The hysterical laughter was in response to me not feeling that weight.” Navigating this uncanny disconnect between perception and reality is the first major challenge for designers. “True mixed reality happens when these digital signals pass the checks and balances of the perception system to a sufficient enough degree that we’re not uncomfortable and we allow ourselves to process these virtual things as we would the analogue things that we already process.”
Designing for the Matrix
Talking about the checks and balances of the user’s personal reality is one thing, but what has Reynolds learned about mixed reality design? “If you want to think about creating experiences for these platforms, the very first thing you have to do is get out of this mindset of you controlling the contexts,” he says.
Unlike a filmmaker or a game designer, in mixed reality the designer does not have the ability to control the perspective of the user through the focal point of a lens or screen. “You have to think about how to tell these stories, how to create these experiences in arbitrary environments. Can I tell the same story that works in someone’s living room as well as it works in someone’s backyard?”
But as with most emergent technologies, there is a dystopian potential to mixed reality, one that Reynolds addresses directly. “If you took the current model of how we interact with the web and social networks, which is this ‘attention economy’ model and applied it to this new technology where we can tap directly into your perceptions and can render virtual content in your real world, it starts to get scary super fast.”
Reynolds argues that this must be at the forefront of designer’s mind as this tech continues to develop. “Mixed reality has to get people to be mindful and attentive. For so long, we’ve been disconnecting more and more. MR is actually an opportunity to reconnect us with each other. To me, thoughtful design always wins out over novelty. Yes, there’s going to be such cool things we can do with this technology. That doesn’t mean we should do them.”