Engaging apprehensive audiences with mixed reality.
The details
Date
June 2018
Methods & Tools
Mixed Reality
User Research
Visual Design
Google Blocks
Collaborators
UX Designer (my role)
UX Designer
UX Design Lead
Developer
The task
In the summer of 2018, Kin + Carta (formerly Solstice), hosted its annual FWD conference where they display new technologies, ideas, and collaborate with clients. Excited to learn about emerging technologies, I donated 3-4 hours a week after work for 4 months in order to create a new experience in a team with two other designers and a developer.
The problem
We were tasked with creating an experience for FWD that engaged both the user in the headset and the user waiting in line while leveraging mixed reality. Some of the principles we were focused on bringing into the experience were:
Fostering an immersive audience experience
Interacting with virtual objects
Achieving a goal through communication and teamwork
The design outcome
Our team created a mixed reality game, FLWR, in which both the user in the headset and users in the audience were engaged to either water flowers or assist in watering flowers. The graphics in the game were designed in Google Blocks, and our final product was showcased at Kin + Carta’s annual conference, FWD in 2018.
The process
We began the process with the constraint of leveraging an already-created mixed reality experience, lovingly-named Space Ventura. This experience required users in the headset to solve a puzzle with only the help of the users outside of the headset. Mixed reality, as we define it, is a bridge between virtual reality and actual reality, where our users were projected into their virtual environment for the outside audience to see.
Idea Creation
To narrow down our concept, we kept our principles in mind and researched best practices for designing in VR. In our desk research we learned that new users are generally quite nervous and afraid of VR, especially if it is their first time using that technology. Knowing our audience at FWD was going to be C-Suite clients who were generally not as willing to put themselves in a vulnerable state in front of their coworkers (which jumping into a VR game could do), we needed to create a concept that was engaging but not overwhelming or intimidating. After brainstorming and pitching our concepts, we landed on a game concept balancing both striking visuals and engaging content.
The VR experience is built in Unity and runs on an HTC Vive VR headset operating in “room scale” configuration, which allows the wearer to move around the virtual experience space by moving in the real world.
In our game, FLWR, the user inside the headset is meant to be “watering flowers” while simultaneously avoiding “weeds” in a 60 second time limit. To water flowers, the user employs the use of the HTC Vive remote and aims the water at any flower bud with an indicator above it. With each flower watered, the user receives positive points, and with every weed watered the user is deducted a higher number of points.
What makes this game unique is that there is no indicator of flower vs. weed for the user inside the virtual world. Only the users in the audience see the difference between flowers and weeds, requiring the audience to guide the user in the headset to both water flowers and stay clear of weeds. This intentionally made the game impossible to play alone - only through teamwork can a user get a great score.
The Technology
Objects in the virtual environment were modeled in the Google Blocks VR design application and were imported into Unity for integration into the experience.
Conceptualizing a 360° Space
After landing on a concept, the designers went to work designing the visual hierarchy and architecture of the virtual environment while our developer took a deep dive into the technology behind Space Ventura to leverage in FLWR. My coworker, Josh Epstein, took the initiative on laying out the environment in a wireframe style to give us a better look at how we wanted to space out our virtual elements in a realistic format for our users. Something we quickly learned was that sketching something on paper looked and felt completely different when it was transformed into the game. A typical wireframe we were used to was now a giant circle where elements lived in a 360° space.
Concept Art by Josh Epstein
Concept Art by Josh Epstein
Defining User Movement
Through countless hours inside the headset, our team worked through placement and sizing of our virtual designs, researching best practices for how close or far away elements needed to be for the user’s eye to understand. While the HTC Vive remote was the only physical object for the user to interact with, many questions arose internally around how to direct their movements inside and outside of the virtual environment: Do we have the user move around to water the flowers? Does the timer or menu live in the environment itself or appear in front of the user’s face?
Thinking through our physical space and audience considerations, we landed on keeping the user stationary as they watered the flowers, as well as placing the timer more realistically inside the virtual world so as to not distract the user.
To enable the mixed reality experience, we used a green screen and handheld video camera (strapped to the second Vive remote), routed through OBS Studio software to create the video display presented to the audience and enabling their participation in the experience.
Designing the Physical Environment
Since FLWR required the participation of the audience in order to function, designing how the audience would view the experience was critical to the success of the game. When demoing the project at the office, our audience was forced to view the user in the headset through a small computer monitor. However, when we eventually brought our concept to the conference center, FLWR was given a massive screen hovering above the green screen, encouraging those who weren’t even actively playing the game to watch and follow along.
Redrawn from an original design by Lauren Blackburn
Asset Creation
Having never designed in VR before, it was a great learning experience to spend hours inside Google Blocks building out the various flowers, trees, and other visual elements in the virtual world they lived in. Sketching out our ideas for how these elements might look was important, but I learned there’s nothing quite like putting on the headset and dragging, twisting, and turning shapes to build out a new virtual experience.
User Testing
Testing our product was an extremely important aspect of the game development, and took place over a couple of weeks after work hours. Since this was a passion project, we didn’t have the same resources as a billable project. Knowing this, getting creative with who and how we were able to test FLWR with was imperative to staying on track with our timeline. We tested not only the game mechanics, but also the talk track given to potential users, balancing how much information to give in a quick elevator pitch to potential and current clients at FWD.
Through some of these initial testing sessions, we discovered valuable user feedback critiquing how fast the game moved, the instructions given, and the visual cues available to the user. To combat the speed of the game, we increased the time available to the user, and tweaked how often new flowers or weeds would spawn.
Additionally, we added in some sound and animation feedback whenever a user successfully watered a flower to increase the sense of accomplishment. We also added in a scoreboard to foster the competitive nature we saw blooming (pun intended) in our users.
FWD 2018
On June 21st, 2018, our team brought FLWR to Kin + Carta’s annual FWD conference, displaying our work to around 400 clients, potential clients, and investors. The game was (and maybe I’m a bit biased here) a complete hit, bringing in a fun and engaging experience for our audience who weren’t used to that type of engagement at conferences they may have attended in the past. The scoreboard brought people coming back again and again, and was the place to be during the happy hour immediately following the conference main content. 🥂
Throughout the day, Kin + Carta user researchers were tasked with following up with participants and captured a massive amount of feedback. Much of the feedback related to the fun and exciting nature of the experience, but an overwhelming amount of participants commented on the lack of connection to what could be possible for their industry or business - how did this game relate to them? As a group we realized we had promoted FLWR as “look how awesome this technology is and tell us how you think it could work in your industry”, when we probably should have spelled that out a bit more for our users.
Office Tour Roadshow
After FWD, the experiences showcased at the conference - including FWD - were added to a tour that was given to any visitors to the office. Taking the feedback from FWD, we were able to tweak our talk-track to speak a bit more to real-world examples where something like mixed reality has been or could be used successfully. Some of the examples we gave included training for dangerous or high-risk careers such as mining or healthcare, where training in a headset with others viewing the user interacting with their virtual environment could be incredibly impactful and even life-saving.
This experience tour was incredibly successful and an exciting role for me personally as I was given the ability to perfect my talk-track and speak to many important visitors to Kin + Carta for over a year, including Mayor Rahm Emannual during a highly-publicized ribbon-cutting ceremony for our new office. I’m very thankful for the opportunity to have been able to interact and build in an emerging technology for such a fun and engaging experience.
What would I have done differently?
If I could re-do the process of this work, I would have taken a service design perspective to the entire FLWR experience. I would look at the user experience outside of the game itself, and think more about how we would interact with the FWD attendees through the layout of the experience from beginning to end.
Additionally, our experience was designed by and created for able-bodied participants, and not created with accessibility in mind. Ideally, this type of experience would be designed by and tested with disabled participants well before it ever made an appearance at a global conference. Even simple impairments like glasses were an afterthought with this new technology, and I would have loved to explore how we made mixed reality a more inclusive technology across the board.