Gamified yoga app for HoloLens | Solo project by Yinghua | Built with Unity, C#, Illustrator, Photoshop, Premiere and Blocks (Google VR)
" A special thanks to Mark Skwarek for constantly supporting the project, Siyuan Qiu for helping improve code efficiency, Rose Alexander for providing professional yoga instructions, locations and users to test, and Rob Ryan, Marlon Evans and, Yen-an Chen for lending me the expensive HoloLens. "
Many people fail to learn yoga through current methods, such as watching Youtube videos, mobile apps, or in-person yoga instructors. These methods all require learners to focus on a screen or the instructor, instead of on the movement of their own body, straining their neck to see the original throughout the session. This is against yoga’s philosophy - listening to the body and gaining inner strength.
YOGAR provides a 3D learning experience through Mixed Reality (MR), which allows people to study yoga interactively in a familiar space with a calming, spatial illusion. Learners see marks in space, hear audio instructions, and watch as relaxing videos guide them through each pose.
" I thought a lot about the differences between VR (Virtual Reality) and AR (Augmented Reality) when choosing the technology and platform for YOGAR. VR cannot provide a safe experience due to the lack of spatial mapping. Mobile AR doesn't allow users to be hands-free. I also debated on adding motion sensing technology, such as Kinect. I didn't use it because 1. it would add a space restriction, 2. it would require users to calibrate the sensor, 3. precise correction feedback is not necessary, since Yoga emphasizes listening to the body rather than forcing the body to reach a standard. Complementary audio instruction on adjustment is sufficient. "
To validate whether 3D marks are an intuitive way to learn yoga, a paper prototype of YOGAR was tested with an experienced yoga instructor - Rose - and her beginning student. Rose was first asked to demonstrate the triangle pose. Her hands and feet positions were marked by hand/foot-prints with the help of a few yoga blocks.
" I specifically chose the triangle pose for the test because it only requires two surfaces to create its 3D marks and its difficulty level is appropriate for beginners. "
After the marks were placed, Rose left the yoga mat and the student was brought in. The student was asked to make the pose based solely on the marks and figured it out immediately. Rose evaluated her pose as correct. Both of them were asked for feedback afterwards.
Takeaways from the prototyping test:
- It is a great way for beginners to self-study yoga poses quickly
- More instructions are needed besides the hands and feet, such as where the hips go, how to use muscles, etc.
- Need to accommodate different heights and body limitations
Mock-up & Iterations
The interface of YOGAR is inspired by minimalist design, since it is mixed with a potentially complicated real world environment.
As for creating a soothing yoga atmosphere, a starry design was adopted over placing decorations on the ground. This works with the most surroundings, since ceilings are rarely complex.
" When I was sketching the wireframe, I found that a traditional 2D wireframe cannot represent a 3D environment. So I sketched it in 3D. "
The mock-up for YOGAR separates poses into scenes. It requires users to tap floating buttons to advance.
Every image asset uses a black background, as everything rendered as black will be transparent when viewing through HoloLens.
" I thought separating poses into scenes was a great idea. It's good for organizing assets and can simplify the code by just using the UnityEngine.SceneManager class. However, I didn't realize that it would complicate the process of switching poses due to spatial mapping. "
Major problems collected from mock-up feedback:
- There is too much instruction text in the opening scene
- Users need to redo spatial mapping when switching the pose
- Clickable area for each button is too limited
- Black background is present when recording the view
" I didn't know black will not be transparent in recording. For demo purposes, I decided to remove all backgrounds in the next iteration. "
- Users only need to do spatial mapping once
- Left and right hands/feet are marked with labels and color cues
- All black backgrounds are removed
" Instead of shortening the length of instruction text, I distributed them into different locations based on where the user will look next. It was still too much, so I decided to replace most of them with an audio in the next iteration. "
People who've never done yoga were invited to participate. Major problems observed during the test include:
- The tapping gesture is very tricky for most people
- Transitions are needed between poses
- The main interface should remain user's active field of view
- Additional instructions for each pose need to be added
" When observing use of the app, I began to ponder why people would like to complete the session, beyond self-motivation and testing purposes. I decided to add story telling in the next iteration to make it more fun. "
- Voice command added to replace tapping
- Interface moves with user's gaze
- Audio instructions added to replace text
- Visual demonstrations added to each pose
- Guide users to hold each pose for 20 seconds before advancing
- Story telling added to motivate users and gamify the experience
" I added an ancient Asia yoga master voice, who encourages users to finish to earn a fortune cookie. "
" I experimented with 3D models to replace the images of feet and handprints. I created 4 assets with Blocks by Google in Oculus and imported them to Unity with transparency. However, the experience turned out unexpectedly bad for the experience, because the 3D models made users feel uneasy stepping on them and blocked unnecessary parts of their view. Therefore, I did not use the 3D assets. They were uploaded to Google's Poly website for other people to use."
NYU ITP Winter Show
YOGAR was presented at the NYU ITP Winter Show and featured by HuffPost, out of over a hundred projects presented.
" I was overwhelmed by people's interest and positive feedback about YOGAR. The demo was in constant demand for the duration of the show. The age of people who tried YOGAR ranged from 5 to 70. "
demo at Microsoft HoloLens Office
In June 2018, I was invited for a demo at Microsoft HoloLens Office at Mountain View.
" It was such an honor for me to present YOGAR in front of the creators of HoloLens. I am very glad to see them enjoying using the app and I am super grateful for their insightful feedback."
" After the Winter Show, I added a 20-second soothing video clip during each pose and produced the following demo video. "
more projects by Yinghua