NASA Jet Propulsion Lab
File Navigation In Augmented Reality
Project Description
The NASA Jet Propulsion Lab (JPL) is a national research facility that specializes in carrying out robotic space and earth science missions.
Engineers at JPL are currently utilizing different Augmented Reality (AR) tools to assist them with the design and assembly of the spacecraft. These tools include incorporating Microsoft's Hololens with Protospace - an AR application developed by JPL.
Currently, the engineers can only select a part of the spacecraft and view it in Protospace. These parts are represented as CAD files within a hierarchy. However, engineers are unable to tell where a specific file is located within the current hierarchy.
Our team was given the task to design a method for visualizing this hierarchy in Protospace's AR interface.
Role: User Researcher, Interaction Designer and Interface Designer
Tools: Sketch, Adobe Illustrator & Photoshop, Principle, Keynote
Type: Academic Case Study
Project Length: 6 Weeks
Design Process
Research
With Mixed Reality being a significant component of this project, our initial research revolved around trying out different devices.
Exploring different environments in AR and VR gave us an idea regarding the potential problems that we could face. For instance, one major takeaway from using the Tilt Brush (with the HTC Vive) was that we understood how the Field of View (FOV) and gestures used by the Hololens was much more complicated.
Our research was not just restricted to user-testing, as we also looked into Sci-Fi movies and literature to seek inspiration. We even went through some YouTube tutorials on AR development to better understand the restrictions when designing for the Hololens.
We used Jacob Nielsen's 10 Usability Heuristics for User Interface Design as a reference during our research (and design) process.
Following our initial research, we interviewed some engineering students. While most of them had no experience with mixed reality, a few of them had used AR applications. Since engineering students tend to use modeling software such as CAD they were able to share their feedback regarding that.
After weeks of primary and secondary research, our team proposed the following solutions:
Integrate the hierarchy into the augmented reality model through the use of color coding system.
Visualize the hierarchy in the environment at an angle that is comfortable to the human eye.
User Personas
I. Engineer (with AR Experience)
Name: James
Age: 28
Occupation: Mechanical Engineer at Protospace
Needs: A tool that allows for rapid visualization of whatever is being built and that at the same time allows for trial and error.
Challenges: Prototyping using conventional 3D software takes time and building a physical model, can result in a lot of material being wasted.
II. Engineer (without AR Experience)
Name: Enes
Age: 29
Occupation: Mechanical Engineer
Needs: He needs to stay updated about his team member's progress. For instance, he faces problems when all team members are working on different part of the same object.
Challenges: Finding a tutorial or application that can teach him AR development in detail through a step by step process.
III. Engineering Student
Name: Jacob
Age: 22
Occupation: Mechanical Engineering Student
Needs: He needs a faster rendering process and would prefer something like the real-time render feature used in Final Cut Pro.
Challenges: The speed of the software being used for modeling is a significant issue, especially with some of the more complex solutions.
Initial Design
For the first prototype, we created sketches to depict our proposed solution better. The main idea behind the solution involved using a holographic cube as a navigation tool and incorporate color coding in the model to help the user navigate between the parent and child files. Below are the first annotated sketches we made:
System Diagram
The System Diagram (below) shows how our planned prototype is integrated within Protospace and how it interacts with the Hololens.
Prototype
User Journey for the First Prototype
There are currently no industry standards for prototyping in AR. Our first prototype was done using cardboard and Adobe Photoshop. In this version, we created a mock scenario to depict how the user will browse through the file hierarchy.
Cardboard Version:
Process:
Make a physical prototype using cardboards.
Photoshop images from JPL to create a user story for a mock scenario.
Use cardboard to frame the FOV.
Incorporate the images into Invision to create a clickable prototype for user testing.
User Testing Feedback:
Why use a Cube as a navigational tool? Why not a more fluid polygon?
How will the user find out their level within the hierarchy?
Color Coding: Color could be used to block out areas that other engineering teams are working on to avoid confusion.
User Journey For Second Prototype
Second Prototype: Interactive Version
Keeping the results of our first user testing session in mind, we made a few alterations to our prototype. The main issue that we tried to resolve in this iteration was regarding how to accommodate larger file sizes.
For this prototype, we decided to use a more fluid polygon to represent the child files and keep the cube to navigate within parent files. The number of child files determines the number of sides of the polygon. Using TinkerCad, a browser-based 3D modeling tool, we created our Child-to-Child file navigation tool (i.e., the fluid polygon).
We created a clickable prototype on Invision using images of our Tinkercad model.
User Testing
Horizontal v. Vertical Navigation: Users prefer the Vertical Navigation style of our prototype. As it feels natural when going from Parent to Child files and reiterates to the user that the files are not on the same levels.
3D Design: Users find Flat shapes make are difficult to interact with in AR and would prefer 3D objects to flat shapes for interaction purposes.
Communication Tool
Ending Thoughts
An interesting problem that we encountered during this project was figuring out a way to accommodate larger file hierarchies (for example 300-500 files) in a VR or AR environment. Similarly, it would be exciting to think of ways which will make the transition from flat screens to headsets more seamless for users as AR Helmet Mounted Display Systems (HMDS) become increasingly common.