Teaching in Collaborative Virtual Reality
User Experience Optimization

Master thesis project

Jan 2021 — Apr 2022

Improving User Experience of an Educational VR Application

Project: an academic research project EduInCIVE at Masaryk University

Duration: January 2021 to April 2022

My responsibilities: UX research, UX design, XR design



What was my task?

Goal

Improve the user experience of teachers who teach English as a second language in virtual reality.

Thesis research topic

The thesis studies user-centered design for virtual reality and specific aspects of teaching in virtual reality. My questions were:

  • How do traditional UX/UI design principles translate from 2D screens into VR?

  • How can we optimize the user experience of teaching in collaborative virtual reality?

Context

Masaryk University develops an educational VR application eDive where a teacher meets with a group of up to four students and leads them through a 50-minute interactive lesson. In 2021, the application was used in an experimental course to teach English as a second language to university students.

Users

English teachers at Masaryk University.

Technology

Figma & ShapesXR. I combined 2D and VR prototyping tools.


1

Research — From Screens to Virtual Reality

2

Discover phase
User interviews
Heuristic analysis

3

Define phase
Find opportunity areas
Define problems

4

Prototyping
Research similar apps
Design new features

The Process

  1. I researched user-centered design for virtual reality. I described new paradigms and challenges UX designers encounter when they switch from 2D applications to VR. (2nd chapter)

  2. I studied how existing applications approach user interface design and interactions in virtual reality.

  3. I analyzed the current state of the application eDive (3rd chapter) and performed a heuristic evaluation of its interface. I conducted in-depth interviews with the teachers and internal stakeholders of the project.

  4. I synthesized the research data, identified several opportunities for improvement, and, finally, selected three design problems to focus on. (5th chapter)

  5. I collaborated with the developers to propose improvements for the application and designed several prototypes for the selected design problems. (6th chapter)


In my work, I followed the double diamond process. Below, you can see a diagram of the process along with individual methods (below the diagram) that I used in each phase.


Teaching in Virtual Reality

In my work, I identified several opportunities for improvement of the application eDive. While some of these opportunities are specific to eDive, others could be generalized to other educational applications.

Each opportunity area emerges from one or more of the research methods: user interviews with teachers, internal stakeholder interviews, or heuristic evaluation. During the data synthesis, I gathered my research notes and clustered them by similar topics:

Affinity map of the topics that emerged from the research. The notes are color-coded according to their source (teacher, stakeholder, heuristic).

From the topics, I derived nine areas of the eDive application that could be improved, and I created Point-of-View statements that reflect the teachers' comments:

Lobby—the First Touchpoint with the Platform. The current lobby of the application didn't meet the standards of a modern interface. In addition, it violated several usability heuristics.

Onboarding Students to Virtual Reality. Teachers need to explain to the students how to work with the controllers because the students have little prior experience with VR, and the platform does not explain it. When the students struggle with technology, it limits their learning capabilities during the lesson.

Environment Design. Teachers prefer fun and colorful environments because they feel like fun environments make the lesson more interesting and that dark, monochrome environments can be depressing to the students.

Comparison of virtual environments in VR software Engage (Moon Lesson, At Court) and in the application eDive (Villa Stiassni, Prepositions).

Realistic or Simplified Avatars. Some teachers prefer customizable realistic avatars because they make it easier for them to distinguish between the students. Other teachers prefer simplified avatars because they lower the chance of personal bias.

Comparison of avatar representation in Engage (realistic) and eDive (bubble).

Managing Students. Teachers need to keep track of student involvement because some students tend to speak less than others, and it can be difficult to involve everyone equally, especially in a large group. Teachers need an efficient way to divide students into groups so that they can monitor all the groups and the groups do not distract each other.

Speaking Activities. Users of collaborative virtual reality need to recognize when another user starts speaking because they expect virtual reality to simulate the flow of a real-life conversation, and in real conversations, people can see each other’s lips moving and hear the voice coming out of their mouths.

Taking Notes and Writing during Lesson. English teachers think it would be useful to take notes or write vocabulary during the lesson, but they fear it would take too much time and be difficult to learn.

Teacher’s Custom Content. Teachers need to use their own notes during the lesson because they follow a specific lesson plan with sometimes complex activities, and each teacher has a different level of teaching experience and a specific teaching style, so they prefer to have their personal notes. Teachers want to add their own educational content (presentations) because each teacher has a specific teaching style, and they sometimes need to adjust the predefined teaching materials.

During the lessons in Engage, teachers can open their private notes in a virtual tablet; eDive currently does not offer any such functionality.

Stress, Time Pressure, and Cognitive Overload. Beginner teachers feel overloaded because they need to focus on their teaching methods as well as working with the technology. Teachers need efficient tools that will not take up much time during the lesson because they need to focus on the students and the teaching itself and the lesson is too short to spend time on complex tools.

Fancy more detail and interview quotations? 👀 See the 5th chapter in my thesis.


Problem Definition

To specify a problem statement, I consulted the opportunity areas with the eDive team and aligned them with the project’s priorities. We narrowed down the focus of this work and selected three problems that cover multiple opportunity areas:

Lobby: How might we improve the first impression of the application?

Controllers and Body-Locked UI: How might we add more functionality while keeping the user interface simple and avoiding cognitive overload?

Desktop Interface: How might we enable teachers to bring their own teaching content to virtual reality?



Prototyping

For each problem, the main developer and I mapped the current state of related eDive’s features and compared it to other existing solutions on the market. Next, I sketched several ideas for each problem. I did not elaborate on every single feature within the prototypes; rather, I focused on establishing a set of user-interface principles and consistent visual language.

I primarily designed horizontal prototypes: my goal was to explore the breadth of the system rather than individual features. The horizontal prototypes will serve as a base for the future development of the eDive platform.

Lobby

The lobby is users’ first touchpoint with the eDive platform. When a user selects the application in the Oculus menu, and it starts up, they first come to a lobby environment, where they can choose their role (student or teacher), set up their display name, and enter the lesson space. Unlike the collaborative lesson space, the lobby is private to each user.

Typically, a lobby consists of three parts: environment, screen interface, and user flow. A basic user flow could include the following steps: a user appears in the lobby, logs in, browses available virtual worlds, selects one of the worlds, reads its description, and enters. Consequently, the user flow affects the design of the screen interface as well as the whole lobby.

In similar VR applications, the lobby's screen interfaces generally differ in size, distance, and layout. Below, you can see some of the common designs:

RecRoom
Oculus Home
AltspaceVR
Gravity Sketch

Prototype

Below, you can see several layouts that I considered; after testing these layouts inside virtual reality (in ShapesXR), I opted for a wide main screen with a narrow side screen on the right side, which is slightly angled towards the user.

This layout could readily fit the required functionality in each step of the user flow. Essentially, the central screen would serve for the main interaction, while the narrow side screen would show the user’s avatar with its settings.

After I designed the first high-fidelity screens, I imported them to ShapesXR. Based on my observations directly in virtual reality, I adjusted some visual attributes of the UI, such as color contrast or text sizes.


Controllers and Body-Locked UI

In the current version of eDive, there is only a world-locked UI: all the controls and tools are attached to objects in the virtual environment. As the platform develops, it will need to accommodate more controls and tools. Some of the new functionality can be placed within the virtual world, but some must always be easily accessible, so it should rather be anchored to the user than the world.

There are currently numerous different approaches to the body-locked UI: handheld tablets, radial wrist menus, holographic HUD interfaces, a triangular prism with menus on its faces, and many more. My main sources of inspiration were virtual reality creation tools and social VR applications.

Open Brush (based on Tilt Brush)
Arkio
Gravity Sketch

Prototype

My first draft of the eDive’s body-locked UI included two main components: a tablet-like menu and a quick tools menu. Each menu was attached to a different controller so that the teacher could hold the tablet (e.g., with lesson notes) in one hand and use the other hand to select and use a tool. During the prototyping process, I iteratively refined the initial idea to meet the requirements.

Quick Tools

The quick tools menu provides quick access to frequently used features: in my designs, there is the 3D pen, pointer, voice note, mute, and open main menu. The set of tools is based on information from the interviews; however, the quick menu should be adjusted in the future according to how the teachers actually use the tools.

I considered several designs of the quick menu. The first design (above) is inspired by Gravity Sketch (gesture-activated menu), the second by AltspaceVR (HUD menu). The latter two designs are based on a radial menu, which the user opens by pressing a button on the controller (B/Y button or the thumbstick).

In the next iteration, I continued with the latter two designs, because they better complied with the accessibility requirements—we wanted to adjust eDive for one-handed interaction.

I continued by drafting settings for the 3D pen tool: during the process, I discovered that the former version of the quick tools menu would be perhaps more universal and applicable to other parts of the UI. With the EduInCIVE team, we tested the following scenario with the floating menu:

  • A user selects the 3D pen tool from the floating menu: they hold the “B” button and point the controller on the 3D pen option in the radial menu floating in front of them.

  • The user starts drawing.

  • The user wants to change the color of the 3D pen: they again hold the “B” button, which opens a context options menu for the 3D pen. There they select a new color by pointing the controller at it.

  • When the user finishes drawing, they exit the pen tool by pointing to the “Exit pen” option.

The interaction felt natural and easy to learn.

Main Menu

The purpose of the main menu is to accommodate functionality that must always be accessible to the user (and thus cannot be placed within the virtual world).

The proposed version of the main menu below includes six primary options (on the left side of the interface): private notes, import content, sticky notes, tools, settings, and notifications. Above the primary options, there is an additional option to detach the main menu from the user’s hand and place it into the virtual space, which gives the user more flexibility and makes the interface accessible for single-handed users.

The interface adapts to its content: while the settings only include a few controls that can fit on one compact screen, the private notes require a larger reading space, and so the interface modularly extends.

The visual style of the body-locked UI matches the high-fidelity prototype of the lobby screen. The two parts of the UI use the same color palette, font, icon set, and other visual elements.


Desktop Interface

There is currently no desktop interface for eDive—the application only runs on a virtual reality headset, and all changes must be made either in VR or in the development environment. Consequently, the teachers cannot edit the educational content without the help of a developer.

In similar applications, there are two types of desktop interfaces. Some social VR platforms allow their users to join virtual events from various devices, including desktop—these desktop interfaces aim to deliver a similar experience as their VR counterparts. In contrast, other VR platforms offer desktop interfaces that support the main experience rather than replace it. Typically, VR creation tools allow their users to import 3D objects from their computers to the virtual environment, giving them more creative freedom and simplifying their workflow.

Landing Pad by Gravity Sketch
Desktop interface of Engage
Desktop interface of ShapesXR

Prototype

After I identified the requirements for eDive’s desktop interface and researched similar solutions, I sketched the main functionality in FigJam. Next, I designed a visual style for the web interface that matches the style of eDive’s virtual reality interfaces. The picture below compares the low-fidelity and high-fidelity prototypes of the private notes.

I continued by designing the remaining parts of the desktop interface. My objective was to outline the structure of the interface and create a horizontal prototype—I did not elaborate on specific functionality, such as the process of editing a set of sticky notes or uploading a presentation.


Future Work

The implementation and evaluation of the prototypes were out of the scope of this work but should be a part of future research. The last phase of the design process is to develop the proposed solution and test it with users. In the case of eDive, the prototypes should be tested not only with the actual users (the teachers) but also in the actual educational setting (during a virtual lesson). The testing will lead to a new design iteration: the design of the platform should be adapted to the users’ needs, developed, and tested again.



If you made it down to this point… Thank you for reading! 🤗 Or scrolling, at least. Next, you can check out the actual thesis.

Any remarks? 💡 I’d be delighted to discuss my workflow, XR design tools, the future of education, life, universe, and everything! Reach me at LinkedIn or via email.