Challenge

The language learning market is saturated with many resources for beginners. However, many intermediate language learners—people who are already adept at reading and writing a second language—often struggle to improve their speaking ability because of lack of confidence and limited availability of native speakers. For my graduate capstone project for the MHCI+D program at the University of Washington, I worked with a group of three others to improve this experience.

Solution

We followed a user-centered design process to frame the problem, conduct research, and design a solution. The result is Dover, a language learning platform that pairs speaking practice with video streaming services like Netflix and Hulu.

Dover provides learners with access to a library of episodes in the language they are learning and allows them to save and practice speaking any line. By practicing privately in their own home, learners can improve pronunciation and build confidence. The video format encourages repetitive practice and provides context around phrases in ways that traditional language exercises do not.

Role

I worked collaboratively with my group on research, framing, and ideation. As the design lead, I worked specifically on UI wireframes, task flows, visuals, prototypes, and branding. I also conducted qualitative research through subject matter expert interviews, analyzed the research data, and served as director and photographer of our concept video.

Process

Our process spanned four months. The earlier stages were focused on conducting research, while the later stages involved design and validating our design through prototyping and testing.

Secondary Research

Literature Review

From our secondary research, we learned that while most people in the US have attempted to learn a second language, a much smaller portion self-report actually speaking a second language very well.

We found that barriers for improving speaking include lack of confidence and motivation. Unlike immersion, where you have to learn the language to get by, improving your speaking when you can’t visit areas the language is spoken in requires much more motivation.

While a majority of those in the US have attempted to learn a second language, only 1 in 4 American adults self-report speaking a second language. Of those 25%, only 43% said they can speak that language ‘very well.’Source: Pew Research

Competitive Analysis

After doing a competitive analysis of language learning products, we plotted those focused on speaking on this framework. We placed them on a spectrum of academic and entertainment, and also on dependent (things that require instructors or language partners) and independent. There was a gap in the bottom right hand corner, which helped us narrow our focus.

Primary Research

Targeting intermediate learners trying to become fluent, we framed our main research question:

How can we help intermediate language learners with limited access to native speakers improve their speaking?

To answer this question, we conducted subject matter expert interviews with language experts, did an observational study at a language meetup, and also conducted many semi-structured interviews with our target users, intermediate language learners.

Insights

Several insights came out of our research―we learned about pain points around speaking and remaining motivated when there is limited access to native speakers. We mapped these findings to requirements for our design.

Ideation

We then moved on to ideation, first translating our insights from research onto sticky notes and grouping them based on affinity. The resulting topics helped us understand which features would be important to include in our design. We used this to inform our user journey map and storyboards.

Affinity diagram

User journey mapping

Storyboarding

Concept

We landed on a concept that came out of insights from our research: Intermediate language learners will frequently consume content in the language they are trying to learn; for example, they will watch TV shows and movies. However, they also struggle with confidence when speaking the language in public. The following storyboard outlines our initial concept.

In the scenario, the user has access to a large library of videos in their target language. While watching the videos on their own timing and in their own home, they can select lines to echo (a proven language learning method), and instantly see feedback on their pronunciation. They also have the option to record their own “dubbed” version of the video, replacing the dialogue with their own voice, and share it with their friends.

How it works

We broke the experience down into three modes. In watch mode, the learner is watching the video in the second language. In practice mode, they are speaking and getting feedback. And in performance mode, they can make their own “dub over” clip and share it with friends.

Task flow

We mapped out the interactions and determined what functionality is needed at each stage. Based on this, we created a task flow with branching logic.

Prototoype

In order to test our idea with users, we created a functional prototype, which we built on Premiere Pro, so we could import closed captions, place dynamic markers in the video timeline, and record audio directly onto the video.

We customized a mobile application called Unified Remote, which we mapped to Premiere keyboard shortcuts, and our participants used this to control the video.

Testing

We did a usability test with four language learners, and asked them to complete various tasks out loud. Some of our takeaways from this were:

• At first, participants felt nervous recording and listening to their voice, but after trying it, they found it to be fun and engaging.
• Because participants’ attention was split between the phone and TV, there were moments of frustration when they pressed the wrong button or missed part of the show.
• Participants felt some uncertainty over whether a swipe left or right should move the video forward or backward.

All of these things helped us to iterate on our design.

Wireframes

After determining the information architecture of the app, we created wireframes for each screen.

UI Flow

We also created a UI flow to demonstrate the steps a learner would go through in practice mode.

Visual Design

I worked on the visual design, creating the logo, color scheme, and a basic style guide for our content.

UI Detail

Setup

1. Sync to your content streaming accounts
2. Set up a weekly goal
3. Browse video library
4. Watch on your phone or project to TV

Watching Mode

1. View the dialogue in multiple languages as you watch
2. Navigate the dialogue by swiping left and right
3. Select a line to echo, which you can practice with now or later

Practice Mode

1. Listen to the dialogue
2. Speak into the phone and copy the dialogue
3. Receive feedback and unlock performance mode

Performance Mode

1. “Dub over” the scene by recording your own voice
2. Save the dubbed over video to your collection
3. Share with friends and invite them to dub over other parts of the scene

Video

We created a product demo video to illustrate our proof of concept in a scenario. We showed this as part of our final capstone presentation (a video of the full presentation can be viewed here).

Future work

Our initial user testing showed that Dover has the potential to help language learners improve their speaking in an engaging way, but our prototype was focused specifically on English-speaking users learning Spanish. If we were to continue working on Dover and incorporate more languages, more user testing would have to be done on a much broader scale to better understand how people across the globe learn languages.

In addition to this, we would take the following considerations into our design:

• Incorporate and test a more obvious remote functionality (to accommodate regular watching)
• Investigate options for voice and gesture control
• Conduct a longer term study to measure retention of knowledge and user satisfaction