How to Learn a Language with Virtual Reality

---

That’s what makes VR such a powerful learning tool — you don’t just get told what to learn. You get an experience. And that makes it all the more memorable.

And that’s why I’m so excited about VR as a tool for learning languages.

In this article, I will summarise the benefits and drawbacks of learning languages in VR, comparing VR to other methods of learning, before going through my favourite VR apps for 2020.

It’s a new decade after all so why not explore a new technology?

What is VR?

VR immerses users in a fully digital environment through a headset or surrounding display. This environment can be computer-generated or filmed with a 360-degree camera. Many readers are probably familiar with the technology, known more for its simulated rollercoaster rides, than its educational apps.

The technology has been called the “next major computing platform” by Mark Zuckerberg, the founder of Facebook. He predicts VR (and Augmented Reality) is set to become as integral to most people's daily lives as smartphone technology is today.

Oculus, a major VR headset manufacturer, was acquired by Facebook in 2014 and has since developed some truly remarkable hardware. Their most recent headset, the Oculus Quest, is a standalone VR headset (no PC, no wires) that gives users access to a wide selection of entertaining games and experiences.

It's also common to slide your phone into a headset and use this for VR experiences. The Samsung GearVR was popular for a few years, but these phone-based headsets are being phased out by manufacturers in favour of more sophisticated devices such as the Oculus Quest.

VR technology is still in its early stages, with a limited but growing user base and a group of dedicated enthusiasts, like myself, who see the potential revolution this technology offers.

What Makes VR Different?

One of my early experiences in VR was with The People's House, a documentary about the White House set in the last few months of Barack Obama's administration. I sat, wearing my pyjamas, listening to Obama's eloquent voice against the backdrop of a gorgeous 360 timelapse from the White House lawn. I could turn and look wherever I wanted to. Suddenly, I was transported inside the building and was sitting two meters away from President Obama himself. He looked into my eyes and shared his experience of working and living in this historic building. I was engrossed, entertained, and embarrassed that I was in my pyjamas while in the presence of a US president!

I had never felt immersion like this before from any video or cinema experience. I was physically sitting in the UK, but my mind believed wholeheartedly that I was at the White House having a one-to-one conversation with President Obama.

This level of immersion is what VR can offer!

Unfortunately, I can only give you a glimpse of the experience through my words — you have to try it to fully appreciate it.

This immersion makes VR a perfect tool to accompany your language learning. Immersion in your target language through realistic scenarios, would normally only be possible by going to the country and is a great tool for learning quickly and effectively.

Why Learn in VR?

VR has been shown to improve students’ test results by providing a more meaningful and memorable learning experience. Schools and corporations are increasingly turning to VR to develop experiences that give direct scenarios that students or employees might encounter, and which are normally impossible to simulate in a training environment.

VR trainees can practise their required skill, whether it's public speaking or medical surgery, in a virtual environment that closely resembles the real thing. These simulations can create the anxiety and emotions one can feel when carrying out the activity for the first time, but without the consequences.

How many language learners freeze up and become anxious when first speaking a foreign language to a native speaker? I know I have.

There's a big difference between speaking out loud during a pause in a podcast and actually replying to a question asked by a vendor in a busy market when you first arrive in a foreign country.

This gap in experience can be bridged in VR with authentic and relevant simulations.

The simulations available in VR range in intensity. Looking someone in the eyes (it really feels authentic in VR) and experiencing the sights and sounds of a busy environment can provoke the same overwhelming feeling that you feel when first speaking a foreign language. On the other hand, a poorly animated and difficult to use VR simulation will only frustrate you and cause you to take off the headset, having learned nothing. In these early days of VR, there are still developments needed to make this a truly transformational teaching technology.

Some apps can check pronunciation using voice recognition, but the reliability of this is still far from perfect. A real human teacher will probably always be better. However, the voice recognition may be ”good enough” for some learners and the machine learning systems are steadily improving.

What's the Difference Between VR and Other Language Apps?

VR is a lot less convenient than most mobile apps. There are VR headsets you can slip on without connecting to a PC, like the Oculus Quest or lower-priced Oculus Go, and these reduce the friction of getting into a virtual environment. But putting a headset on is a lot harder than thumbing through Duolingo on your phone.

Five minutes queuing at the bank is better spent with Duolingo than putting a VR headset on… and you'll get a lot fewer stares!

VR is more useful for those focussed periods of study when you want to be transported into a realistic foreign environment, and really push yourself in challenging simulations. You are fully engaged with the environment that surrounds you, with no distraction from other apps or the internet in general. This makes your time in VR much more focussed than your time spent using most mobile or desktop software.

The 3D sensory experience that surrounds you in VR activates your wonder and curiosity, just like the feeling you get when exploring a foreign country. And the best part is, you don't have to travel for hours to get this immersion — you can do it from the comfort of your home!

Not all VR language learning apps are created equal, just like mobile apps aren't suited to every style, so choosing the right one is essential when getting started.

As of 2020, I see the various VR language learning apps as fitting one of three styles:

• Animated Worlds
• Social Scenarios
• Simulated Real-Life Encounters.

The next part of the article will go through the different types of apps, detailing the benefits and disadvantages of using them.

Learning a Language with VR: Animated Worlds

These are apps that place the VR user in an animated world, where they are taught vocabulary by cartoon characters, often with robotic voices.

There are a few apps available, including House of Languages and Busuu, but these are limited and don't offer much to your average user. They are free though, so worth checking out if you're on a budget!

Mondly, with its popular mobile app and desktop site, was one of the first companies to build a VR language learning app in 2017. It is frequently updated and has the widest range of languages compared with any other VR app.

You choose your native language and your target language and it has three sections:

• Vocabulary
• Conversation
• Multiplayer

Vocabulary places you opposite an animated lady who mainly speaks in your native language and then tells you the word you're learning in your target language. Seeing an animated elephant squirt water from its trunk is entertaining, but spending ten minutes on eight words is not the most efficient way to learn. I'd also rather the whole experience was in my target language, or at least had the option to change this, but currently, this isn't possible.

The main issue is the mixture of real and robotic voices depending on the exercise. This can feel unnatural and doesn’t have the expressiveness required for language learning. I also noticed a couple of mistakes in pronunciation made by these robot voices. The voice recognition was not great either, and didn't even recognise my native English at times.

The Conversation section puts you in a scenario, such as a hotel reception or restaurant and you interact with an animated being. You can set the difficulty of the encounter, which changes whether you see the “Things you can say” box or not. There is a set of responses you can give, which is fine for practise but doesn’t capture the true richness of a foreign language encounter. For example, when the taxi driver said my ride was $20 and I replied, “here's $100, keep the change”, he didn’t understand me because it wasn’t one of the set responses. But I'd expect even a fictional taxi driver to be grateful for such a generous tip! Note: I'm only this generous with my virtual money!

MondlyVR is a great tool to practise with but it isn’t something essential for language learners… Yet. It is great for someone who is going on a short trip away and wants to learn a few basic phrases in the local language to help them with some standard activities.

The Multiplayer section of MondlyVR is in Beta, placing you in a room with avatars of other real people and learners of the language. I couldn’t find anyone to speak to in either Spanish or English – I suppose there just aren't enough people using the app for this to be useful yet. However, this SocialVR aspect brings us nicely to the next type of app I'd like to consider.

Social Scenarios in VR

The Silicon Valley start-up, Immerse, offers a VR play space with various environments including airports, offices, and an average American home. For now, it has English lessons only and is tailored towards businesses, mainly in Japan and Korea.

I went through a tech demo with the Immerse team, who explained how it works. Multiple students in VR login to a specific teacher's area. The teacher is trained on the platform using a desktop computer, so they can lead a class through the tailored lesson plans for each of the scenarios. Students can watch interactions between animated characters (with recorded human voices, not robots!) and the teacher can explain any new vocabulary or questions the students might have. The VR space allows for more focussed learning with fewer distractions than normal online lessons.

The VR world is beautifully designed and very intuitive, and having curated VR spaces for interaction in your target language in teacher-led scenarios is a fantastic idea.

It is only available for English students from Asia who can arrange a free consultation.

However, there are other open VR platforms where you can interact with avatars of real people and speak to them. These apps are called SocialVR and the most popular platforms are VRChat, AltspaceVR, and Rec Room. These virtual spaces are not tailored towards language education, but communities of language learners could meet in standard or custom-built areas to practise their languages.

If anyone does this, please get in touch! I'd love to hear about it!

Simulated Real-Life Encounters

The final type of VR experience is one where a scene has been filmed using a 360 camera and the student feels like they are talking to a real person, not an animated character. I find these experiences the most immersive in VR, like the Obama meeting that I mentioned earlier.

ImmerseMe has several 360 scenarios that seem like they'd be very fun to practise with in VR. However it currently has no VR support, as of early 2020, so I can't consider it as a VR app.

Panolingo has a mobileVR app for teaching English, using 360 panoramas. This is a great way to learn English but the lack of an App for any major VR headset is a real limiting factor.

Finally, there’s the app my team at Morning Calm Productions has been developing.
We've created Dynamic Spanish, which utilises an immersive 360 environment to teach you Spanish from scratch. The VR-led course is supported by worksheets, vocabulary recall games, listening activities, and transcripts outside of VR, too.

Dynamic Spanish is available for Oculus headsets and GearVR, and involves a series of 360 video lessons. You prepare by learning and practicing your speaking and listening skills in a fully immersive classroom before you step out onto the streets of Valencia to engage in roleplay-based encounters.

The Dynamic Spanish Starter Pack is for beginner Spanish learners to start their language learning journey.

Summary

VR is a new technology that offers a lot of promise in the self-taught education space. It can place users in simulated scenarios so they can practise their languages in realistic environments.

There are no killer VR apps for language learning yet, but as the technology becomes more widespread, this new decade could see the start of full language immersion from the comfort of your living room

It's a very exciting time for language lovers!

Please share any experience you have using VR to learn languages below.