Interactive Language Learning

An exploration of immersive learning using VR and NLP

To read the full paper, kindly visit Paper on IEEE Xplore or Paper on ResearchGate

Introduction

During the final year of my undergraduate degree, I authored this paper titled “Interactive Language Learning with VR and NLP Assistance” along with my teammates under the guidance of our research mentor. The paper was presented in the 2022 IEEE 7th International conference for Convergence in Technology (I2CT) which was a 3 day conference - from the 7th to the 9th of April 2022.

This experience was invaluable to me and has taught me a lot. During the ideation phase of the process, we conducted a market survey to gain a better understanding of how people in our community approach online learning and their opinions of it - especially considering that Covid had made online learning a necessary part of everyone’s life. Though we received varied responses, the general consensus remained that online learning wasn’t as engaging as in-person learning. To find a solution to this problem we began our research and narrowed down our topic to language learning and how Virtual Reality (VR) and Natural Language Processing (NLP) could provide a solution.

Abstract

In the past years, society has evolved exponentially in terms of technology and so has people’s need for growth and learning. Education has encountered a structural redefinition. As companies and governments open up to a hybrid economic and work model, it can be assumed that teaching/learning increasingly rely on a combination of personal and digital learning spaces. Due to the importance of languages from an individual perspective as well as from business and cultural perspectives, it is imperative that language learning also be incorporated into these virtual spaces. Research has proven that immersion is the best method of learning a language. This paper explores the use of virtual reality to provide a more engaging and rewarding experience. VR boasts a number of features that are useful for education: it displays environments in 3D, offers interactivity and even presents the user with audio, visual and haptic feedback. As opposed to videos and other passive learning methods, VR allows for direct interaction with the environment.This learning system can further be complemented with the help of natural language processing (NLP) to broaden the scope of interactions and improve engagement. And these topics have been explored in detail in this review paper.

Prototype Demo

During the course of our research, we developed a prototype application. In the development of this prototype, I used Unity and C# scripting to create interactions and used Python.NET so my C# scripts could interact with the Python code developed by my teammates. In this demo, the user wakes up in their VR bedroom, first chooses to learn using fill-in-the-blank style MCQ conversations and then practices conversations with the chatbot. Both the learn and practice module use the scenario of ordering food at a restaurant as an example.

• 2023 3
• 2022 3
• 2021 1