Role-playing VR game simulates discrimination
09 January 2023
Image: Alex Shipps/MIT CSAIL
A new virtual reality experience, 'On the Plane', allows players virtually to walk a mile in someone else’s shoes, to encourage a greater understanding of xenophobia and racism.
Have you ever been advised to “walk a mile in someone else’s shoes?” Considering another person’s perspective can be a challenging endeavour – but recognising our errors and biases is key to building understanding across communities.
By challenging our preconceptions, we confront prejudice, such as racism and xenophobia, and potentially develop a more inclusive perspective about others.
To assist with perspective-taking, MIT researchers have developed “On the Plane,” a virtual reality role-playing game (VR RPG) that simulates discrimination.
In this case, the game portrays xenophobia directed against a Malaysian-American woman, but the approach can be generalised. Situated on an aeroplane, players can take on the role of characters from different backgrounds, engaging in dialogue with others while making in-game choices to a series of prompts.
In turn, players’ decisions control the outcome of a tense conversation between the characters about cultural differences.
As a VR RPG, “On the Plane” encourages players to take on new roles that may be outside of their personal experiences in the first person, allowing them to confront in-group/out-group bias by incorporating new perspectives into their understanding of different cultures.
Players engage with three characters: Sarah, a first-generation Muslim American of Malaysian ancestry who wears a hijab; Marianne, a white woman from the Midwest with little exposure to other cultures and customs; or a flight attendant. Sarah represents the out group, Marianne is a member of the in group, and the flight staffer is a bystander witnessing an exchange between the two passengers.
“This project is part of our efforts to harness the power of virtual reality and artificial intelligence to address social ills, such as discrimination and xenophobia,” says Caglar Yildirim, an MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) research scientist who is a co-author and co-game designer on the project.
“Through the exchange between the two passengers, players experience how one passenger's xenophobia manifests itself and how it affects the other passenger.
“The simulation engages players in critical reflection and seeks to foster empathy for the passenger who was ‘othered’ due to her outfit being not so ‘prototypical’ of what an American should look like.”
“It is not possible for a simulation to give someone the life experiences of another person, but while you cannot ‘walk in someone else’s shoes’ in that sense, a system like this can help people recognise and understand the social patterns at work when it comes to [issues] like bias,” says Harrell, who is also co-author and designer on this project.
“An engaging, immersive, interactive narrative can also impact people emotionally, opening the door for users’ perspectives to be transformed and broadened.”
This simulation also utilises an interactive narrative engine that creates several options for responses to in-game interactions based on a model of how people are categorised socially.
The tool grants players a chance to alter their standing in the simulation through their reply choices to each prompt, affecting their affinity toward the other two characters. For example, if you play as the flight attendant, you can react to Marianne's xenophobic expressions and attitudes toward Sarah, changing your affinities. The engine will then provide you with a different set of narrative events based on your changes in standing with others.
To animate each avatar, “On the Plane” incorporates artificial intelligence knowledge representation techniques controlled by probabilistic finite state machines, a tool commonly used in machine learning systems for pattern recognition.
With the help of these machines, characters’ body language and gestures are customisable: if you play as Marianne, the game will customise her mannerisms toward Sarah based on user inputs, impacting how comfortable she appears in front of a member of a perceived out group. Similarly, players can do the same from Sarah's or the flight attendant’s point of view.
In a 2018 paper based on work done in a collaboration between MIT CSAIL and the Qatar Computing Research Institute, Harrell and co-author Sercan Sengün advocated for virtual system designers to be more inclusive of Middle Eastern identities and customs.
They claimed that if designers allowed users to customise virtual avatars more representative of their background, it might empower players to engage in a more supportive experience. Four years later, 'On the Plane' accomplishes a similar goal, incorporating a Muslim’s perspective into an immersive environment.
“Many virtual identity systems, such as avatars, accounts, profiles, and player characters, are not designed to serve the needs of people across diverse cultures,” they note.
“We have used statistical and AI methods in conjunction with qualitative approaches, to learn where the gaps are.
“Our project helps engender perspective transformation, so that people will treat each other with respect and enhanced understanding across diverse cultural avatar representations.”
Harrell and Yildirim’s work is part of the MIT IDSS’s Initiative on Combatting Systemic Racism (ICSR).
The researchers’ latest project is part of the ICSR’s broader goal to launch and coordinate cross-disciplinary research that addresses racially discriminatory processes across American institutions. Using big data, members of the research initiative develop and employ computing tools that drive racial equity. Yildirim and Harrell accomplish this goal by depicting a frequent, problematic scenario that illustrates how bias creeps into our everyday lives.
“In a post-9/11 world, Muslims often experience ethnic profiling in American airports. ‘On the Plane’ builds off of that type of in-group favouritism, a well-established finding in psychology,” says MIT Professor Fotini Christia, Director of the Sociotechnical Systems Research Center (SSRC) and Associate Director or IDSS.
“This game also takes a novel approach to analysing hardwired bias by utilising VR instead of field experiments to simulate prejudice.
“Excitingly, this research demonstrates that VR can be used as a tool to help us better measure bias, combating systemic racism and other forms of discrimination.”
“On the Plane” was developed on the Unity game engine using the XR Interaction Toolkit and Harrell’s Chimeria platform for authoring interactive narratives that involve social categorisation.
The game will be deployed for research studies later this year on both desktop computers and the standalone, wireless Meta Quest headsets.