Small-scale Studies to Measure the Impact of Educational Games

The MEMENTOES project aims to create video games which teach users about history and encourage the development of empathy for the victims of past and present injustices. In order to do so, we need to understand what elements make certain games effective educational tools. As such, we designed a study to gain insights into how video games influence young people’s attitudes.  

The evaluation was organised by dividing the students into two groups. The first group played “Path Out” from the Austrian studio Causa Creations, which tells the story of Abdullah Karam, a young Syrian artist who escaped the civil war in 2014. The second group played “Beecarbonize” from the Czech studio Charles Games, which focuses on survival strategy in the context of climate change. The study had a sample size of more than 50 players and was conducted by the War Childhood Museum team in Sarajevo in November 2023. Participants were selected based on their knowledge of English and had no prior exposure to the tested games.  

Specifically, the study assessed how much video games can contribute to changing opinions about important contemporary issues, in this case refugees, by comparing the attitude change towards refugees expressed by students who played “Path Out” (which addresses the experiences of refugees) to those who played “Beecarbonize” (which does not address refugee-related themes). The study employed two approaches to measuring players’ attitudes. The first focused on explicit attitudes, which can be measured through questionnaires asking participants whether they agree or disagree with a given statement. The second approach measured participants’ implicit attitudes, what we might call the “gut feeling.” This is measured through less direct means, such as reaction-time tests.   

This study is part of a broader element of the MEMENTOES project focused on the evaluation of games. Within this framework, the other two museum partners, Le Bois du Cazier and Gulag.cz, also conducted testing sessions in order to measure game quality, as well as historical awareness and empathy. The results of these studies are not yet published, but will feed into the development process for the MEMENTOES games. 

 As we progress through the MEMENTOES project, larger studies will be conducted to evaluate the three games being developed by project partners. Therefore, the main goal of the current small-scale studies is to assess the effectiveness of the MEMENTOES game quality evaluation methodology and the tools being used. By doing this, we aim to refine our research design and the organisation of future testing sessions.  

Although the study focused strictly on measuring attitude changes, it also revealed some unexpected insights about players’ thoughts on games with social messages. Mia Babić, one of the testing coordinators, said “The students were dedicated and emphasised the importance of games focusing on socially relevant topics.” This reflects the potential impact of combining thoughtful, empathy-inducing cultural lessons with entertainment media. MEMENTOES aims to do this scientifically, by conducting studies like the ones explained in this blog and publishing our findings to enable other game developers and cultural heritage organisations to do the same. 

 

Author: Fernanda Flores, War Childhood Museum

Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the Research Executive Agency. Neither the European Union nor the granting authority can be held responsible for them.