You really have been learning with portals.
Val Shute studies the psychology of education at Florida State University. She also loves Portal 2 – a mind-bending, physics-based adventure from Valve Corporation, the creators of Half-Life, Counter-Strike, and Left 4 Dead. So it was natural for her to want to design a study featuring the game. But to see if the problem solving in Portal 2 was really paying off, she would need a comparison. Shute and her team chose Lumosity, a popular “brain-training” software.
“Portal 2 kicks Lumosity’s ass,” Shute told Popular Science.
Reporting in a study in the journal Computers & Education late last month, Shute and her colleagues pitted Lumosity against Portal 2 in a study designed to find out whether extended periods of play of either game would produce cognitive benefits. More specifically, Shute wanted to see if test results on problem solving, spatial skill, and “persistence” questions would increase after gaming in either situation, and see if those results were significantly different from each other.
77 college undergraduates were recruited into the study and were randomly assigned to either play a total of eight hours of Portal 2 or Lumosity (spread across four sessions). To compare the results, Shute had the students complete the problem solving, spatial skills, and persistence questionnaires both before and after gaming.
In every way, Portal 2 players improved more on the cognitive tests than Lumosity players. Those that completed the brain training games even did slightly worse on the cognitive ability questions than they did before trying Lumosity.
The Game Is A Lie
Before you pull your kids out of school to try and figure out those darn light bridge levels with you, there are some caveats to Shute’s study. First, the sample size was small. For the relatively minor increases in cognitive performance the study found, it’s hard to put real weight behind those conclusions with only 77 people tested. Second, the study session wasn’t very long. It’s possible that the supposed benefits of Lumosity come after eight hours of play, while you could finish all of Portal 2 in that time.
And I wonder if the low results from Lumosity were from the other student participants knowing that they weren’t getting to play one of the best games of the last decade.
But Shute’s research corroborates what we’ve already found in gaming-related science in two important ways: video games do seem to have positive effects on the brain, and so-called “brain training” games don’t work.
For all the flashy marketing, the claims of brain training regimens like Lumosity have not lived up to the hype. The research that we have doesn’t back up brain training in general and Lumosity in particular. The big business of brain games is quickly bounding over and beyond what the science says.
“Contrary to media claims that Lumosity improves specific problem solving skills,” Shute and colleagues conclude, “[the results] show that while there were increases in scores from pretest to posttest for those in the Portal 2 group, all three of the problem solving test scores slightly decreased for the Lumosity group, from pretest to posttest.”
Aside from providing another nail for brain training’s coffin, Shute’s Portal 2 study does point out something interesting – if playing a problem-solving game actually can increase your cognitive performance in some way, what are brain training games missing, and what is Portal 2 getting so right? Is it a combination of puzzles with humor and gorgeous visuals? Is it immersion or extended periods of play? Those are questions for another study to answer.
I’m just GLaD I can make a note here: Huge success.
Kyle Hill is the Chief Science Officer of the Nerdist enterprise. Follow on Twitter @Sci_Phile.
IMAGES: Screenshot from the game Portal 2; Valve Corporation