Game Jam brings Alberta students together to build accessible brain-controlled video games for kids with CP

Mind control is something straight out of science fiction, but a new technology, brain-computer interfacing (BCI), makes it possible for people to control video games using just their brains.

"Brain-computer interfaces allow individuals to interact with computers, video games or robots using only their brain signals through electroencephalography (EEG). EEG uses small electrodes on the surface of the scalp to read electrical activity of the brain.

The BCI then takes that brain activity reading and, based on known patterns of brain activity, attributes it to a desired action," explained Kim Adams, associate professor with the University of Alberta's Faculty of Rehabilitation Medicine. Adams is also a professional engineer and her research and teaching at the faculty focuses on assistive technology for rehabilitation applications.

With the BCI technology in mind, a group of game developers, scientists, engineers and clinicians driven to create access to gaming for all children organized the Brain-Computer Interface (BCI) Game Jam. The hackathon event, which was held November 8-10, 2019 in both Calgary and Edmonton, was meant to inspire the creation of accessible video games for children who may never have been able to play one before, including those with quadriplegic cerebral palsy or other severe neuromuscular disorders.

The effort was led by Eli Kinney-Lang, a postdoctoral researcher with the University of Calgary who works at the Alberta Children's Hospital, with the main events held in Calgary. The Edmonton-based events were hosted by the I CAN Centre for Assistive Technology at the Glenrose Rehabilitation Hospital, who also provided occupational therapy and engineering support. Adams led the faculty's partnership in the event and recruited University of Alberta students and staff to participate, including computing science students from the Faculty of Science's Department of Computing Science, members of the Bionic Limbs for Improved Natural Control (BLINC) Lab and her graduate student Matheus Mussi, who is working towards a Master of Science in Rehabilitation Medicine degree at the Faculty of Rehabilitation Medicine.

"It was a good experience in terms of learning the challenges of game design for the kids who need to use brain-computer interfaces (BCI). Making games both fun and simple is difficult," said Adams.

The event kicked off with some live-streamed technical instruction sessions and a panel of three kids and their families in Calgary who have previously tried BCI games. The families shared their experiences and gave advice about what they would like to see in games to play using BCI.

For the rest of the weekend, teams worked on their games. They had until Sunday, November 10 at 11:59 p.m. to complete their games and submit them for judging.

"This was my first experience programming games, and I'm working with brain-computer interfaces [in my research], so I was curious about how we can connect games and this technology and how it might be used in research," said Mussi.

His team built a simple, 'endless runner' type of game, in which a stick man has to jump over obstacles.

"During the levels, several types of obstacles appear and what the kid (or user) has to do is focus on which action he/she wants to execute. All the commands are on the screen whenever the character enters the 'action zone'. Thinking about the children, I also implemented a slow-motion region, in which time slows down and they have more time to take action in case they need it," explained Mussi.

"I learned a lot from this experience, for sure! First, a little bit of Unity programming, which got me thrilled because now I am familiar with the basics. Second, it was neat to see parents and children talking about their interest in games and how to make them in a way children get more engaged," said Mussi.

Another team of UAlberta computing science students made a game in which the user knocks over cakes with a happy ball.

A third UAlberta team was comprised of staff, students and volunteer members of the BLINC lab and led by Michael (Rory) Dawson, lead research engineer at BLINC Lab from the Faculty of Medicine & Dentistry, Division of Physical Medicine & Rehabilitation Medicine. They developed three different games to be played on a physical or virtual version of their robotic arm, the Bento Arm.

"The first game was called 'Bento Buddy Rescue' and involved having the player pick up a ball and transport it from one cup to another with the goal of going as fast as possible. The second game was called 'Bento Bowling' and involved knocking off balls arranged in towers around the play area, without knocking over the towers. Points were awarded depending on how many balls were successfully knocked over and subtracted depending on how many towers might have fallen over.

The third game was called 'Bento Maze Madness' and involved navigating the arm through from a starting point to an end point in a 2-D maze with the goal of doing it as fast as possible. This was actually the only game we had time to implement in the virtual environment in the time allotted, but eventually we hope to implement all of the games on the virtual system in order to be more accessible to players that do not have access to the physical arm," said Dawson.

"The event was really fun and we all had a blast! It was also a great opportunity for us to all sit down and solve a focused problem together. We look forward to making more games in the future."

Event sponsors and partners included the Calgary Pediatric Stroke Program, the Alberta Children's Hospital, Alberta Health Services and the Glenrose Rehabilitation Hospital, University of Alberta's Faculty of Rehabilitation Medicine, University of Calgary's Faculty of Science, Schulich School of Engineering, VRKade (based in Calgary) and Insert-Coin (based in Zurich, Switzerland).

The games are now in the process of being tested and evaluated by children with disabilities as well as experts in the field and prizes will be awarded for the top submissions in early 2020.