NSF REU project at Software Recomposition Lab, NCSA, UIUC. Collaboration with Jodi Byrd. Research assisstant: Nicole Pontaza.

How do players of contemporary video games perceive and process race-based information as part of their gaming experience? What role does that perception play in split-second decision-making processes in terms of character combat, map navigation, and attitudes towards cultural information throughout game environments? This project at the NCSA Critical Technology Studies Laboratory will combine eye tracking of visual attention during game play, data analysis of that attention, and visualization of the results towards a critical understanding of race and cultural difference in video games. We will use the Irrational Games' 2013 AAA title, BioShock Infinite, to examine the effects of both active (e.g., visible race of characters and combatants in the game) and passive (e.g., racial background materials embedded in environments, level designs, and other artistic content) processing of race-based visual data. The game Never Alone—a collaboratively designed game with input from Alaskan Native elders that includes activities and artifacts intended to foreground indigenous cultural difference—will be used to explore the effectiveness of such an approach for creating cultural awareness and learning among gamers. The results of this research will lead to new insights about how the designs of video games affect gamer attitudes towards race and difference, and will suggest new approaches for future game designers aiming to positively affect such attitudes in the future.

We will employ commodity eye tracking hardware and software to capture gamer attention during gameplay. The data produced by these systems requires significant post-processing to understand. Furthermore, such systems aren't designed to temporally synchronize that data with active gaming experiences. Therefore, our students will develop software to process the eye tracking data, synchronize it over time with a video capture of the gameplay, and visualize aspects of the data on top of that video capture. Such visualizations, especially when comparing data from multiple subjects, will illustrate the relationships between race-based content in video games and player perceptions and actions. The software developed will be hosted on the Critical Technology Laboratory's GitHub account, made available via the NCSA Open Source License, and offered to the critical gaming academic community for use and comment. The students will work with Grosser and Byrd on all aspects of the project from critical understandings of race and race-based imagery in video games to consideration of how design of gaming interfaces affects user attention. Outcomes will include data visualization and software development, and the application of this research will include the analysis software itself, visualizations of results, publications of findings, and art exhibitions derived from the research.