Members of the ADL Next Generation Learner Team are currently studying how video games can be leveraged for learning and understanding cognitive capabilities. We recently studied the effects of playing a game with certain design characteristics, and of amount of game play experience in general, on cognitive adaptability, a skill that involves responding to uncertainty and to changing circumstances and monitoring one's own thinking processes. The study used Portal 2, a game developed by Valve Software (http://www.valvesoftware.com/), for its perceived inclusion of the five design characteristics (implicit rule sets; implicit shifting of rules; dynamic, shifting environments; implicit reinforcement of actions to achieve a goal; and relatively open-ended gameplay) proposed to help improve cognitive adaptability. But in order to understand empirically whether Portal 2's design really fits those five characteristics, as well as capture and understand the cognitive processes of Portal 2, the team is performing what is known as a cognitive task analysis (or CTA) using several expert Portal 2 players.
Cognitive task analysis is a technique that has traditionally been used by researchers and industry professionals to capture both the behavioral and cognitive processes and activities that go into accomplishing a task at an expert level. This includes decision-making processes, recognizing and responding to critical cues, responding to environmental conditions, utilizing tools, performing smaller sub-tasks, and analyzing and altering one's own performance. While CTA has been applied to educational game design in some ways, such as Boyle et al.'s (2012) study which utilized CTA to aid in the design of a digital game to support research and statistics education, very little cognitive task research has been done on the cognitive and behavioral performances of expert gamers or the cognitive requirements of existing games.
To help bridge this gap, we on the games sub-team of the Next Generation Learner Team are observing expert Portal 2 players—those who have played through the entire game multiple times—playing through each chapter and levels in the game. Using a talk-aloud protocol consisting of a combination of real-time narration of the players' actions and thoughts throughout gameplay and after-action interviews centered around completed recorded (FRAPS recordings) levels, we hope to clearly map out a clear cognitive and functional picture of each level. This includes determining what affordances and micro-puzzles are present in each level, as well as what manual steps, cognitive steps, and requisite knowledge are required in order to complete the level. Additionally, after each level, we are conducting a post-play interview to examine what rules are present and if, how, and where they change; what new or altered environmental features arise in each level; what feedback or cues informed the player's decisions (whether implicitly or explicitly); and what, if any, constraints to successful gameplay the player perceived. We are analyzing player feedback in this regard to pinpoint the presence of the five game design characteristics listed above that are thought to influence adaptability in thinking as well as the qualitative decision making required to work through instances of these design characteristics.
We anticipate that by performing a CTA on Portal 2, we will be able to pinpoint both what expert performance looks like cognitively and how the game's design features influence player cognition during gameplay. This is essential for understanding the cognitive impact of gameplay and game design, a key component of the growing effort to leverage games for educational purposes. The exercise also is perceived to be a groundbreaker on developing a framework and set of best practices for performing cognitive task analysis on games in the future.
Boyle, E., van Rosmalen, P., MacArthur, E., Connolly, T., Hainey, T., Johnston, B., Ger, P. M., Manjon, B. F., Karki, A., Pennanen, T., Manea, M., & Starr, K. (2012, October). Cognitive Task Analysis (CTA) in the Continuing/Higher Education Methods Using Games (CHERMUG) Project. Paper presented at the 6th Annual European Conference on Games-Based Learning, Cork, Ireland, UK.