Beyond Playing to Win: Gameplaying Agents Provided with Distinct Behaviours
My work focuses on expanding the research on general game-playing agents beyond the objective of winning at them. It covers the following approaches:
1) Broadening the scope by diversifying agents goals and heuristics
2) Broadening the vision by proposing a team of agents to assist game development
3) Broadening the usage by eliciting diverse automated gameplay
4) Broadening the horizon by analysing the strengths of the agents from a Player Experience perspective instead of their performance.
Games have had a big role in the improvement of AI and this work can return the favour by expanding the use of general agents by providing them distinct behaviours, related to different types of players and ways to interact with the game.
The ultimate goal of my research is to have an impact in games and provide methods to assist on their development.
2021
Cristina Guerrero-Romero and Diego Perez-Liebana. “MAP-Elites to Generate a Team of Agents that Elicits Diverse Automated Gameplay”, Proceedings of the 2021 IEEE Conference on Games (CoG), 2021.
Diego Perez-Liebana, Cristina Guerrero-Romero, Alexander Dockhorn, Dominik Jeurissen and Linjie Xu. “Generating Diverse and Competitive Play-Styles for Strategy Games”. Proceedings of the 2021 IEEE Conference on Games (CoG), 2021.
2020
Cristina Guerrero-Romero, Shringi Kumari, Diego Perez-Liebana and Sebastian Deterding. “Studying General Agents in Video Games from the Perspective of Player Experience”, Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE), 217-223, 2020.
2019
Damien Anderson, Cristina Guerrero-Romero, Diego Perez-Liebana, Philip Rodgers and John Levine, “Ensemble Decision Systems for General Video Game Playing”, Proceedings of the 2019 IEEE Conference on Games (CoG), 1-8, 2019.
2018
Cristina Guerrero-Romero, Simon M Lucas and Diego Perez-Liebana, “Using a Team of General AI Algorithms to Assist Game Design and Testing”, Proceedings of the 2018 IEEE Conference on Computational Intelligence and Games (CIG), 1-8, 2018.
2017
Cristina Guerrero-Romero, Annie Louis and Diego Perez-Liebana, “Beyond Playing to Win: Diversifying Heuristics for GVGAI”, Proceedings of the 2017 IEEE Conference on Computational Intelligence and Games (CIG), 118-125, 2017.
This demo shows the different behaviours of a general agent (Random Search) when using four different heuristics (goals): Winning Maximization (WMH), Exploration Maximization (EMH), Knowledge Discovery (KDH) and Knowledge Estimation (KEH).
The game is one of the games provided in the GVGAI framework, Butterflies. To win the game, all the butterflies must be captured before the time runs out or before all the cocoons are opened. Catching butterflies increases the score +2, so the more butterflies captured, higher the final score is.
Winning Maximization Heuristic (WMH). The goal is winning or, if not possible, maximising the score. In the beginning, the agent stays around the same area without moving much because the butterflies, which gives the score, are too far to be aware of them. At the moment one of the butterflies is close enough to be noticed by the agent, it is clear how it moves to catch it and, as a result, the rest of the butterflies around. The agent manages to capture all the butterflies and win the game.
Exploration Maximization Heuristic (EMH). The goal is visiting as many different positions on the map as possible. In the video, the spots already visited by the agent are highlighted. The agent moves all the time until the map is completely visited, not caring about the butterflies around.
Knowledge Discovery Heuristic (KDH). The aim, in this case, is interacting with the game as much as possible. This heuristic is focused on maximising the discovery of the different sprites present in the game, and trying to perform interactions with all of them. If no new sprites emerge in the next state reached, it prioritises carrying out new interactions. Ultimately, if no new interactions are at their disposal either, the heuristic rewards interactions that have occurred in the past, but if they happen now in different locations of the level. In this game, there are only two elements to interact with: trees and butterflies. The agent moves around the map close to the trees, interacting with them in different positions. It rarely covers the big spaces in the middle of the map, unless there are butterflies around, as it tends to interact with them as well.
Knowledge Estimation Heuristic (KEH). The goal of this heuristic is to predict the outcome of interacting with sprites, both changes in the victory status and in score modifiers. Unlike with the Knowledge Discovery Heuristic, it does not try to interact with elements based on their position but aims for a uniformed interaction with all the elements in the game. At the end of the game, estimations with each of the elements are calculated, trying to predict the score change resulted for each of them and the possible outcome (win/lose) of the game. For the trees (id 0), it is not predicted score change at all (and there is not in the game). For butterflies (id 5) it is predicted a score change of 2, which is accurate.
Detailed information about the heuristics and their implementation can be found in the paper “Beyond Playing to Win: Diversifying Heuristics for GVGAI”.
Feel free to drop me a line to c.guerreroromero[at]qmul.ac.uk