Frank Dignum

Until recently the game industry heavily relied on manual playtesting to test the games it produces. Even if the benefits of introducing automated testing are acknowledged, it is rarely done in practice. Some of the main hurdles include the lack of automated testing tools that can target computer games as well as the complexity of automated game plays which are much more difficult to program than typical simple test sequences. This article presents an agent-based testing framework called aplib that comes with a Domain Specific Language (DSL) that allows complex playtests to be programmed more abstractly. A so-called goal structure is used to abstractly formulate a playtest scenario in terms of main goals and their decomposition into subgoals. Scenarios that are not too complicated can be formulated using static goal structures. More complex scenarios may need a test agent that can dynamically adapt its play according to the situation that evolves during the play. To handle such cases, aplib allows dynamic goals to be expressed as well. Invariants and pre-/post-conditions are used to assert the properties that a play is expected to satisfy. They include differential properties that allow constraints on the current state to be related to that of past states. Three case studies are included in the article. The first one aims to evaluate the performance of playtests programmed with aplib . The second shows that the approach can also be combined with other automated testing approaches, in this case reinforcement learning. The third shows the applicability of such playtests in a 3D setup and for non-functional testing.

Modern computer games have become very complex, so they can benefit from automated testing. However, their huge and fine grained interaction space makes them very challenging for automated testing algorithms. Having a model of a system would greatly improve the effectiveness of a testing algorithm. However, manually constructing a model is expensive and time‐consuming. This paper proposes an online agent‐based search approach to solve common testing tasks for computer games, in particular games that involve elements of world navigation and exploration. On the fly, the approach also constructs a model of the system, which is then exploited to solve the given testing task. The effectiveness of the approach is studied via a case study called Lab Recruits and its simulation of another game called Dungeons and Dragons Online. The study showed that the approach is superior in its ability to complete testing tasks and its completion time compared to evolutionary algorithm, Q‐learning and MCTS. This paper extends a previous work presented in ATEST by including evaluation on large game levels, evaluation of the achieved coverage and fault detection and the aforementioned comparison with other algorithms.