Towards Multi-Platform Mutation Testing of Task-Based Chatbots
Authors:
- 21 October 2025
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 205-208
Abstract
Chatbots, also known as conversational agents, have become ubiquitous, offering services for a multitude of domains. Unlike general-purpose chatbots, task-based chatbots are software designed to prioritize the completion of tasks of the domain they handle (e.g., flight booking). Given the growing popularity of chatbots, testing techniques that can generate full conversations as test cases have emerged. Still, thoroughly testing all the possible conversational scenarios implemented by a taskbased chatbot is challenging, resulting in incorrect behaviors that may remain unnoticed. To address this challenge, we proposed Mutabot, a mutation testing approach for injecting faults in conversations and producing faulty chatbots that emulate defects that may affect the conversational aspects. In this paper, we present our extension of Mutabot to multiple platforms (Dialogflow and Rasa), and present experiments that show how mutation testing can be used to reveal weaknesses in test suites generated by the Botium state-of-the-art test generator.Keywords
All Related Versions
Funding Information
- MUR (2020W3A5FY)
This publication has 17 references indexed in Scilit:
- Towards the Assessment of Task-based Chatbots: From the TOFU-R Snapshot to the BRASATO Curated Dataset2025
- Wodel-Test: A model-based framework for engineering language-specific mutation testing toolsSoftwareX, 2025
- Test Case Generation for Dialogflow Task-Based Chatbots2025
- A Review of Quality Assurance Research of Dialogue Systems2022
- Ontology-based metamorphic testing for chatbotsSoftware Quality Journal, 2021
- Research Progress of Flaky Tests2021
- Chatbots: History, technology, and applicationsMachine Learning with Applications, 2020
- Testing Chatbots with Charm2020
- Testing Chatbots Using Metamorphic Relations2019
- Assessing the Robustness of Conversational Agents using Paraphrases2019