Assessing Performance of Cloud-Based Heterogeneous Chatbot Systems and A Case Study
Open Access
- 6 May 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 81631-81645
Abstract
Recently, human-machine digital assistants gained popularity and are commonly used in question-and-answer applications and similar consumer-supporting domains. A class of more sophisticated digital assistants (chatbots) employing more extended dialogs follows the trend. Chatbots have become increasingly popular in recent years. Nowadays, chatbot deployments in the cloud have become a common practice because of their benefits, including flexibility, scalability, reliability, security, remote working, cost, and power outages. However, measuring the cloud-based chatbot systems’ performance is challenging as human-machine information exchanges are performed in heterogeneous environments such as cloud hosting platforms, information processing units, and several machine-to-machine and human-machine communication channels. This paper investigates different methodologies for assessing the performance of such heterogeneous deployments and identifies performance metrics for evaluating the performance of cloud-based chatbot deployment. The study employs chatbot performance measurements with both real users (human) and automated (simulated) users. The experimental results discuss communication metrics such as response time, throughput, and load testing (connection loss) through the performance assessment of a case study deployment that utilizes an automated protocol chatbot development framework. The findings presented in this paper can further help practitioners to better understand the performance characteristics of a cloud-based chatbot and assist in making informed decisions related to the chatbot development and deployment options.Keywords
This publication has 35 references indexed in Scilit:
- Metaheuristics for scheduling of heterogeneous tasks in cloud computing environments: Analysis, performance evaluation, and future directionsSimulation Modelling Practice and Theory, 2021
- Performance evaluation metrics for cloud, fog and edge computing: A review, taxonomy, benchmarks and standards for future researchInternet of Things, 2020
- Function-as-a-Service performance evaluation: A multivocal literature reviewJournal of Systems and Software, 2020
- A Comparison of MQTT Brokers for Distributed IoT Edge Computing2020
- Performance Evaluation of IoT Data Management Using MongoDB Versus MySQL Databases in Different Cloud EnvironmentsIEEE Access, 2020
- Performance evaluation of heterogeneous cloud functionsConcurrency and Computation: Practice and Experience, 2018
- Benchmarking Heterogeneous Cloud Functions2018
- Microservices: Yesterday, Today, and Tomorrow2017
- Budget‐constraint stochastic task scheduling on heterogeneous cloud systemsConcurrency and Computation: Practice and Experience, 2017
- Performance evaluation of cloud computing platforms using statistical methodsComputers and Electrical Engineering, 2014