V2S: A Tool for Translating Video Recordings of Mobile App Usages into Replayable Scenarios
Authors:
- 1 May 2021
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 65-68
Abstract
Screen recordings are becoming increasingly important as rich software artifacts that inform mobile application development processes. However, the amount of manual effort required to extract information from these graphical artifacts can hinder resource-constrained mobile developers. This paper presents Video2Scenario (V2S), an automated tool that processes video recordings of Android app usages, utilizes neural object detection and image classification techniques to classify the depicted user actions, and translates these actions into a replayable scenario. We conducted a comprehensive evaluation to demonstrate V2S’s ability to reproduce recorded scenarios across a range of devices and a diverse set of usage cases and applications. The results indicate that, based on its performance with 175 videos depicting 3,534 GUI-based actions, V2S is able to reproduce ≈ 89% of sequential actions from collected videos. Demo URL: https://tinyurl.com/v2s-demo-videoKeywords
All Related Versions
This publication has 7 references indexed in Scilit:
- Translating video recordings of mobile app usages into replayable scenarios2020
- Assessing the quality of the steps to reproduce in bug reports2019
- ImageNet classification with deep convolutional neural networksCommunications of the ACM, 2017
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksIEEE Transactions on Pattern Analysis and Machine Intelligence, 2016
- Automatically Discovering, Reporting and Reproducing Android Application Crashes2016
- Auto-completing bug reports for Android applications2015
- RERAN: Timing- and touch-sensitive record and replay for Android2013