On the Experimental Biases in User Behavior and QoE Assessment in the Lab
Open Access
- 1 December 2016
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-6
Abstract
User behavior is one of the key components of customer engagement and abandonment, which result from a good or bad Quality of Experience. However, methods to evoke and measure user behavior are still understudied. This paper presents an in-depth look at a study in which we measured user behavior during video streaming consumption in a controlled laboratory environment. We confronted subjects with typical streaming problems such as stalling and quality fluctuations. The subjects were not informed about the real purpose of the test; their behavior was tracked unobtrusively. The results suggest that the method can elicit responses to the inserted problems, such as seeking, pausing, or reloading the web page. However, a third of the subjects acted apprehensively, meaning that they changed their behavior due to being part of a test. In this contribution, we elaborate on the underlying reasons for those experimental biases. We discuss the suitability of different test designs for behavioral assessment and give guidelines on how to quantify and combat biasing factors introduced by the test procedure.Keywords
This publication has 10 references indexed in Scilit:
- (Re-)actions speak louder than words? A novel test method for tracking user behavior in web video services2016
- QoE matters more than QoS: Why people stop watching cat videos2016
- Factors Influencing Quality of Experience2014
- Best Practices for QoE Crowdtesting: QoE Assessment With CrowdsourcingIEEE Transactions on Multimedia, 2013
- Video Stream Quality Impacts Viewer Behavior: Inferring Causality Using Quasi-Experimental DesignsIEEE/ACM Transactions on Networking, 2013
- Inferring the QoE of HTTP video streaming from user-viewing activities2011
- Assessing Quality of Experience of IPTV and Video on Demand Services in Real-Life EnvironmentsIEEE Transactions on Broadcasting, 2010
- 2010 Amendments to the 2002 "Ethical principles of psychologists and code of conduct".American Psychologist, 2010
- 5 Demand Characteristics and the Concept of Quasi-Controls2009
- Artifacts in Behavioral Research2009