Vassilis-Javed Khan

AI systems are increasingly being positioned to assist people in decision-making. However, recent empirical studies show critical concerns that people over-rely on AI advice without analytically engaging with it. While HCI research explores how people rely on AI advice, we argue that it largely overlooks an important aspect: replicating realistic decision-making scenarios. Human-AI interaction factors influence people’s reliance on AI advice. To understand human-AI interaction factors and their interplay, we conducted an analytical review of recent studies in human-AI reliance literature. We analyzed the decision-making tasks in research and their validity in application-grounded contexts. Our findings show that user engagement is a precious commodity for relying on AI advice; however, it comes at a cost. We also discuss factors contributing to “appropriate reliance”, existing research gaps, and recommendations for intervention design for human-AI reliance. Our work contributes to the critical body of research on building appropriate reliance on AI advice.

Interactive Machine Learning (IML) apps permeate all aspects of businesses, including sales. Customer segmentation is integral for sales, to identify customer groups to serve them appropriately. However, the novelty of such apps in a sales context raises the question: what challenges designers of such apps will face, in a sales context? To explore this question, we report our reflections on an IML study for customer clustering. We used data from a global manufacturing company to cluster customers using the Recency, Frequency, and Monetary (RFM) method. We applied a machine learning clustering model (K-Means) and discussed with seven seasoned sales managers the interaction with clusters. We report four challenges and foresee that designers of such systems will face, in the context of sales operations.

Crowdsourcing in China is a thriving industry. Among its most interesting structures, we find crowdfarms, in which crowdworkers self-organize as small organizations to tackle macrotasks. Little, however, is known as to which practices these crowdfarms use to tackle the macrotasks, and this goes hand in hand with the current practice of the HCI research community to treat all forms of complex crowdsourcing work as practically the same. However, macrotasks differ substantially regarding structure and decomposability. Treating them under one umbrella term - macrotasking - can lead to an imprecise understanding of the workforce involved. We address this gap by examining the work practices of 31 Chinese crowdfarms on the four main macrotask types, namely: modular, interlaced, wicked, and container macrotasks. Our results confirm essential differences in how these nascent crowd organizations address different macrotasks and shed light on what platforms can do to improve the uptake of such work.

In our evaluation of the B2B e-commerce site of a global manufacturing company we conducted a user test with employees and customers. We found statistically significant differences in usability, user experience and NPS metrics between employees and customers, with the employees being more critical compared to the customers. We postulate and present some evidence that this difference is due to employees implicitly comparing B2B with B2C e-commerce sites and therefore expecting the experience of a B2C site for a B2B site. Such a comparison, fosters a bias, which has implications for businesses that host B2B e-commerce sites. We conclude by sketching recommendations for practitioners on how to go about such a bias.