Mathias Broth

Playing with a robot's face detection capabilities constitutes a praxeological approach to learning about new technology. We explore how family members discover suitable formations for a Cozmo robot to scan a single face, drawing on video recordings from Swedish homes. Taking an ethnomethodology and conversation‐analysis perspective, we demonstrate how family members jointly establish a face‐to‐robot‐face formation, moving themselves and the robot camera to frame a relevant face . Other family members engage in backgrounding their own faces and foregrounding the face to be scanned. We discuss how participants' emergent troubleshooting practices reveal differences between robot perception and human situated seeing.

Methods for understanding the felt and situated experience of controlling autonomous systems are crucial for designing systems that are adjusted to the complexity of human interaction. Prior work has tended to overlook the bodily and experiential dimensions of monitoring systems at a distance, particularly in moments of losing control. We combined two approaches, ethnomethodology and conversation analysis, and soma design, to explore a case where a semi-autonomous system crashed when controlled by an inexperienced operator, captured in video ethnographic fieldwork. We report on our methodological approach, combining sequential video analysis of the unfolding sequence and interviews inspired by microphenomenology to unpack the operator’s experience of losing control. We contribute methodological considerations for interaction designers seeking to explore the felt experience of having and losing control of autonomous systems and discuss how insights gained through this combination of methods, rooted in phenomenology, support a designerly appreciation of safety operators’ work.

To this day, turn-taking models determining voice agents’ conduct have been examined primarily from a technical point of view, while the ways in which they emerge as interactional constraints or resources for human conversationalists in situ remain underexplored. Drawing on a detailed analysis of corpora of naturalistic data, we document how humans’ conduct was produced in reference to the ever-present risk that, each time they spoke, their talk might trigger a new uncalled-for contribution from the artificial agent. We examine this phenomenon in interactions involving rule-based robots from a ‘pre-LLM era’ as well as the most recent voice agents. This ‘omnirelevance of human speech’ (i.e., the possibility that a conversational agent may erroneously respond to any speech it detects) emerged as a constitutive feature of these human-agent encounters. We describe some of the practices through which humans managed these artificial agents’ turn-taking conduct. Given recent improvements in voice capture technology, we ask whether this ‘omnirelevance of human speech’ weighs even more heavily on human practices today than in the past.

We compare two types of situations involving monitoring of car driving: a driving instructor overseeing a trainee driver and a safety driver overseeing the performance of autonomous vehicle (AV) software. Our focus is on instances of monitoring that precede (1) interventions that are aborted before impacting the driving and (2) actual interventions on the driving in these two different settings. We find that in both cases the monitoring party “waits and sees” whether each type of “driving agent” will notice the impending trouble and adjust the driving appropriately. However, there are also marked differences: In the case of a driving instructor, the monitoring and intervening is a socially accountable event that may be responded to by the trainee driver, while in the case of an autonomous vehicle there are no such considerations, resulting in a difference in how the monitoring activity is embodied.

This paper analyzes honks directed at autonomous vehicles (AVs) by other drivers. As honks often mark problems, this focus allows us to better understand the challenges that AVs face in real traffic. Performing a sequential video analysis of 63 honk incidents uploaded by Tesla beta testers on YouTube, we identify how problematic situations emerge as honkable Traffic Gestalts. We identify four types of situated problems with AV driving performance marked by other drivers’ honks: they may wait too long, steer inconsistently, stop instead of going, and go too fast. We further show how a honk may be understandable as a warning, a nudge or a reprimand. Our work suggests designing honks for AVs to focus on relevant contexts, supported by developing bidirectional interfaces and audio analysis methods that consider the interplay of auditory and visual information in traffic.

Using publicly uploaded videos of the Waymo and Tesla FSD self-driving cars, this paper documents how self-driving vehicles still struggle with some basics of road interaction. To drive safely self-driving cars need to interact in traffic with other road users. Yet traffic is a complex, long established social domain. We focus on one core element of road interaction: when road users yield for each other. Yielding – such as by slowing down for others in traffic – involves communication between different road users to decide who will ‘go’ and who will ‘yield’. Videos of the Waymo and Tesla FSD self-driving cars show how these systems fail to both yield for others, as well as failing to go when yielded to. In discussion, we explore how these ‘problems’ illustrate both the complexity of designing for road interaction, but also how the space of physical machine/human social interactions more broadly can be designed for.

This study demonstrates how interpreters in a Swedish video relay service (VRS) between deaf and hearing users can simultaneously accomplish two different actions, each directed to a particular user of the service. The study takes a multimodal, ethnomethodological conversation analysis (EMCA) perspective and is empirically based on a corpus of 25 recordings from authentic video calls. Our analysis shows how interpreters, through what we call dual action design, are able to: (1) offer the floor to one party while informing the other party, (2) refer to one of the participants using different forms of deictic reference for the two users of the service, and (3) request confirmation of a source statement from one party while rendering a statement to benefit the other party. The study contributes to current discussions relating to sequentiality, simultaneity, and positioning in interpreting studies and multimodal interaction research.

Conceptualizing agency is a long-standing theoretical concern. Taking an ethnomethodological and conversation analytic perspective, we explore agency as the oriented to capacity to produce situationally and sequentially relevant action. Drawing on video recordings of families interacting with the Cozmo toy robot, we present a multimodal analysis of a single episode featuring a variety of rapidly interchanging forms of robotic (non-)agency. We demonstrate how agency is ongoingly constituted in situated interaction between humans and a robot. Describing different ways in which the robot’s statuses as either an agent or an object are interactionally embodied into being, we distinguish “autonomous” agency, hybrid agency, ascribed agency, potential agency and non-agency.