Wing Yee Jenifer Ho

The widespread availability of generative artificial intelligence tools has had a dramatic effect on language education, transforming how we think about and go about writing. This includes ‘multimodal composing’, which combines moving image, soundtrack, speech, and on-screen writing in digital communication. In this article, we focus on the potentials and pitfalls of generative artificial intelligence in a pedagogy of digital multimodal composing. The article aims to explore how the affordances and constraints of generative artificial intelligence tools powerfully shape the multimodal composing process in a way that enables or constrains individual agency, creativity, and criticality; and conceptualize a model of best practice for the use of generative artificial intelligence in digital multimodal composing. As language and literacy educators, we report the findings of a practitioner inquiry study into our own pedagogically motivated digital multimodal composing practices, using generative artificial intelligence tools to collaboratively complete a digital multimodal composing project (a digital video scientific documentary) designed for second language learners of English for science. We recorded this process by keeping copies of input and output as well as maintaining regular research journals. The analysis identified five themes relevant to agency, creativity and criticality in digital multimodal composing, namely: (a) feelings; (b) relating to generative artificial intelligence: the iterative, collaborative loop; (c) ideological positioning; (d) modal interactions: process and product; and (e) voice and authenticity. We suggest an integrated critical digital literacies model to inform digital multimodal composing pedagogy that engages with generative artificial intelligence at three key levels, extending previous work by highlighting the role of affect: a macro level focusing on feelings, materialities, and ideology; a meso level focused on the design of a collaborative, iterative loop in generative artificial intelligence digital multimodal composing workflow; and a micro level focused on interaction with generative artificial intelligence.

Applied linguistics researchers are increasingly working with a wider range of data, including audio-recording data, and video-recording data collected in online and internet-based contexts. Transcription of such multimodal data with digital affordances is an important part of the research process. Moreover, the emergence of transcription tools that make use of generative artificial intelligence technologies has made available a variety of web-based transcription tools and apps to choose from, such as Otter.ai and Happy Scribe. This chapter will introduce readers to different types of transcription and transcription tools, specifically in the context of multimodal transcription. In addition, several example studies which adopt such tools will be described, along with ethical considerations related to the use of transcription tools. Finally, directions for future research will be discussed.

Many language teachers create short, engaging, and accessible English content on social media, accumulating large numbers of followers. Drawing on existing conceptualizations of authenticity and qualities of internet celebrities, this study presents a multimodal analysis of Instagram accounts of two microcelebrity British English language teachers, each with over a million followers. The findings reveal instances where teachers position themselves as the authority for linguistic or non-linguistic knowledge highlighting different forms of cultural capital including ordinary and mundane aspects of their everyday lives. The analysis also reveals exclusive representations of the West (specifically the UK) including vocabulary associated with British culture, idiomatic expressions, and British pronunciation of some words. The study provides novel and critical insights into the emerging forms of language teaching on social media platforms shaped by the neoliberal landscape of education with important implications for English language teaching professionals and teacher education programs.

Watching flight safety videos is a mandatory component of global air travel. While previous studies have primarily focused on using experimental techniques to assess passengers’ information recall and strategies to increase passengers’ level of attention, little has been done on the transmodal design aspects of these videos which can open up new spaces for branding. This paper first discusses how this genre has transitioned from solely delivering safety information, to increasingly being crafted as a space for airline and destination branding in a globalized context. Using the notion of assemblage and transmodalities , this study presents a comparative analysis of two airlines to investigate how they are similar or different in terms of how they communicate safety information and engage in airline and destination branding. This study aims to raise awareness of flight safety videos as not only an instructional genre but also a space for intercultural communication between airlines and passengers who may or may not have a shared understanding of cultural and safety knowledge. Taken together, the study contributes to a nuanced and evolving understanding of the flight safety video genre.

Digital multimodal composing (DMC) allows students to mobilize a wide range of multimodal resources to make meaning. While studies in DMC tended to focus on language-learning contexts, few of them examine its use in content-based courses whereby students are proficient L2 users expected to demonstrate understanding of abstract concepts using DMC. Drawing on semi-structured interviews with English-major students who attended a writing course in an English-medium instruction university in Hong Kong, the study investigated how the students’ view of DMC was transformed after exposure to DMC pedagogy. The study findings suggest that students developed an increased awareness of the affordances and constraints of (new media) writing, and they strategically bring in and bridge ‘in-class’ and ‘out-of-class’ digital practices. Students also reported on the opportunities and challenges of DMC pedagogy through reflecting on their DMC experiences. The study proposes suggestions for implementing DMC in diverse teaching and learning contexts.

While the effectiveness of facemasks against COVID-19 has now become largely uncontroversial, at the beginning of the global pandemic, wearers of facemasks were often the target of sometimes racially tinged attacks. Wearing facemasks (or not) became not just a question of science, but evolved into a more complex issue of social identity, morality and global citizenship embedded within the “tribal thinking” of mask-wearers and non-mask-wearers. This paper explores to what extent two bilingual YouTube influencers participated in either accentuating or softening of boundaries of the two “tribes” by embedding facemasks in their videos. Based on multimodal transcriptions of the two videos (Wang, Yilei, Dezheng Feng & Wing Y. J. Ho. 2021. Identity, lifestyle, and face-mask branding: A social semiotic multimodal discourse analysis. Multimodality & Society 1(2). 216–237), three moments were identified where facemasks were employed by the social actors to perform everyday activities, such as grocery shopping and family brunch. I then examine the interactional stances (Dubois, John W. 2007. The stance triangle. In Robert Englebretson (ed.), Stancetaking in discourse: Subjectivity, evaluation, interaction, 139–182. Amsterdam: John Benjamins) taken by the actors towards facemasks through language and other semiotic resources. By exploring their multimodal stance-taking, it is argued that the two YouTubers’ intercultural trajectories, their performances of authenticity, and their established influence on social media provided them unique means for participating in tribalizing discourses around facemasks by making perceived differences between different groups materials for cultural consumption. The paper concludes by discussing the opportunities and challenges of vernacular health communication through social media influencers.