Erica Sandlund

Research on what should count as evidence of foreign/second language interactional competence (L2 IC) has grown exponentially, particularly drawing on work in Conversation Analysis (CA) (Hall, Hellerman, and Pekarek Doehler 2011). Recurrent participation in L2 conversation is viewed as key to IC development, and learners gradually master an increasing range of L2-appropriate social actions (Salaberry and Kunitz 2019). We address L2 IC in two educational contexts—Norway and Flanders—where learners’ exposure to English is high but where the starting age of formal instruction differs. We focus on one facet of L2 IC: the interactional management of agreement sequences. Previous research has focused on speakers’ practices for doing disagreement (e.g., Čekaitė 2007), showing that learners deploy more advanced methods for disagreement as their proficiency and L2 participation increases. In this study, which is based on generated speaking test data using pre-set test topics that learners are to discuss, relatively few disagreements were found. However, for high-scoring learners, extended and collaborative agreement sequences revealed various practices for jointly agreeing on a stance on the pre-set topics. From a collection of 24 speaking tests collected in grade 10, a subsample of the four highest-scoring learners in each context (N = 8) and their co-participants was analyzed further with a CA approach. Findings show the learners’ awareness of a preference for agreeing with first assessments (cf. Pomerantz 1984; Sacks 1987), and of the conditional relevance of accounts following expressions of agreement with a test prompt. Participants also show affiliation with each other’s contributions, and for the most part align with prior action in their responses (cf. Steensig 2019). Participants also display competence in drawing on vocal, verbal, and embodied resources in displaying agreement and affiliation.

Swedish students’ oral proficiency in English as a foreign/second language (L2) is tested annually in a part of the English national test entitled Speaking (test construct: oral production and interaction), which students’ own teachers both administer and assess. Despite access to extensive assessment guidelines from the Swedish National Agency for Education, many teachers choose to construct their own scoring rubrics for assessment. In this study, we examine teachers’ professional knowledge about assessment of oral proficiency in L2 English, as it emerges from policy transformation in these teacher-generated rubrics. Data consists of 20 rubrics for assessment of Speaking in years 6 and 9. Rubrics were collected through professional networks and two English-teacher groups on Facebook. Qualitative content analysis was used to analyze teachers’ conceptualizations of the test construct, as well as similarities and differences between conceptualizations. Results showed consensus of what to assess, but not necessarily how, which was particularly salient for interaction. Analysis of policy transformation indicated that when teacher-generated scoring rubrics were used for assessment of Speaking, each criterion was considered separately, making them beneficial for formative feedback. The professional knowledge that emerged is therefore formed by teachers’ dual roles – as examiner and teacher – in the test situation.

Chapter 10 was previously published non-open access. It has now been changed to open access under a CC BY 4.0 license and the copyright holder updated to ‘The Author(s)’. The book has also been updated with this change.

English is taught in classrooms across the globe to learners of all ages, from very young learners in primary school to older learners who have reached retirement and occupy their time in the so-called third age by studying English

Vocabulary experts recommend first language (L1) translation equivalents for establishing form–meaning mappings for new second language (L2) words, especially for lower proficiency learners. Empirical evidence to date speaks in favor of L1 translation equivalents over L2 meaning definitions, but most studies have investigated bi‐ rather than multilingual learners. In our study, we investigated instructed English vocabulary learning through an intervention study in six language‐diverse secondary school English classrooms in Sweden (N = 74) involving three conditions for presentation of word meanings: (a) definitions in the L2 (English), (b) translation equivalents in the shared school and majority language (Swedish), and (c) translation equivalents in the shared school and majority language plus other prior languages among the learners (Swedish and other). Based on overall weighted mean effect sizes and mixed‐effects modeling, the results showed that conditions that involved L1 translation equivalents yielded higher scores than did target language definitions in immediate posttests with a small effect size but no differences in delayed posttests.

Many tests of second language (L2) oral proficiency (OP) include speaking tasks designed to generate narrative talk. From an assessment perspective, frequent turn shifts and a displayed ability to understand and build upon prior talk are generally favored. As storytelling operates through a temporary suspension of ordinary mechanisms for turn-taking, tellings in tests may present challenges for test-takers as well as examiners. This study draws on a corpus of 71 recorded high-stakes tests of oral proficiency and interaction in English in Swedish compulsory school. Test-takers are Swedish 9th graders participating in the compulsory National Test of English, a paired or small group test using topic cards to prompt peer interaction. Drawing on a conversation analytic approach to test interaction and interactional competence, (Young and He, in Talking and testing: Discourse approaches to the assessment of oral proficiency, John Benjamins, (1998); Salaberry and Kunitz, in Teaching and testing L2 interactional competence: Bridging theory and practice, Routledge, (2019)), the study centres on when and how participants recruit small stories for task accomplishment by inviting, resisting, or volunteering tellings. The analysis identifieswhentellings are made relevant across task types, andhowthese local occasionings are oriented to by test-takers. Findings point to the complexity of storytellingsin test contexts, as test-takers often do not treat narratives as relevant or appropriate contributions in the institutional frame of testing.

Assessment of foreign/second language (L2) oral proficiency is known to be complex and influenced by the local context. In Sweden, extensive assessment guidelines for the National English Speaking Test (NEST) are offered to teachers, who act as raters of their own students’ performances on this high-stakes L2 English oral proficiency (OP) test. Despite guidelines, teachers commonly construct their own NEST scoring rubric. The present study aims to unveil teachers-as-raters’ conceptualizations, as these emerge from the self-made scoring rubrics, and possible transformations of policy. Data consist of 20 teacher-generated scoring rubrics used for assessing NEST (years 6 and 9). Rubrics were collected via personal networks and online teacher membership groups. Employing content analysis, data were analysed qualitatively to examine (i) what OP sub-skills were in focus for assessment, (ii) how sub-skills were conceptualized, and (iii) scoring rubric design. Results showed that the content and design of rubrics were heavily influenced by the official assessment guidelines, which led to broad consensus about what to assess—but not about how to assess. Lack of consensus was particularly salient for interactive skills. Analysis of policy transformations revealed that teachers’ self-made templates, in fact, lead to an analytic rather than a holistic assessment practice.

The main objective of this study was to examine whether a Rater Identity Development (RID) program would increase interrater reliability and improve calibration of scores against benchmarks in the assessment of second/foreign language English oral proficiency. Eleven primary school teachers-as-raters participated. A pretest–intervention/RID–posttest design was employed and data included 220 assessments of student performances. Two types of rater-reliability analyses were conducted: first, estimates of the intraclass correlation coefficient two-way random effects model, in order to indicate the extent to which raters were consistent in their rankings, and second, a many-facet Rasch measurement analysis, extended through FACETS®, to explore variation regarding systematic differences of rater severity/leniency. Results showed improvement in terms of consistency, presumably as a result of training; simultaneously, the differences in severity became greater. Results suggest that future rater training may draw on central components of RID, such as core concepts in language assessment, individual feedback, and social moderation work.

While paired student discussion tests in EFL contexts are often graded using rubrics with broad descriptors, an alternative approach constructs the rubric via extensive written descriptions of video-recorded exemplary cases at each performance level. With its long history of deeply descriptive observation of interaction, Conversation Analysis (CA) is one apt tool for constructing such exemplar-based rubrics; but to what extent are non-CA specialist teacher-raters able to interpret a CA analysis in order to assess the test? This study explores this issue by comparing two paired EFL discussion tests that use exemplar-based rubrics, one written by a CA specialist and the other by EFL test constructors not specialized in CA. The complete dataset consists of test recordings (university-level Japanese learners of English, and secondary-level Swedish learners of English) and recordings of teacher-raters’ interaction. Our analysis focuses on ways experienced language educators perceive engagement while discussing their ratings of the video-recorded test talk in relation to the exemplars and descriptive rubrics. The study highlights differences in the way teacher-raters display their understanding of the notion of engagement within the tests, and demonstrates how CA rubrics can facilitate a more emically grounded assessment.