Luke Plonsky

The relationship between computer-assisted language learning (CALL) and second language acquisition (SLA) has been studied both extensively, covering numerous subdomains, and intensively, resulting in hundreds of primary studies. It is therefore no surprise that CALL researchers, as in other areas of applied linguistics, have turned in recent years to meta-analysis as a means to synthesize quantitative results across studies. To date, nearly 30 CALL-SLA syntheses and meta-analyses have been conducted, covering topics from hypertext glosses (Yun, 2011) to synchronous computer-mediated communication (Ziegler, 2015) to game-based learning (Chiu, Kao, & Reynolds, 2012). Despite these individual contributions, the overall effects of CALL on SLA across domains have yet to be addressed. In this 'second-order' review, we provide a thorough account of substantive findings and a critical description and evaluation of methodological practices of CALL-SLA meta-analyses. We begin by describing the generally substantial effects of CALL on L2 learning along with an examination of different types of technology such as CALL glosses and computer-mediated communication (CMC). Results of the methodological review reveal wide variability overall and in several practices associated with rigor, transparency, and utility of meta-analytic reviews. At the close of our article, we provide empirically identified recommendations for future primary and meta-analytic research.

Ensuring internal validity in quantitative research requires, among other conditions, reliable instrumentation. Unfortunately, however, second language (L2) researchers often fail to report and even more often fail to interpret reliability estimates beyond generic benchmarks for acceptability. As a means to guide interpretations of such estimates, this article meta‐analyzes reliability coefficients (internal consistency, interrater, and intrarater) as reported in published L2 research. We recorded 2,244 reliability estimates in 537 individual articles along with study (e.g., sample size) and instrument features (e.g., item formats) proposed to influence reliability. We also coded for the indices employed (e.g., alpha, KR20). The coefficients were then aggregated (i.e., meta‐analyzed). The three types of reliability varied, with internal consistency as the lowest: median = .82. Interrater and intrarater estimates were substantially higher (.92 and .95, respectively). Overall estimates were also found to vary according to study and instrument features such as proficiency (low = .79, intermediate = .84, advanced = .89) and target skill (e.g., writing = .88 vs. listening = .77). We use our results to inform and encourage interpretations of reliability estimates relative to the larger field as well as to the substantive and methodological features particular to individual studies and subdomains.

Tasks are frequently used to elicit learner language in second language (L2) research. The purposes for doing so, however, vary widely, covering a range of theoretical models, designs, and analyses. For example, task-based researchers have examined a range of linguistic and interactional features (e.g., accuracy, language-related episodes) that are found in learner production and that vary as a function of task conditions (e.g., +/− complex), modes (oral, written, computer-mediated), and settings (second vs. foreign language). This article presents a synthesis of substantive interests and methodological practices in this area. We first collected a sample of 85 primary studies of task-based language production published from 2006 to 2015. Each study was then coded for the target features it analyzed as well as other contextual and demographic variables. We also coded for methodological features related to study designs, sampling, analyses, and reporting practices. The results indicate a strong preference toward analyses of grammar, vocabulary, accuracy, and different features of L2 interaction, and very little interest in task-induced pronunciation, pragmatics, and the quality of task performance. More fundamentally, this domain may be hindered by a lack of theoretical and operational consistency. The data also point to a number of concerns related to research and reporting practices (e.g., low statistical power; missing data). Based on our findings, we outline a number of pointed recommendations for future research in this domain.