A Bayesian Approach to Measuring Evidence in L2 Research: An Empirical Investigation

Reza Norouzian, Michael De Miranda, Luke Plonsky

Research output: Contribution to journalArticlepeer-review

19 Scopus citations

Abstract

Null hypothesis testing has long since been the ‘go-to analytic approach’ in quantitative second language (L2) research (Norris, 2015, p. 97). To many, however, years of reliance on this approach has resulted in a crisis of inference across the social and behavioral sciences (e.g., Rouder et al., 2016). As an alternative to the null hypothesis testing approach, many such experts recommend the Bayesian hypothesis testing approach. Adopting an open-science framework, the present study (a) re-evaluates the empirical findings of 418 t-tests from published L2 research using Bayesian hypothesis testing, and (b) compares the Bayesian results with their conventional, null hypothesis testing counterparts as observed in the original reports. The results show that the Bayesian and the null hypothesis testing approaches generally arrive at similar inferential conclusions. However, considerable differences arise in the rejections of the null hypothesis. Notably, in 64.06% of cases when p-values fell between.01 and.05 (i.e., evidence to reject the null), the Bayesian analysis found the evidence in the primary studies to be only at an ‘anecdotal’ level (i.e., insufficient evidence to reject the null). Practical implications, field-wide recommendations, and an introduction to free online software (https://rnorouzian.shinyapps.io/bayesian-t-tests) for Bayesian hypothesis testing are discussed.

Original languageEnglish (US)
Pages (from-to)248-261
Number of pages14
JournalModern Language Journal
Volume103
Issue number1
DOIs
StatePublished - Mar 1 2019

Keywords

  • Bayes factor
  • Bayesian hypothesis testing
  • null hypothesis testing
  • open science framework
  • p-value
  • quantitative L2 research

ASJC Scopus subject areas

  • Language and Linguistics
  • Linguistics and Language

Fingerprint

Dive into the research topics of 'A Bayesian Approach to Measuring Evidence in L2 Research: An Empirical Investigation'. Together they form a unique fingerprint.

Cite this