TY - JOUR
T1 - Quantitative research methods and study quality in learner corpus research
AU - Paquot, Magali
AU - Plonsky, Luke
N1 - Funding Information:
We thank our MA students, Geoffrey Bodart, Dan Brown, Deirdre Derrick, and Sophie Longueville for their invaluable help with the coding of the LCR references. The first author gratefully acknowledges the financial support of the Belgian Fund for Research (FNRS) and the Language & Communication Institute (UC Louvain). Our gratitude also goes to the reviewers who helped shape this manuscript into its present form. All remaining errors remain ours.
Publisher Copyright:
© John Benjamins Publishing Company.
PY - 2017
Y1 - 2017
N2 - This study aims to provide the first empirical assessment of quantitative research methods and study quality in learner corpus research. We systematically review quantitative primary studies referenced in the Learner Corpus Bibliography (LCB), a representative bibliography of learner corpus research maintained by the Learner Corpus Association which contained 1,276 references when the current study began. Each primary study in the LCB was coded for over fifty features representing six dimensions: (a) publication type (i.e. conference paper, book chapter, journal article), (b) research focus (e.g. lexis, grammar), (c) methodological features (e.g. keyword analysis, error analysis, use of reference corpus), (d) statistical analyses (e.g. X2 t-test, regression analysis), and (e) reporting practices (e.g. reliability coefficients, means). Results point to several systematic strengths as well as many flaws, such as the absence of research questions, incomplete and inconsistent reporting practices (e.g. means without standard deviations), and lack of statistical literacy (i.e. LCR studies generally overrely on tests of statistical significance, do not report effect sizes, rarely check or report whether statistical assumptions have been met, and rarely use multivariate analyses). Improvements over time, however, are clearly noted and there are signs that, like other related disciplines, learner corpus research is slowly undergoing methodological reform.
AB - This study aims to provide the first empirical assessment of quantitative research methods and study quality in learner corpus research. We systematically review quantitative primary studies referenced in the Learner Corpus Bibliography (LCB), a representative bibliography of learner corpus research maintained by the Learner Corpus Association which contained 1,276 references when the current study began. Each primary study in the LCB was coded for over fifty features representing six dimensions: (a) publication type (i.e. conference paper, book chapter, journal article), (b) research focus (e.g. lexis, grammar), (c) methodological features (e.g. keyword analysis, error analysis, use of reference corpus), (d) statistical analyses (e.g. X2 t-test, regression analysis), and (e) reporting practices (e.g. reliability coefficients, means). Results point to several systematic strengths as well as many flaws, such as the absence of research questions, incomplete and inconsistent reporting practices (e.g. means without standard deviations), and lack of statistical literacy (i.e. LCR studies generally overrely on tests of statistical significance, do not report effect sizes, rarely check or report whether statistical assumptions have been met, and rarely use multivariate analyses). Improvements over time, however, are clearly noted and there are signs that, like other related disciplines, learner corpus research is slowly undergoing methodological reform.
KW - Quantitative research methods
KW - Reporting practices
KW - Research synthesis
KW - Study quality
UR - http://www.scopus.com/inward/record.url?scp=85048032092&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85048032092&partnerID=8YFLogxK
U2 - 10.1075/ijlcr.3.1.03paq
DO - 10.1075/ijlcr.3.1.03paq
M3 - Review article
AN - SCOPUS:85048032092
SN - 2215-1478
VL - 3
SP - 61
EP - 94
JO - International Journal of Learner Corpus Research
JF - International Journal of Learner Corpus Research
IS - 1
ER -