Using Corpus-Based Register Analysis to Explore the Authenticity of High-Stakes Language Exams: A Register Comparison of TOEFL iBT and Disciplinary Writing Tasks

One of the central considerations in the validity argument for the TOEFL iBT is the relationship between the language on the exam and the language required for university courses. Corpus linguistics has recently been shown to be an effective way to explore this relationship, which can also be considered as an aspect of authenticity. Applying Multi-Dimensional Analysis, our study is the first to directly compare the lexico-grammatical characteristics of texts produced by second language writers on a high-stakes standardized exam (the TOEFL iBT) to the characteristics of texts produced by the same writers for disciplinary writing tasks required as part of their academic career (e.g., lab reports, argumentative essays). Our findings show that the language of TOEFL iBT tasks have patterns of both similarity and difference from disciplinary tasks with respect to four linguistic dimensions of language use: (a) Compressed Procedural Information versus Stance toward the Work of Others, (b) Personal Stance, (c) Possible versus Completed Events, and (d) Information Density. Integrated iBT tasks show similarities to certain kinds of disciplinary tasks, while independent iBT tasks differ linguistically from almost all disciplinary writing tasks. These general patterns hold regardless of the score levels of test takers. We discuss the implications of these findings for language teaching and assessment.