Limited aspects of reality: Frames of reference in language assessment

  • Glenn Fulcher
  • Agneta Svalberg
Keywords: language testing, criterion-referencing, norm-referencing, domain description, specific purpose testing, scoring criteria, standard setting

Abstract

Language testers and educational measurement practitioners operate within two frames of reference: norm-referenced (NRT) and criterion-referenced testing (CRT). The former underpins the world of large-scale standardized testing that prioritizes variability and comparison. The latter supports substantive score meaning in formative and domain specific assessment. It has recently been claimed that the criterion-referenced enterprise is all but dead; its one legacy being the way in which test results are communicated (Davidson, 2012, p. 198). In this article, we argue that the announcement of the demise of CRT is premature. But we acknowledge that what for the most part passes as CRT is in fact not criterion-referenced, and is based upon a corruption of the original meaning of “criterion” as domain-specific performance. This distortion took place when NRT co-opted the term “standard” to serve as a rationale for the measurement enterprise of establishing cut-scores to retrofit NR tests with meaning derived from external scales. We argue that this is not CRT, and the true heirs of the CRT movement in applied linguistics are researchers who base test design in the careful analysis of construct and content in domain specific communication.

Author Biographies

Glenn Fulcher
Glenn Fulcher is Professor of education and language assessment and Head of the School of Education at the University of Leicester in the United Kingdom. His main research interests are language testing and the philosophy of educational assessment. He has extensive experience of working on test design (paper and computer based) and development projects and specific expertise in the design and development of rating instruments for performance assessments, particularly for L2 speaking and communication. He has worked for a number of major testing companies in a research or consultancy capacity on test development projects, alignment studies, and conducting item writer/rater training. He has published widely in the field and is co-editor of Language Testing (Sage).
Agneta Svalberg
Agneta Svalberg has lived in Greece and Mexico where she worked as a Foreign Language teacher of English. These experiences developed an interest in the workings of English grammar and this resulted in a PhD thesis on tense-modality-aspect in English and Modern Greek (University of Sydney, Australia). Following this, Agneta lectured in Applied Linguistics at Universiti Brunei Darussalam where she developed an interest in non-standard use of grammar, and grammar teaching pedagogy. She joined the School of Education in 2000 and she teaches on the MA TESOL and MA Applied Linguistics and TESOL programmes, both the campus-based and distance learning options. Agneta additionally tutors on the EdD programme and supervises EdD and PhD students.
How to Cite
Fulcher, G., & Svalberg, A. (1). Limited aspects of reality: Frames of reference in language assessment. International Journal of English Studies, 13(2), 1-19. https://doi.org/10.6018/ijes.13.2.184061