THE INFLUENCE OF PRIOR EXPERIENCE ON THE CONSTRUCTION OF SCORING CRITERIA FOR ESL COMPOSITIONS: A CASE STUDY - Erdosy, M. Usman (International Journal of English Studies, 2001)
Before a principled explanation of variability in raters’ judgements of ESL compositions can be offered, the process of constructing scoring criteria and the manner in which prior experience enters this process must be analyzed. Therefore, utilizing protocol and intewiew data collected in the context of a comparative study, a case study will describe how one experienced rater dealt with the following operations while assessing a corpus of 60 TOEFL essays: establishing the purpose of assessment, developing a reading strategy to deal with a corpus of essays, and collecting context-specific information. Within each operation, the influence of background variables such as teaching and assessment experience will be examined, particularly on determining what type of information to collect, and on articulating expectations concerning test takers, test scores and the textual qualities of essays. The results of the study will be used to specific directions for future research into explaining inter-rater variability.