Developing a Rating Scale for Classroom Assessment of the Argumentative Writing of Chinese EFL College Students Majoring in English

The argumentative writing ability of Chinese EFL college students majoring in English is frequently assessed with both formative and summative assessments in the classroom. However, rating scal es used in these assessments fail to provide precise and clear scoring criteria and fail to adequately evaluate the quality of arguments/argumentation. By adopting a theoretically based data driven approach proposed by Knoch (2009) 2009), this study aims to de velop a rating system appropriate for classroom assessment of the argumentative writing ability of Chinese EFL college students majoring in English and toinvestigate its usability by ra ters Chinese EFL writing teachers This study was undertaken in two phases. In phase one, 258 writing scripts were selected from a corpus of argumentative writing of Chinese EFL college students built for thi s study, and were analyzed using discourse analytic measures . The results of the main study were used to create descriptors of the new rating scale. In phase two, 30 writing scripts, representing a wide range of writing proficiency levels, were rated by th ree writing teachers (or raters). Their ratings using the new rating scale were analyzed using inter rater reliability analysis, and follow up questionnaire feedback from the raters was analyzed using content analysis. The new rating scale was shown to be generally adequate to represent the construct of argumentative writing ability, useful for providing detailed feedback for the teaching and assessment of argumentative writing ability in the classroom, and reliable for three of five trait scales mechani cs, fluency and argumentation. The new rating scale was also shown to be not practical, not authentic as a whole and not reliable for the scales of accuracy and coherence, although no exact reason for the unreliability was found. Findings are discussed in terms of their implications for assessment of argumentation ability, rating scale development, and score reporting in classroom based assessment in the Chinese EFL context.