Summative Assessment of Cognitive Skills: an Experience Involving Good Practices for Writing Multiple Choice Tests and Exam Composition

ABSTRACT Objective items or multiple choice questions with just one correct answer are among the most widely used methods for cognitive skills assessment, especially in exams designed for summative purposes. Assessments related to the cognitive domain using multiple choice questions are mostly used in high-stake exams, i.e. where the risks of failing are associated with serious consequences for the candidates. The widespread use of objective items for assessing learning in the cognitive domain may be explained by the fact that this exam modality fulfills both validity and reliability requirements, with the additional advantage that they are practical for use in exams with large numbers of candidates. Nevertheless, the validity and reliability requirements, in particular, will only be properly fulfilled when the process of writing multiple choice questions follows the rules of good practices for constructing exams and writing tests. This manuscript describes some of the rules for developing high quality multiple choice tests, based on both national and international published sources, as well as on the author’s experience. These rules relate to the content, language and presentation of the questions. This paper also addresses the importance of following appropriate rules for blueprinting construction, in order to show the alignment between assessment and curriculum and thereby contribute to meeting the validity requirements. It also briefly describes and discusses a successful experience of team work for constructing items and organizing exams. This experience exemplifies the combination of an organized process for constructing high quality questions for a well-balanced examination with an institutional strategy for faculty development in the field of learning assessment.