Multiple-Choice Questions and Written Questions matched according to levels of cognitive ability in an applied course: Evidence and practical implications

Whether or not the scores for multiple-choice questions (MCQs) and written questions (WQs) in formative assessments are the same, has been a subject of intense scrutiny. However, the evidence for their similarity at different levels of cognitive ability in applied courses has not been sufficiently documented. This study analysed the comparability of scores for equivalent MCQs and WQs at each level of cognitive ability, namely, 'application', 'analysis', 'synthesis' and 'evaluation', in an applied course. It was found that MCQ scores were higher than WQ scores at the levels of 'application' and 'analysis', while they were the same as WQ scores at the levels of 'synthesis' and 'evaluation'. Furthermore, MCQ ranking of students' scores at the level of 'evaluation' was inconsistent with its ranking at lower levels of cognitive ability. Thus, it is recommended that MCQs be pitched for sufficiently high levels of cognitive ability, albeit not the highest, to achieve similar scores to WQs.