More on MCQs

There’s been an interesting discussion over the last couple of days on the Computer Assisted Assessment JISCMail list around delivery of multiple choice questions. 

The question of how long should be allowed for multiple choice questions produced a consensus of around ‘a minute per question plus a wee bit’ for a ‘typical’ MCQ, but that difficulty level or the use of negative marking or more sophisticated questions would impact on this.  Sandra Gibson cited research by Case and Swanson which suggests that

good students know the answers and … select the right one in very little time (seconds), poor students try and reason out the answers which takes longer. It depends how long you want to give the poorer students to try to work it out, which then impacts on the validity, reliability and differentiation of your assessment.

Discussion broadened to cover the issue of sequential delivery, i.e. when a candidate is unable to return to questions and revise their response once they have moved on to the next question in the test.  There were some compelling educational arguments in favour of this, for example, a series of questions building on or even containing the answers to previous questions; and less satisfactory justifications such as technical limitations on delivery software.    Fascinatingly, a number of posters reported the same (sadly anecdotal) finding that where students revise their response, the likelihood is that they’ve changed a correct answer to a wrong one.   It was also noted that tests which do not permit candidates to revise their responses required less maximum time than those that do.

It’s a good discussion that’s still going on, so well worth following or contributing to!

2 thoughts on “More on MCQs

  1. Hmm. Why not comply with psychometric standards? Rules-of-thumb are fine for quick costings but not really right for question-level timings. Try the AERA standards?

  2. This format is called computer adaptive format.Here returning to the previous question or the previous section is not allowed.At times this get much more difficult for the exam takers to handle because they can’t revise their answers.Personally I favor the non adaptive based computer exam where option of returning to the previous question or section is present.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>