There’s been an interesting discussion over the last couple of days on the Computer Assisted Assessment JISCMail list around delivery of multiple choice questions. 

The question of how long should be allowed for multiple choice questions produced a consensus of around ‘a minute per question plus a wee bit’ for a ‘typical’ MCQ, but that difficulty level or the use of negative marking or more sophisticated questions would impact on this.  Sandra Gibson cited research by Case and Swanson which suggests that

good students know the answers and … select the right one in very little time (seconds), poor students try and reason out the answers which takes longer. It depends how long you want to give the poorer students to try to work it out, which then impacts on the validity, reliability and differentiation of your assessment.

Discussion broadened to cover the issue of sequential delivery, i.e. when a candidate is unable to return to questions and revise their response once they have moved on to the next question in the test.  There were some compelling educational arguments in favour of this, for example, a series of questions building on or even containing the answers to previous questions; and less satisfactory justifications such as technical limitations on delivery software.    Fascinatingly, a number of posters reported the same (sadly anecdotal) finding that where students revise their response, the likelihood is that they’ve changed a correct answer to a wrong one.   It was also noted that tests which do not permit candidates to revise their responses required less maximum time than those that do.

It’s a good discussion that’s still going on, so well worth following or contributing to!