The most common way of testing knowledge, or learning, in an elearning course is the multiple choice question (MCQ). There is debate on how effective this is and the relationship it has to on-the-job behaviour. For now I want to look at how many options a MCQ should have. Typically a MCQ follows the form of a stem and four, sometimes five, options, one of which is right and the others are incorrect and called distractors. Something like this:
What is the most common number of options in a multiple choice question?
Popularity doesn’t mean that this is the correct way to do something (see http://rationalwiki.org/wiki/Argumentum_ad_populum). The paper, “Three Options Are Optimal for Multiple-Choice Items: A Meta-Analysis of 80 years of Research” by Michael C. Rodriguez that there are good reasons to have three options in MCQs.
The arguments boil down to:
- That less time is needed to create questions if less distractors are needed as one of the most difficult parts of creating a question is writing plausible distractors. It easy to write distractors that are not plausible. (Poor quality distractors are chosen less than 5% of the time.)
- It takes less time to read and answer three-option MCQs as compared to four or five-option MCQs. This means that the learners can answer more questions in a shorter time.
- By increasing the number of high quality test question that can be completed in a period of time test validity and reliability is increased.
One concern expressed that reducing the number of options will increases the number of correct answers by random guessing. This is really only a concern for the least able students who are stressed for time and just guess for the last questions. The reality is that because of the difficulty in creating high quality distractors it usually easy to quickly eliminate one, or two, distractors and bring the number of possible options down to three. So little difference is seen when the least probable distractors are eliminated from five-option MCQs to four or three-option MCQs.
Be brave, the next time you are writing test questions use only three options. You will save time, you will save your learner’s time, and you will still have a valid test.
Rodriguez, M. C. 2005. Three options are optimal for multiple-choice items: A meta-analysis of 80 years of research. Educational Measurement: Issues and Practice, 24 (2), pp. 3–13