You Won’t Believe How Many Options You Should Have!

The most common way of testing knowledge, or learning, in an elearning course is the multiple choice question (MCQ). There is debate on how effective this is and the relationship it has to on-the-job behaviour. For now I want to look at how many options a MCQ should have. Typically a MCQ follows the form of a stem and four, sometimes five, options, one of which is right and the others are incorrect and called distractors. Something like this:

What is the most common number of options in a multiple choice question?

  1. 2
  2. 3
  3. 4
  4. 5

Popularity doesn’t mean that this is the correct way to do something (see http://rationalwiki.org/wiki/Argumentum_ad_populum). The paper, “Three Options Are Optimal for Multiple-Choice Items: A Meta-Analysis of 80 years of Research” by Michael C. Rodriguez that  there are good reasons to have three options in MCQs.

The arguments boil down to: 

  • That less time is needed to create questions if less distractors are needed as one of the most difficult parts of creating a question is writing plausible distractors. It easy to write distractors that are not plausible. (Poor quality distractors are chosen less than 5% of the time.)
  • It takes less time to read and answer three-option MCQs as compared to four or five-option MCQs. This means that the learners can answer more questions in a shorter time.
  • By increasing the number of high quality test question that can be completed in a period of time test validity and reliability is increased.

One concern expressed that reducing the number of options will increases the number of correct answers by random guessing. This is really only a concern for the least able students who are stressed for time and just guess for the last questions. The reality is that because of the difficulty in creating high quality distractors it usually easy to quickly eliminate one, or two, distractors and bring the number of possible options down to three. So little difference is seen when the least probable distractors are eliminated from five-option MCQs to four or three-option MCQs.

Be brave, the next time you are writing test questions use only three options. You will save time, you will save your learner’s time, and you will still have a valid test.

 Rodriguez, M. C. 2005. Three options are optimal for multiple-choice items: A meta-analysis of 80 years of research. Educational Measurement: Issues and Practice, 24 (2), pp. 3–13

Advertisements

I am a instructional designer and developer specializing in Health and Safety Training, and Technical Training on a variety of subjects.

Tagged with: , ,
Posted in Uncategorized
One comment on “You Won’t Believe How Many Options You Should Have!
  1. Justin says:

    Very interesting post. I have used three choice questions before, but only in specific circumstances (behavior related answers when there are really only three possible answers). I will consider using them in more traditional tests in the future. I’ll also likely add more questions too.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: