Photo by Nathan Dumlao on Unsplash

Designing Multiple Choice Questions

Students of the world can blame Benjamin Wood for the Multiple choice test. You don’t have to search hard on the Twitterverse to see how impactful question design can be:

[twitter_collection id=”992643547122745345″ height=”400″ dnt=”TRUE” chrome=”nofooter, noborders”]

A well written multiple question will test knowledge and understanding. The question is – what is good design practice for creating a multiple choice question?

The Basic Form

Stem, distractors and key

The classical form of a multiple choice is (1) question stem and (2) the options. The options are divided into distractors (incorrect answers) and key (the correct answer). The structure applies to various forms of multiple choice:

Traditional

Alternate Choice

True/False

Complex Multiple Choice

(Not recommended as good practice)

What is Good Question Design Practice?

So what does the research (Brame, C., 2013, Costigan et al, 2004 and Haladyna et al, 2002) say about good practice for writing a question?

General principles:

  • For low stakes, humour is acceptable especially if it is consistent with teaching style.  However, for high stakes assessment, it is best avoided.
  • Avoid trick questions e.g. trivial content, too fine answer discrimination, ‘best-of’ multiple correct answers, content presented opposite from instruction, and high ambiguity.

Writing the Question Stem:

  • Relevant to the intended learning outcomes and topic.
  • Deal with only one subject or problem, containing the central idea.
  • Use a complete question: the options should not complete the stem.
  • Ensure it is unambiguous, grammatically correct and appropriate language for the students.
  • Avoid irrelevant details.  Otherwise it may test comprehension rather than understanding or knowledge.
  • Avoid negatively formulated questions (e.g ‘Which is not a…’ ) – especially for higher cognitive order questions.
  • Don’t make it easy or obvious
  • Avoid repetition of words in the key or distractors.
  • All questions should be independent of other questions (do not build on previous questions).

Writing the Options:

All options (key and distractors) should:

  • Be clear, unambiguous and concise;
  • Have consistent grammar.
  • Avoid repetition of an important concepts.
  • Avoid “all/none of the above” answers ~ unless the question would be otherwise ‘too easy’. This approach tests ability to rule out wrong answers rather than knowing the correct one.
  • Have numerical options in either ascending or descending order.
  • Contain at least a total of three options for effective assessment (except for True/False questions).

Specifically, distractors should:

  • Be plausible alternatives.
  • Be based on common misconceptions or student errors (a very effective strategy).
  • Not be suggestive of the answer.
  • Not be close to the correct answer – distractors should substantially differ from distractors. This is perceived as trying to trick students.

Instant feedback

It may seem obvious – but consider learning possibilities through feedback in your question design. Most systems allow for both general question feedback and specific distractor/key feedback.

For example, consider a multiple choice question about converting medication dosage units. If a student gets it wrong, they will want to know why and how not to get it wrong next time. Remember – it could be you being cared for by that student!

References

Bush, M. (2001). A Multiple Choice Test that Rewards Partial Knowledge. Journal of Further and Higher Education, 25(2), 157-163.

Bush, M. (2014). Reducing the need for guesswork in multiple-choice tests. Assessment & Evaluation in Higher Education, 1-14.

Brame, C., (2013) Writing good multiple choice test questions. Retrieved 3/6/2018 from https://cft.vanderbilt.edu/guides-sub-pages/writing-good-multiple-choice-test-questions/

Brown, G., Bull, J., & Pendlebury, M. (1997) Assessing Student Learning in Higher Education, London. Routledge.

Costigan, F., Khaled, O., & Wodger, S. (2004) Good Practice Guide in Question and Test Design, Pass-IT. Retrieved 30/4/2019 from http://wayback.archive-it.org/1961/20100806070228/http://www.pass-it.org.uk/resources/031112-goodpracticeguide-hw.pdf

Haladyna, T., Downing, S., & Rodriguez, M. (2002). A Review of Multiple-Choice Item-Writing Guidelines for Classroom Assessment. Applied Measurement in Education, 15(3), 309-333.

Leave a reply

Your email adress will not be published. Required fields are marked*