Reducing searchability
A range of cognitive levels can be assessed (recall, apply and evaluate etc), but is always important to link the questions to the desired learning outcomes. Note that factual recall questions without applied context are often easily searchable.
Some ways to reduce searchability can include:
- Use a novel context (hypothetical scenarios can work well)
- Use common mistakes & misconceptions as distractors
- Multiple select (1 or 2 correct)
- Shift the focus from recall questions to application-style questions
On the other hand embracing open book exams for what they are and having searchable questions can test a variety of other skills. Searching can also act as a hindrance and distraction.
Constructing effective question stems and distractors
Planning and designing the questions is one of the most important part of using MCQs. Both the question stem and distractors should be constructed effectively.
Tips for effective question stems
- Make the stem meaningful on its own – ask a question which directs the student to the learning outcome, where you need to have some knowledge to understand the question.
- Don’t include irrelevant material – make sure other text in the question is vital (e.g. adds context).
- Use negative phrasing carefully – double negatives and using negative phrasing can confuse students and distract from the learning of the MCQ.
Tips for writing effective distractors
- All of the distractors should be plausible – The distractors aren’t functional if they are not plausible, as students who don’t understand the learning outcomes can still be rewarded. Use common student errors – these make the best distractors.
- All of the distractors should be clear and concise – Students who know the correct answer should not have to worry about unclear and wordy answers and grasping what the answer is actually trying to say.
- Make sure they are mutually exclusive – A ‘trick’ question can sometimes lead to multiple answers being correct. Don’t have this ambiguity – make the answers mutually exclusive.
- They should have no clues to the correct answer – Savvy test takers can spot clues to correct answers such as differences in grammar, length, formatting, and language choice.
MCQ Question writing pitfalls
The following lists are a useful checklist of pitfalls to avoid and can help you identify areas for improvement in your question banks.
Cognitive load
- Are the question stems unfocused or not meaningful on their own?
- Is there unimportant content/minutia?
- Do the distractors contain superfluous information?
- Are there negative stem or answer options? (if necessary, be clear to emphasise the negative element “Which of this is NOT…”)
- Are the options confusingly ordered?
Wording clues
- Do you have absolute terms (easy to dismiss) or vague terms (open to interpretation)?
- Are there any grammatical or other clues to the correct answer?
Systematic
- Are some distractors very long/short (specific/vague)?
- Do you have “All/none of the above” options? (e.g. trivial if 2+ match)
- Are there logical clues? (avoid mutually exclusive pair options)
- Are there convergent options – correct answer has similarities with others?
Further information
- Faculty of Science: Writing Effective Multiple Choice Questions Workshop (Recording of presentation, Download slides)
- MCQs for self-regulated learning and feedback – D. J. Nicol & D. Macfarlane‐Dick (2006)
- Information on Bloom’s Taxonomy by Vanderbilt University
- Perspective article on applying Bloom’s Taxonomy to MCQs – Zaidi et al. (2018)
- Writing Good Multiple Choice Test Questions – Brame, C (2013)
- Pitfalls to avoid when writing MCQs – Webb et al. (2015)