What do we mean?
  • Assessment for Learning (AfL): feedback-based process by which lecturers elicit evidence of students’ learning via informal evaluation strategies.
  • Formative Assessment: Low-stakes assessments to develop students knowledge and skills and provide opportunities for students to receive, reflect on, and implement tutor feedback.
  • Summative Assessment: High-stakes assessment designed to evaluate student learning.

Quick Guides - Writing Effective Multiple Choice Questions

Multiple Choice Questions (MCQs) can empower students as self-regulated learners, as long as time is taken into planning, writing and managing the MCQs. They are not only a tool for assessment, but for learning, revision and feedback too. Where time is spent creating effective questions and distractors, time is saved through automated marking and easy-to-access test analytics. However, MCQs are not always suitable and each situation needs to be assessed to work out if an MCQ would help the students reach the learning outcomes.

Reducing searchability

A range of cognitive levels can be tested (recall, apply and evaluate), but is always important to link the questions to the desired learning outcomes. However, fact recall questions can be easily searchable.

Some ways to reduce searchability can include:

  • Use a novel context (hypothetical scenarios can work well)
  • Use common mistakes & misconceptions as distractors
  • Multiple select (1 or 2 correct)
  • Shift the focus from recall questions to application-style questions

On the other hand embracing open book exams for what they are and having searchable questions can test a variety other skills of students. Searching can also act as a hindrance and distraction.

Constructing questions and distractors

Planning and designing the questions is one of the most important part of using MCQs. Both the questions and distractors should be constructed effectively.

Tips for constructing effective questions:

  • Make the stem meaningful on its own – ask a question which directs the student to the learning outcome, where you need to have some knowledge to understand the question
  • Don’t include irrelevant material – make sure other text in the question is vital (e.g. adds context)
  • Use negative phrasing carefully – double negatives and using negative phrasing can confuse students and distract from the learning of the MCQ

Tips for constructing effective distractors:

  • All of the distractors should be plausible - The distractors aren’t functional if they are not plausible, as students who don’t understand the learning outcomes can still be rewarded. Use common student errors – these make the best distractors.
  • All of the distractors should be clear and concise - Students who know the correct answer should not have to worry about unclear and wordy answers and grasping what the answer is actually trying to say
  • Make sure they are mutually exclusive - A ‘trick’ question can sometimes lead to multiple answers being correct. Don’t have this ambiguity – make the answers mutually exclusive
  • They should have no clues to the correct answer - Savvy test takers can spot clues to correct answers such as differences in grammar, length, formatting, and language choices

Pitfalls to Avoid

If you have a question bank which you often refer back to, try to manage this effectively. Use this checklist of pitfalls to avoid to identify any points of improvement:

Cognitive load:

  1. Does it contain superfluous information?
  2. Are the question stems unfocused or not meaningful on their own?
  3. Is there unimportant content/minutia?
  4. Are there negative stem or answer options? (if necessary, be clear to emphasise the negative element “Which of this is NOT…”)
  5. Are the options confusingly ordered?

Wording Clues

  1. Do you have absolute terms (easy to dismiss) or vague terms (open to interpretation)?
  2. Are there any grammatical or other clues to the correct answer?

Systematic

  1. Unequal option lengths?
  2. Do you have “All/none of the above” options? (e.g. trivial if 2+ match)
  3. Are there logical clues? (avoid mutually exclusive pair options)
  4. Are there convergent options – correct answer has similarities with others?

Further reading and useful links

 

tel@bath.ac.uk

Updated on: 24th November 2020