Multiple-choice question (MCQ) examinations are increasingly used in higher education to reduce marking workloads, provide rapid feedback, and support growing student cohorts. However, many academics remain uncertain about how to evaluate the quality of MCQ questions, judge appropriate levels of difficulty, or ensure that MCQ assessments measure student learning as effectively as traditional written examinations.

This talk from Dr Peter Sloan (Department of Physics) presents outcomes from a Teaching Development Fund project, undertaken with Dr Paul Snow, that aims to improve MCQ exam analysis, feedback, and question development. Drawing on assessment data from over a decade of undergraduate physics examinations, we have developed a framework for analysing exam questions in terms of difficulty and discrimination, allowing direct comparison between written and MCQ assessment formats.

Our analysis reveals important differences in how students experience MCQ and written examinations and highlights limitations in our ability to judge the quality of individual short-answer or MCQ-style questions during exam design. By treating written examinations as a reference standard, we show how post-exam item analysis can be used to identify effective and ineffective questions, build evidence-based question banks, and construct more balanced assessments that better reflect student ability.

The project is also developing tools to automate aspects of exam construction and feedback generation through the creation of a machine-readable database of assessment items. Combining educational data analysis with practical workflow improvements offers the potential to reduce administrative burden for staff while supporting more reliable, consistent, and transparent assessment practices.

The session will provide an overview of the methodology, key findings to date, and lessons that may be transferable to colleagues using MCQs and other forms of large-scale assessment across the University.