Designing effective open book exams

What skills and knowledge are you assessing? How will students provide evidence of achievement? Which skills will they need to make use of their knowledge?

Intended learning outcome verbs should provide direction towards the choice of question type. Verbs such as identify, select, arrange, distinguish, match indicate that students need to determine the correct response. Verbs such as analyse, interpret, compare, summarise, indicate that students should construct a response.

Online open book exams require questions which encourage greater engagement with critical and analytical thinking, requiring students to apply knowledge, rather than simply recalling it.

What examples can you find? What does this data show? Is anything missing? How could you provide this? 

Multi-step questions, where one question builds on the previous one, can work well. 

Think about your choice of verb – if a student could simply look up the answer, you may want to find a different approach to the task that would make this more challenging. A range of example questions built from different verbs are available at the bottom of this page.

Case-based or problem-based exam questions require students to apply critical reasoning skills in response to a given scenario. This type of scenario-based question requires a more multi-faceted, longer answer response and can help students put themselves into the role of a professional in their field. Longer questions will also test a student’s ability to reason, create, analyse, synthesise, and evaluate and demonstrate higher level skills and knowledge.

Using specific case studies will reduce the chances of an essay mill being able to provide a stock answer. 

Locating data or resources should be followed-up with the need for students to probe and apply their findings, rather than just re-write information. Students can submit workings, calculations, proofs, or justifications for their answers.

Present relevant qualitative or quantitative data and then ask interpretative and application questions: 

What does the data show?

What relevance does this data or does the scenario have in terms of…? 

What other factors could potentially affect this data?

How would you test for these?  

Give clear instructions to help students answer the question e.g. 'Provide a hand-drawn graph...'.

Asking questions which are linked to a specific element of a course requires students to apply their knowledge, demonstrate their critical thinking, and synthesise new ideas e.g. ‘Using the materials and your notes from weeks 6-9, evaluate the importance of...'.

Recommendations for encouraging good academic practice by your students include:

• Setting questions that require students to interpret or apply subject material rather than simply locating, defining or rewriting information. 
• Requiring students to submit workings, calculations, proofs or justifications for their answers. 
• Requiring student responses to be contextualised to their own experience. 
• Providing guidance on ethical scholarship, such as reminding students to work through the Academic Integrity training on Referencing and Avoiding Plagiarism.

For some exam types, the ability to randomise questions by drawing them from a larger base, or shuffling the sequence, can be beneficial and help reduce opportunity for academic misconduct (Aparna Chirumamilla et al. 2020).

Where students are required to spend time searching for and interpreting information or data in an open book exam, it is important to make allowances for the length of expected answers as most will not be able to write as much per hour as in a traditional closed book exam. 

Add a suggested word length for essay-style answers, both to guide students and to manage marking. Where relevant, you can provide an explicit limit on what is needed, such as 'Provide three bullet points to justify your answer' or 'Provide one short sentence for each issue you feel relevant'. Students should spend time searching for and interpreting information/data in an open book exam, so may not be able to write as much per hour than a closed book exam. 

Allow students to practice and familiarise themselves with open book exams and different styles of questions before any summative open book assessment. This could be in your unit, or in other units in the year.

Ask students to time themselves when doing a set task with particular resources on their own, then discuss their results in a class session where they can mark their own or each other’s work.  

Formative assessment should include any tools that you are expecting students to use in the exam (e.g. Matlab).

Students can reflect on their approach to an open book exam: How did they revise or prepare notes? How effective was their approach?

If enabling students to access online library resources during an open book exam, please liaise with your Subject Librarian beforehand. This is to address concerns flagged by Library and Academic Registry around possible technical issues with access to library resources during an exam.