What do we mean?

An online open book exam allows students to access their notes and other resources whilst they complete the exam. Open book exams need to test students’ ability to apply knowledge and do things with the information, rather than their ability to simply locate information or state what has been learned on the course.  

Designing effective open book exam questions

When developing online open book exams, question design is key to set the appropriate level of challenge and promote academic integrity (e.g. limiting opportunities for students to cheat or collude). Questions should be clearly written and straightforward, so that students understand what they are being asked to do.

Explain to students how they should approach the online open book exam and which resources they are expected to use during the exam. This will help provide clarity and reduce exam-related anxiety for students.

Top tips for writing effective questions

1. Start with the intended learning outcomes

Design your questions with the intended learning outcomes in mind. What skills and knowledge are you assessing? How will students provide evidence of achievement if theses intended outcomes?  

2. Encourage greater engagement with critical and analytical thinking

Use questions which encourage greater engagement with critical and analytical thinking, requiring students to apply knowledge rather than simply recalling it. What examples can you find? What does this data show? Is anything missing? How could you provide this? Multi-step questions, where one question builds on the previous one, can work well. 

3. Think about your choice of verb

Think about your choice of verb – if a student could simply look up the answer, you may want to find a different approach to the task that would make this more challenging. Verbs such as analyse, interpret, compare, summarise, indicate that students should construct their own response.  

See our online exam page for a range of examples of questions built from different verbs.

4. Use case-based scenarios

Case-based or problem-based exam questions require students to apply critical reasoning skills in response to a given scenario. This type of scenario-based question requires a more multi-faceted, longer answer response and can help students put themselves into the role of a professional in their fieldUsing specific case studies will reduce the chances of an essay mill being able to provide a stock answer. 

5. Ask for application

Locating data or resources should be followed-up with the need for students to probe and apply their findings, rather than just re-write information. Students can submit workings, calculations, proofs, or justifications for their answers. 

6. Ask for interpretation

Present relevant qualitative or quantitative data and then ask interpretative and application questionsWhat does the data show? What relevance does this data or does the scenario have in terms of…? What other factors could potentially affect this data? How would you test for these? 

7. Be specific

Remind students not to over-answer or spend too long on a question. Give clear instructions to help students answer the question. E.g. 'Provide a hand-drawn graph...'. Ask questions linked to a specific element of a course. This requires students to apply their knowledge, demonstrate their critical thinking, and synthesise new ideas. E.g. ‘Using the materials and your notes from weeks 6-9, evaluate the importance of....’  

8. Vary the question order

For some exam types, the ability to randomise questions by drawing them from a larger base, or shuffling the sequence, can be beneficial and help reduce opportunity for academic misconduct (Aparna Chirumamilla et al. 2020).

For more information contact the TEL team.

9. Provide clear guidance

Add a suggested word length for essay-style answers, both to guide students and to manage marking. Where relevant, you can provide an explicit limit on what is needed, such as 'Provide three bullet points to justify your answer' or 'Provide one short sentence for each issue you feel relevant'. Students should spend time searching for and interpreting information/data in an open book exam, so may not be able to write as much per hour than a closed book exam. 

10. Make the most of formative assessment

Time management during open book exams is key. Allow students to practice and familiarise themselves with open book questions before any summative open book assessment. This could be in your unit, or in other units in the year. Formative assessment should include any tools that you are expecting students to use in the exam (e.g. Matlab).

Further information

For more on designing effective online assessment please see the links in the left-hand menu which cover a range of relevant areas. Please contact the Curriculum Development team if you would like to discuss anything further.

curriculumdev@bath.ac.uk

Updated on: 25 February 2021