What do we mean?
An online open book exam allows students to access their notes and other resources whilst they complete the exam. Open book exams need to test students’ ability to apply knowledge and do things with the information, rather than their ability to simply locate information or state what has been learned on the course.
Designing effective open book exam questions
Online open book exams can be an effective test of skills and knowledge. When developing online open book exams, question design is key to set the appropriate level of challenge and promote academic integrity (e.g. limiting opportunities for students to cheat or collude). Questions should be clearly written and straightforward, so that students understand what they are being asked to do. Effective open book questions will be different from the type of questions used in closed book examinations which tend to focus on recall.
Explain to students how they should approach the online open book exam and which resources they are expected to use during the exam. This will help provide clarity and reduce exam-related anxiety for students.
Top tips for writing effective questions
Design your questions with the intended learning outcomes in mind. What skills and knowledge are you assessing? How will students provide evidence of achievement if theses intended outcomes?
Use questions which encourage greater engagement with critical and analytical thinking, requiring students to apply knowledge rather than simply recalling it. What examples can you find? What does this data show? Is anything missing? How could you provide this? Multi-step questions, where one question builds on the previous one, can work well.
Think about your choice of verb – if a student could simply look up the answer, you may want to find a different approach to the task that would make this more challenging. Verbs such as analyse, interpret, compare, summarise, indicate that students should construct their own response.
See our online exam page for a range of examples of questions built from different verbs.
Case-based or problem-based exam questions require students to apply critical reasoning skills in response to a given scenario. This type of scenario-based question requires a more multi-faceted, longer answer response and can help students put themselves into the role of a professional in their field. Using specific case studies will reduce the chances of an essay mill being able to provide a stock answer.
Locating data or resources should be followed-up with the need for students to probe and apply their findings, rather than just re-write information. Students can submit workings, calculations, proofs, or justifications for their answers.
Present relevant qualitative or quantitative data and then ask interpretative and application questions: What does the data show? What relevance does this data or does the scenario have in terms of…? What other factors could potentially affect this data? How would you test for these?
Remind students not to over-answer or spend too long on a question. Give clear instructions to help students answer the question. E.g. 'Provide a hand-drawn graph...'. Ask questions linked to a specific element of a course. This requires students to apply their knowledge, demonstrate their critical thinking, and synthesise new ideas. E.g. ‘Using the materials and your notes from weeks 6-9, evaluate the importance of....’
For some exam types, the ability to randomise questions by drawing them from a larger base, or shuffling the sequence, can be beneficial and help reduce opportunity for academic misconduct (Aparna Chirumamilla et al. 2020).
For more information contact the TEL team.
Add a suggested word length for essay-style answers, both to guide students and to manage marking. Where relevant, you can provide an explicit limit on what is needed, such as 'Provide three bullet points to justify your answer' or 'Provide one short sentence for each issue you feel relevant'. Students should spend time searching for and interpreting information/data in an open book exam, so may not be able to write as much per hour than a closed book exam.
Time management during open book exams is key. Allow students to practice and familiarise themselves with open book questions before any summative open book assessment. This could be in your unit, or in other units in the year. Formative assessment should include any tools that you are expecting students to use in the exam (e.g. Matlab).
For more on designing effective online assessment please see the links in the left-hand menu which cover a range of relevant areas. Please contact the Curriculum Development team if you would like to discuss anything further.