What do we mean?

An online open book exam allows students to access their notes and other resources whilst they complete the exam. Open book exams need to test students’ ability to apply knowledge and do things with the information, rather than their ability to simply locate information or state what has been learned on the course.

Designing effective online open book exams

Online open book exams can take a variety of formats, for example, timed conditions with access to notes, with a summary of  notes, with any resources, or pre-seen questions. This format of exam can also simulate conditions that students will face in their future professional lives, with the need to use selected resources and apply knowledge to solve a problem, discuss a topic, or test a hypothesis.

Top tips for writing effective questions

The formation of questions for online open book exams is crucial in setting the appropriate level of challenge and in promoting academic integrity (e.g. limiting opportunities for students to cheat or collude). Use our top tips for writing effective questions:

1. Start with the intended learning outcomes

What skills and knowledge are you assessing? How will students provide evidence of achievement? Which skills will they need to make use of their knowledge?

Intended learning outcome verbs should provide direction towards the choice of question type. Verbs such as identify, select, arrange, distinguish, match indicate that students need to determine the correct response. Verbs such as analyse, interpret, compare, summarise, indicate that students should construct a response.


2. Encourage greater engagement with critical and analytical thinking

Online open book exams require questions which encourage greater engagement with critical and analytical thinking, requiring students to apply knowledge, rather than simply recalling it.

What examples can you find? What does this data show? Is anything missing? How could you provide this? Multi-step questions, where one question builds on the previous one, can work well. 

3. Think about your choice of verb

Think about your choice of verb – if a student could simply look up the answer, you may want to find a different approach to the task that would make this more challenging. A range of example questions built from different verbs are available at the bottom of this page.


4. Use case-based scenarios

Case-based or problem-based exam questions require students to apply critical reasoning skills in response to a given scenario. This type of scenario-based question requires a more multi-faceted, longer answer response and can help students put themselves into the role of a professional in their field. Longer questions will also test a student’s ability to reason, create, analyse, synthesise, and evaluate and demonstrate higher level skills and knowledge.

Using specific case studies will reduce the chances of an essay mill being able to provide a stock answer. 


5. Ask for application

Locating data or resources should be followed-up with the need for students to probe and apply their findings, rather than just re-write information. Students can submit workings, calculations, proofs, or justifications for their answers.


6. Ask for interpretation

Present relevant qualitative or quantitative data and then ask interpretative and application questionsWhat does the data show? What relevance does this data or does the scenario have in terms of…? What other factors could potentially affect this data? How would you test for these? 


7. Be specific

Give clear instructions to help students answer the question e.g. 'Provide a hand-drawn graph...'.

Asking questions which are linked to a specific element of a course requires students to apply their knowledge, demonstrate their critical thinking, and synthesise new ideas e.g. ‘Using the materials and your notes from weeks 6-9, evaluate the importance of...'.


8. Encourage good academic practice

Recommendations for encouraging good academic practice by your students include:

  • Setting questions that require students to interpret or apply subject material rather than simply locating, defining or rewriting information. 
  • Requiring students to submit workings, calculations, proofs or justifications for their answers. 
  • Requiring student responses to be contextualised to their own experience. 
  • Providing guidance on ethical scholarship, such as reminding students to work through the Academic Integrity training on Referencing and Avoiding Plagiarism.

For some exam types, the ability to randomise questions by drawing them from a larger base, or shuffling the sequence, can be beneficial and help reduce opportunity for academic misconduct (Aparna Chirumamilla et al. 2020).


9. Provide clear guidance

Where students are required to spend time searching for and interpreting information or data in an open book exam, it is important to make allowances for the length of expected answers as most will not be able to write as much per hour as in a traditional closed book exam. 

Add a suggested word length for essay-style answers, both to guide students and to manage marking. Where relevant, you can provide an explicit limit on what is needed, such as 'Provide three bullet points to justify your answer' or 'Provide one short sentence for each issue you feel relevant'. Students should spend time searching for and interpreting information/data in an open book exam, so may not be able to write as much per hour than a closed book exam. 


10. Make the most of formative assessment

Allow students to practice and familiarise themselves with open book exams and different styles of questions before any summative open book assessment. This could be in your unit, or in other units in the year.

Ask students to time themselves when doing a set task with particular resources on their own, then discuss their results in a class session where they can mark their own or each other’s work.  

Formative assessment should include any tools that you are expecting students to use in the exam (e.g. Matlab).

Students can reflect on their approach to an open book exam: How did they revise or prepare notes? How effective was their approach?

Example questions

Adapted from: A Guide for Academics – Open book exams 

Applying  What examples can you find to…? 

How would you solve X? 

How would you use…? 

What would happen if…? 

What effect would that have? 

Analysing   What are the parts/features of..? 

Compare and contrast A and B? 

What is the relationship between A and B? 

Why is X different to Y? 

Creating  How would you design a …? 

What changes would you make…? 

What alternatives are there to…? 

How would you evaluate…? 

Evaluating  What does this data show? 

Is anything missing? How could you provide this? 

What methods would be effective?  

Which method is best? 

Reasoning   What would be an example of this? 

What other information do we need? 

Can you explain your reasoning? 

Is there reason to doubt this data/evidence? 

Implications and consequence questions 

 

What effect would that have?  

What is an alternative? 

What are you implying by that?  

If that happened, what else could happen as a result? Why? 

Viewpoint questions  How would other groups of people respond to this question? Why?  

How could you answer the objection that ______would make?  

What might someone who believed _____ think?  

What is an alternative?  

How are ____ and ____’s ideas alike? Different? 

curriculumdev@bath.ac.uk

Updated on: 27 October 2020