Published on: 10/07/2023 · Last updated on: 03/09/2024
Sources of difficulty
Have you set an assessment, only to be bombarded with queries from students about how to approach the task? Or, marked work to find that the cohort has missed the point of the assessment?
Writing a clear assessment brief is a skill that is fraught with challenges and requires practise. Assessments should have elements of difficulty built in that support academics in making valid judgments about the quality of the response – these are legitimate difficulties.
However, where assessments include or omit information that benefits some groups of students over others, this prevents valid judgements from being made regarding student ability and instead increases awarding gaps and time spent on fruitless feedback. These, often unconscious barriers, are known as illegitimate difficulties.
Legitimate sources of difficulty
Intentional demands of the assessment question. They allow assessors to make judgements about the quality of a response:
- Open v closed questions
- Abstract v concrete concepts
- The number of cognitive processes
Illegitimate sources of difficulty
Unintentional features of an assessment brief that give unfair advantage or disadvantage to some students:
- Inaccessible language
- Cultural-specific language or examples
- Irrelevant additional materials
Checking your brief
It is also good practice to consider the ‘bigger picture’ of an assessment from the perspective of a student, especially if several colleagues have collaborated to create a single exam or assessment. Items such as diagrams and images intended to be a support, could instead be a point of confusion.
Check for:
- Additional resources (diagrams, images, case studies): do they provide useful support?
- Clarity: are there any superfluous words, negatives, ambiguous terms/words with more than one meaning?
- Presentation of the brief: do the layout, marks available, command terms align with what you would like students to do?
- Any intended/unintended clues that students may use to support their response.
- Intended level of difficulty: is the word count or number of reasoning/ processing steps appropriate for the marks and time available?
- Clarity of expectations: is there one clear way of a approaching the task? Would different approaches be acceptable?
Further approaches
Where time and budget allow, employing a ‘scribe’ can be an excellent insight into the effectiveness of an assignment brief. A scribe’s role is to attempt an assessment before it is set for the majority of students. They complete the task and provide feedback on the level of difficulty involved- this would usually be someone who has completed the relevant level of the course previously and should be able to complete the task with relative ease.