Situational judgement assessment

So I think it is fair to say, Covid-19 has thrown us all into having to think outside of the box when it comes to developing the usual, more ‘normal’, course assessment delivery types. This couldn’t be more true for our academics within the Paramedic Cert HE course. 

A few weeks ago, I was asked to support a colleague within this team to help deliver an online assessment that would replicate what would have been a practical examination on adult and paediatric resuscitation. 

With COVID not going away anytime soon, and practical assessment dates looming, we had a very short space of time to develop an online assessment that would best simulate ‘hands-on’ resuscitation scenarios.

As a response to the need to deliver an alternative assessment, Jane Reid – the Course Leader for the Paramedic Cert HE course – wrote four situational judgement assessments; each scenario had sequential situations and each situation had a series of serious judgement questions to test a student’s knowledge of actions to take within the ever-changing scenario.

Situational judgement assessment has been used in healthcare for years. It allows participants to experience as close to real-life scenarios as possible, without risk (in our case COVID), enabling them to identify, in order, their responses to given situations.

Storyboarding a situational judgement test:

1. Scenario – Includes detailed descriptive text, containing key information that sets the scene. This can also include images, audio or video to further illustrate the scenario.
2. Situation 1 – Content that builds on the initial scenario, it contains the next layer of information relating to the scenario at hand and includes the first set of serious judgement questions.
3. Situation 2 – Content builds on the previous situation and includes the next set of serious judgement questions… and so on until the end of the scenario.

There were many considerations that had to be made whilst developing this assessment type – mainly to keep the assessment as authentic as possible. For example:

• providing media to set the scene; 
• keeping the narrative on track – ‘time’ is of the essence in any resuscitation scenario, so it was important to include timely details within the situations; and 
• replicating the quick thinking process that would be required in a real-life situation by using the sequential format in the quiz, so that students had to take notes or work from memory as they couldn’t return to previous situations to guide them.

The student experience was another really important factor in delivering this assessment – most of whom may never have experienced this type of examination. It was essential to provide clear and consistent instructions to guide them through this process. Before the main assessment, we also created a formative version of the quiz so that students could familiarise themselves with what was expected from the assessment.

We used a Google Document, with tables, to structure the content in the development stage and a Moodle Quiz Activity (Multiple Choice) to build and deliver the assessment which worked very well. The feedback from both students and examiners has been really positive, with more scope for using this assessment as a CPD exercise for practitioners. 

Interest has also been shown by academics at other universities who wish to explore this particular method of assessment along with Ambulance Trust managers. The methods for assessing the learning of resuscitation has seen little evolution from the traditional OSCE format therefore, this format that was created for a small group of students may well develop over time.

Developing this was by no means easy given the time constraints. However, it is a great example of an alternative assessment that has been developed from creative thinking during the lockdown.