Adventures in Technology Enhanced Learning @ UoP

Tag: exams

Online assessment in the time of Covid

In pre-Covid times, exams delivered via Moodle were limited by the availability of suitable physical spaces. Exam rooms represented a bottleneck to the number of students taking exams concurrently.

For the last year, we’ve used Moodle (and integrated platforms) to deliver the majority of our teaching and assessment online.

A visualisation of the online assessment mix at the University of Portsmouth:

Diagram of how the Assignments and the Exams overlap during assessment period

In May 2020 many academics who had previously planned to deliver paper-based exams had to quickly adapt and deliver online assessments. In some cases, these required students to scan or take pictures of their work and upload these to assignments (Moodle or Turnitin) for marking. 

In recent months, newer platforms to handle this workflow and ease the marking burden for academics have been developed – platforms such as Turnitin Gradescope and CrowdMark. These platforms leverage the similarities in students’ answers so academics can mark many answers at once. When time allows, we hope to be able to evaluate these platforms in more detail.

In the diagram above you can see “Assignments under exam conditions” as the meeting point between traditional essays and restricted online exams. This year we have seen a big growth in this area as academics move from paper-based written exams to time-restricted assignments. An obvious caveat here is that these haven’t been conducted under true exam conditions and so are best described as open book exams. Many digital assessment platforms now include various types of proctoring and would be able to handle remote time-restricted essays (and other assessment types) securely. There are, however, a number of ethical issues to be considered with online proctoring, and we need to proceed cautiously here. 

As a University, I feel we should also be looking to expand our capacity for online assessment as over the next decade we will probably see the end of paper-based exams in favour of typed essay papers delivered online due in part to student expectations.

Academics have had a year to adapt to exams in lockdown and many have discovered the benefits of Moodle quizzes for exams that offer automatic marking. (And note that Moodle is excellent at delivering scientific and mathematical exam questions as well as longer coursework assignment submissions.) Generally speaking the Technology and Science and Health faculties deliver the majority of our Moodle quiz based exams and the number of exams has grown significantly during the lockdown. Many academics don’t want to go back to paper.

In Technology Enhanced Learning we oversee online exams and assessments in terms of supporting and evaluating the digital tools and making sure Moodle can handle the number of exams thrown at it. The number of online exams has increased substantially over the last year, all funnelled into two exam windows. As a team we work closely with colleagues in IS to provide more capacity in Moodle and with timetabling to ensure the exams are evenly distributed to avoid terminal peaks of concurrent users, providing a stable Moodle platform for all users.

Without the bottleneck of physical exam rooms, the January 2021 exams were initially weighed in the favour of academic requests around having exams earlier in the day and only using the first week of the exam window to maximise available marking time. Unfortunately, this translated into a scenario that would have presented a significant number of terminal peaks of concurrent users on Moodle. Members of TEL worked closely with the central timetabling unit to level out these peaks and with the exception of one or two slow points, we all delivered a successful exam window in January.

In advance of the May/June exams, we have gone further and set hard parameters around how many exams (quizzes) or timed assignments (Turnitin or Moodle assignments) can be timetabled in any given time slot. We’d like to thank CTU for their tireless effort to make this happen. It wasn’t an easy job to manage all the necessary requirements but it’s given us an exam timetable that looks like the image below. This really is invaluable work to the University when assessment represents so much effort by students, academics and support staff.

A screenshot of the exams for a week, days, dates, section, then slips into assignments and exams then the total of students expected to be in Moodle during that period

Our increasing reliance on online assessment means, I think, that we should investigate new technologies to support that function. Platforms such as Gradescope or CrowdMark could help relieve the marking burden; one of the many platforms such as Wiseflow or Mettl or Inspera could provide extra exam capacity (with the functionality to proctor exams if that was something the University wanted to do). Moodle, with its advanced quiz and assignment capabilities, would continue to play a key role.

I believe we will get through this coming assessment period well, but as our reliance on online assessment grows so must our technologies to support it. 

As a University the Covid-19 pandemic has been a driver for the uptake of online learning and assessment. As a University community, we need to harness this positive momentum and diversify our offering of assessment platforms to support students and staff.

Credit Image: Photo by MayoFi on Unsplash 

Assessment online – Are we past the “hand-in” date?

Introduction

In eLearn, we have just reached the end of the exam period with our faculties intact (excuse the pun) and with very little drama (which is not normally the case). The sight of nervous students queuing up outside of Spinnaker for an exam inside a gym hall bought all those memories of dread I had experienced nearly 20 years ago flooding back.

When I think about how much has changed in the teaching landscape in terms of the integration of technology into teaching, as well as the diverse ways in which people attend university, I can’t help but feel this method of summative assessment is rather antiquated.

This could very easily turn into a blog about the nature of summative assessment, which I wrote far too many assignments about in 2004 as part of my teaching degree. I don’t want this to turn into a virtual trip down memory lane for myself but a means to highlight what is different and future possibilities.

The wonder of Turnitin

With my teacher hat firmly still on my head, I can’t be more positive about this technology when it comes to marking, having lived the late nights devoted to marking never ending piles of papers. True, it has its faults and the late nights may have merely been transferred from pen and paper to in front of a screen but it has so many facets designed to make the experience easier for both marker and student. You can’t help but feel its implementation has been a large forward step in the progression of assessment. Being able to customise and apply quickmarks across assignments prevents the numerous occasions “RTQ” would have to be written. The possibility of copy and pasting comments or highlighting text to directly link to aspects of a rubric are all seemingly small things that actually take hours when going through the work of 90 students and that is before you give personalised feedback that moves learning on.

The student gets a rich visual experience that can be accessed on any device and feedback is so easily obtainable/downloadable that it could only promote reflective practise. The hand-in process has changed dramatically with the long line outside of the faculty admin office with bound assignment in hand is a thing of the past and it can now be submitted in bed with a cuppa. Don’t get me wrong, you will still get students who will leave it till the last minute and those who perhaps have been a little too influenced by other sources within their writing but nevertheless a snapshot of this process in 2019 vastly differs from 2009 and is a world away from 1999. The same of which can’t be said for the end of year exam.

Quizzes – More than just for daytime tv

Perhaps it is slightly unfair to portray examinations at university to be solely desk based due to the increase in exams being carried out online using Moodle Quiz. The Quiz tool is far more powerful and robust than perhaps people realise. Yes you can use it to create multiple choice “pop quizzes” for the end of topic or to elicit prior conceptions at the start of something new but it can also be used to make 100 questioned essay-based behemoths which include a variety of different question types. Safe Exam Browser allows for it to be taken under true exam restrictions and the ease in which times and restrictions can be customised makes them far more accessible than its paper-based counterpart. Claro Reader software can be used to overlay colours and intuitively applies text-to-speech (dependant on how the exam has been written of course!). The possibility of including image or video within an exam assessment not only opens up a wealth of ways to question but leads me on to my next point.

The Audiovisual Essay

I was very fortunate to have witnessed a presentation from the inspiring Dr Catherine Grant who spoke about the concept The Audiovisual Essay in Film & Moving Image Studies. I would certainly recommend visiting the website, which explores the concept in great detail. There are some amazing examples and relevant research that has been undertaken about the subject. For those who are unfamiliar with this form it is essentially the expression of critical, analytical and theoretical work using the resources of audiovisuality (images/sound/video in montage) I begrudge trying to pigeon hole the genre further but it truly flies in the wind against sitting in a hall for 3 hours writing an English Literature exam. While it lends itself to creative, historical, visually rich courses and cannot be applied across the board, the premise of it being a “different” way to demonstrate understanding is valid.

Final Thoughts

This brings us back to assessment types and again perhaps explains the shift towards the greater emphasis on coursework-based assessment models. That in my eyes is a different debate, this blog is exploring whether sitting in hall to carry out an end of year assessment still has a place in modern university life. You have to question over their time in Higher Education, how many opportunities students get to sit at a desk for a considerable time and demonstrate their understanding in that way. Are we providing students with a rather unnatural medium by which to demonstrate their understanding? Does that in turn affect their ability to reach their true potential? Particularly as the end of year summative assessment the culmination of the blood, sweat and tears of their learning journey, do we not owe it to the learner to reassess the way we make this final assessment. The flip side of this is to give students more exam practice and opportunities but is this a direction where we want to go? To me that seems to be a practice that would be looking in the rear view mirror where I would argue we should have our eyes on the road ahead.

 

Featured Image:

Image by PublicDomainPictures from Pixabay

Online Exams in Moodle

We’ll start to see a lot of online Moodle exams from Monday 29th April.

At this time of year a lot of time is dedicated to preparing for the formal exam weeks to begin. Academics and Online Course Developers are creating questions and testing exam quizzes. The TEL team are testing the Moodle infrastructure and exam reporting, and working with IS to ensure we have enough server resources at key times during the exam period. This year the formal examination weeks will run from Monday 13th May 2019 through to Friday 7th June 2019. It’s worth making a note in your diary about these dates but as I’ll highlight in this blog post, you’ll probably want to set these dates to start from Monday 29th April.

What constitutes an online Moodle exam?

We encourage staff to tag Moodle-based exams with the ‘exam-official’ tag and set appropriate dates. This information, along with data provided by Online Courses Developers, helps us build a picture of the when exams are happening. It means we can spot ‘pinch points’, times where we expect a lot of concurrent exam attempts, which could represent a need for more server resources to be made available. In short if you don’t tag your assessments we’ll find it difficult to guarantee a seamless exam experience for your students.

Tagging also means you can run your online exams in one of our supported secure exam browser environments (Safe Exam Browser, FAQs available here or Chromebooks in secure exam mode). This means students can’t easily access other websites during their exam and makes invigilation a lot easier.

How many online exam attempts were there last year?

During the official exam period last year (13.05.2018–07.06.2018) there were 5426 exams attempts that we all helped support. However, this figure isn’t the full picture. In the two week preceding the exam period 2737 official online exam attempts took place. That’s quite a staggering number of exam attempts happening outside of the time-frame that staff are focussed on supporting and something we should all be mindful of this year. Monday April 29th is the start date for your diaries.

Exams change freeze

IS and TEL enforce a number of Moodle change freezes throughout the academic year. These are periods when no updates can be made to Moodle. We have one in place during the exam period to ensure changes don’t inadvertently interrupt exams. The exam change freeze this year will run from Monday 29th April through to Friday 7th June (inclusive).

How do we monitor exams are going well?

The TEL team gather data from a number of sources to help monitor what’s happening through the exam period. We combine it into a real-time dashboard of where exams are happening and how Moodle is performing. You can see a picture below from last year.

Exam Analytics Dashboard

We rely on support from a number of sources to produce this dashboard. Lead Online Course Developers have their finger on the pulse of when exams will happen and any last minute amendments. In the very near future we’ll be asking again for your help to populate a spreadsheet with the exams you know about. We combine this information with data from student records, which is thorough but lacks local knowledge. We also use Google Analytics, Moodle database and server infrastructure reporting to keep an eye on how our systems our performing. This combined data is extremely useful at spotting pinch points and monitoring how things are running but it’s not as effective as people in faculties such as exam invigilators and online course developers ensuring all is running well and reporting to elearn@port.ac.uk any issues that are encountered.

A new exam theme for Moodle

It’s worth mentioning at this point that we’re close to final testing of an updated version of the Moodle exam theme. This is a stripped down version of our theme intended for use in exams. We’ll provide more information on this in the next few weeks. It will look very similar to the existing exam theme but will be a bit closer to our regular theme in terms of question layout and styles.

Thank You

We just wanted to say a big thank you to everyone involved in making sure the exam period goes well. It’s very much a big team effort and all the work we put in ensures students get as stress-free an experience as possible. I’m sure they appreciate it, I know that the team in TEL certainly do.

If you have any questions about online exams, please get in touch at elearn@port.ac.uk

 

Thank You Image

Image credits: Photo by Lip on Unsplash

Guest Blogger: Lucy Sharp – Sleeping your way to a good degree

This may not be what you think it is, at least I hope not!

Humans and animals all need to sleep, how much, when and where will vary but the constant is that sleep is an essential part to living and learning. The quality and quantity of our sleep is a major indicator of our overall health and wellbeing.

We spend up to a third of lives asleep and most of us know that getting a good night’s sleep is important, but too few make the recommended 8 hours between the sheets. This can lead to having a sleep debt and forgetting the feeling of being truly rested. This third of our life is far from unproductive as it plays a direct role in how energetic and successful the other two thirds of our life can be.

When we’re asleep the body re-energises cells, clears the brain of waste, and supports learning and memory; two pretty important factors when you’re a student. It also affects the way we look, behave, perform and impacts on our overall quality of life. At different ages we need different amounts of sleep. Typically teenagers need at least 8 hours—and on average 9¼ hours—a night of uninterrupted sleep to leave their bodies and minds time to be rejuvenated for the next day. If sleep is cut short or disturbed the body doesn’t have time to complete all of the phases needed for muscle repair and memory consolidation, neither does the brain have time for its complex clean-up operation (brain cleaning). The effect is that we wake up less prepared to concentrate, make decisions, or engage fully in the learning experience.

The sleep cycle follows a pattern of NREM (non-rapid eye movement) and REM (rapid eye movement), throughout a typical night the pattern repeats itself every 90 minutes. In the NREM phase one, we begin to transition from being awake to falling asleep. In stage two the onset of sleep begins where we disengage from our surroundings, heart rate and breathing becomes regular and body temperature drops. Stage three is our deep and restorative sleep: muscles are relaxed, tissue growth and repair takes place along with the release of growth hormones and our energy is restored. After about 90 minutes we enter the REM phase, this is when we dream, our brains are active and our eyes dart back and forth and our body is immobile and fully relaxed as our muscles are turned off.

So there’s a lot going on when you tuck yourself in at night, but of course in reality we don’t always get a full and restful night’s sleep and the impact is far reaching. The effect of a poor night’s sleep may stay with you for about 48 hours. Other impacts are more immediate, such as feeling groggy, irritable and the urge to consume sugary drinks, food and extra carbohydrates. Therefore, if your weight is increasing, try spending an extra hour in bed!

As well as the health reasons for getting a good night’s sleep, there are also the physiological and psychological reasons why sleeping well will help you to study and learn.

Physiologically, a sleepy brain has to work harder and isn’t as efficient as a rested brain. This is due to diverting more energy to the prefrontal cortex to stave off tiredness. The effect when we’re learning is that our short and long term memory is shot. This means the brain holds a smaller amount of information for a shorter period of time. The impact is that we go round in circles trying to remember what we’ve learnt, and we find it difficult to perform complex tasks and sometimes even simple tasks, such as reading text. The common example is reading a body of text and not remembering what has been read, or simply reading the same sentence over and over again. This isn’t a great state to be in when you’re studying at a higher level.

A healthy amount of sleep is needed for the plasticity of brain which is a vital component of our ability to adapt to input. If we sleep too little then our ability to process information is lessened, as is the process of remembering what we have learnt and then recalling it in the future, such as in exam settings.

Psychologically, lack of sleep may be the culprit if you’re feeling low in mood, less enthusiastic about activities you used to enjoy and it can chip away at your happiness. In a nutshell, not getting enough good quality sleep heavily influences your outlook on life, energy levels and emotions. 

A regular sleep pattern is the foundation needed to enjoy life and engage fully with the learning experience. Without it, it can affect health, wellbeing and the learning process, the ability to perform at your best in exams and presentations, and achieve those high marks you know you’re capable of reaching.

Tips on how to get a good night’s sleep:

  • Sleep at regular times this allows the body to get into a routine.
  • Make your bedroom sleep-friendly by keeping it dark, clean and tidy. Your bedroom is for two things; sleep and sex.
  • Wind down before going to bed, switch off the TV and electronic devices. The blue screen tricks your brain into thinking it’s daytime. Install software that enables your computer’s display adapt to the time of day, such as f.lux
  • Calm your body and mind (Mindfulness Exercises for Everyday Life).
  • Try not to spend the night in the library revising and studying. Nighttime activity disrupts the circadian rhythm. This is your body clock, this responds to environmental cues such light and temperature.
  • Avoid caffeine and energy drinks 46 hours before heading to bed.
  • Read University Health Service SLEEP HYGIENE.
  • and Skills for sleep at UoP for more advice.

Image Credits:  Photo by Toa Heftiba and Cassandra Hamer on Unsplash

 

Continue reading

Chromebooks

Have you ever wanted to incorporate some online activity into your session, but don’t have the facility to do so?  Here, in the TEL department we have 30 Chromebooks which are available for morning and afternoon sessions or can be booked for the entire day.

So how do Chromebooks work?
The Chromebooks have two preset profiles that can be assigned through the admin panel. The first is defined as “Classroom mode”, the second is “Exam Mode”.

In Classroom mode the Chromebooks loads a Google login box where the users university details are added. Chrome OS then loads and allows the user to access their work Chrome profile, this will include access to email, drive and any other documents within their Google profile.

Exam mode is much more stringent, and automatically logs into the device and displays the exam landing page. The student would then choose the exam they are expected to take, at which point they are then asked to sign into Moodle with their credentials. They are taken to the title screen of that exam which will display start time, end time (if set) and duration of the exam.

Should there be another requirement for a different Chromebook profile then through discussion with IS it may be possible to create one that would suit the potential need. As an example: Science made a request for exam mode to be enabled with access to a shared Google Drive document that still limited any other web access. This took over a month of testing and development between Science and IS to get the framework working and in place to use. Some requests that have been made however, were not possible and subsequently implementation was not possible.

Unlike standard Chromebooks or laptops, the TEL Chromebooks require a University of Portsmouth Google account as they’re subject to authentication  by Google.  So if you’re thinking of borrowing the Chromebooks to use with external participants, IS will need at least 72 hours notice to give them time to create dedicated accounts. If you required a large number of external accounts you would need to contact IS directly: servicedesk@port.ac.uk

Booking the Chromebooks
If you would like to borrow the Chromebooks we would require you to complete the TEL Chromebook Booking, Enquiry Form

This form asks:

  • How many Chromebooks do you require?
  • Which mode do you require?
  • Session Date
  • Session Start Time
  • Session End Time
  • Session Name
  • Session Location

You’ll need to complete an individual form for each session that you require the Chromebooks for. To make sure that the Chromebooks are in the correct mode we require a minimum of 72 hours from your initial booking to when you require the devices. Chromebooks are transported in wheel-able flight cases (15 Chromebooks per case), therefore it would be your responsibility to get them transferred to where you need them.  We’ll make sure that they’re ready at least 30 minutes before your session starts, for you come to collect them.

If you’re interested in borrowing the Chromebooks, but not sure in what capacity and would like further explanation or demonstration then please contact elearn by either telephoning extension 3355 or email us at elearn@port.ac.uk and we can provide some advice on how they have been used before around the University.

Image credits: Photo by Helloquence on Unsplash

Making online exams work for you

When it comes to online exams there are a number of questions that cause headaches for support staff and academics. Where am I going to find the time to create all the questions? How do I make sense of all these settings in a Moodle Quiz? How can I keep an eye on so many students during the exam itself?

The simple answer to all these questions is normally to speak to the right people. The first port of call, if you’re interested in getting started with online assessment, it’s your friendly Faculty Online Course Developer (or the central eLearn team), who will be happy to advise or point you in the right direction.

Moodle is of course not the only tool for conducting online exams, but it is very good at handling large groups of students who are attempting many questions all at the same time. These questions generally have a right or wrong answer, most of which can be automatically marked. Essay questions can also be posed, but these will require manual grading. (Many students these days have difficulty in writing by hand  for three hours, so if your exam is heavily essay-based you might want to investigate a tool such as DigiExam, which  allows students to type their answers (contact the eLearn team for more information about DigiExam).

A tremendous amount of question-writing effort has already been made at UoP by staff across faculties. There are close to a million questions already in Moodle, most created directly by staff but with a significant percentage having been imported from existing Word documents, shared by colleagues in other departments or institutions, purchased from commercial suppliers or imported from older systems. You don’t always have to start from scratch, as many academics already have treasure troves of questions that can be adapted or imported.

Once you have the questions you wish to pose, your next step will be setting up the quiz that will be used to deliver the questions. This annotated pdf of typical Moodle exam settings walks you through the various quiz settings (many of which are set to the optimum setting by default). Your Faculty Online Course Developer will be able to help out here, and also assist with the important job of testing the quiz or exam.

By this point you’ll have a working, thoroughly tested Moodle quiz that you could use for a summative assessment. As a member of staff you’ll have gone through a process of familiarisation. It’s important that you allow your students the same familiarisation with the online exam process (what to expect on exam day, how the software works and so on), not to mention any administrative staff and moderators who will be involved. It’s advisable to schedule some mock exam sessions well in advance of your first exam so your students are fully prepared when it comes to the real thing. Although it’s by no means compulsory, Safe Exam Browser (SEB) can be leveraged here. SEB is a web browser, available on all student PCs, which locks students down to a single Moodle quiz and prevents them from accessing other web sites or resources. SEB will help you keep an eye on large groups of students and be certain they are concentrating on the task at hand. Take a look at this Safe Exam Browser FAQs if it’s something you might be interested in. DCQE also have a set of 30 Chromebooks which can be locked down into exam mode potentially turning any wifi enabled room into an exam room. More information along with the Chromebook booking form can be found here.

Hopefully this blog post has sparked your enthusiasm for giving online exams a go. The keys to success are (i) getting in touch with your faculty online course developer who can help you at various points along the way, and (ii) starting with non-critical familiarisation exercises which give room for finding the edges of online assessment. It’s fair to say that you will have to dedicate a bit of time to start with creating quiz questions, but the downstream benefits of online assessment can be significant.

Some useful resources

eAssessment at the University of Portsmouth

Quiz support materials for staff

Quiz questions examples and templates

DigiExam

Image credits: https://pixabay.com/p-1828268/?no_redirect

© 2024 Tel Tales

Theme by Anders NorénUp ↑