Adventures in Technology Enhanced Learning @ UoP

Tag: assessment (Page 2 of 3)

Great feedback is essential

Wouldn’t it be great if students could read the feedback they’ve received for their assignment, write a short reflection on what they could do to improve (perhaps also identifying what they’d like to receive feedback on next time round) and then see their grade? 

Our current online assessment tools (Turnitin and Moodle Assignment) don’t allow us to do this. Luckily we know an assignment tool that does – and it has many other modern assessment feedback mechanisms too.

I’m passionate about helping improve assessment feedback for students. It’s one of the things I’ll be working on in my new secondment as a Senior Lecturer in Digital Learning & Innovation. On Mondays, Tuesdays and Wednesdays I’ll be working between the TEL and AcDev teams to help coordinate projects to better support academics, Online Course Developers and students with a focus on digital education. In particular, I’ll be working to help get a small pilot off the ground for EdWord – a fantastic new assessment tool that promises to address many of the requirements of modern assessment and feedback. If you’re interested in taking part in this pilot please let me know.

In addition, I’ll also be helping to establish an online staff community alongside the APEX programme featuring special interest groups. This will be a great place to make contact with like-minded staff from other faculties and exchange ideas.

Tom Langston and I will be creating a support mechanism for Online Course Developers who are interested in completing their CMALT portfolio and who might be interested in taking part in future elearning projects with TEL.

I’ll also be doing a bit of lecturing on the Research Informed Teaching programme, which I’m looking forward to. So this will be a busy year for me!

Please get in touch if you’ve got any ideas or projects we can help you with. Both the TEL and AcDev teams would appreciate  your feedback as we work to ensure we’re offering the services that will provide value to you and your students (you can reach me on ext. 3194).

Image credit: https://commons.m.wikimedia.org/wiki/File:Paper_Plane_Vector.svg

 

Minerva – the university rethought?

On 25 June I attended an Adobe/Times Higher forum called “Making digital literacy a pillar of education”, along with representatives from 40 or so other HE institutions.

There was no disagreement at the forum about the recent recommendation from the DCMS Select Committee that digital literacy should sit alongside the “3R’s” as a fourth pillar of education. Everyone agreed that, as the pace of technological change quickens, employers are less interested in a student’s knowledge than in their personal qualities – and in particular their ability to engage in lifelong learning. But there was no consensus on how universities can best prepare their students for life in a world in which digital technology will play an increasingly important role.

Of the institutions present at the forum, undoubtedly the most innovative approach to Education 4.0  was that adopted by the Minerva Schools. Minerva built a first-year undergraduate curriculum from scratch, but rather than base the curriculum on subject-specific knowledge they built it around 81 “habits of mind” and “foundational concepts”. Students engage in cross-contextual learning activities in small-seminar format, all of which require or exercise the use of those foundational concepts. Through these activities students pick up subject knowledge, but they are assessed on how well they satisfy the foundational concepts. 

In the first year of study the Minerva School’s students are based in San Francisco. Subsequently they spend time in Seoul, Berlin, London, Hyderabad, Bangladesh, Buenos Aires and Taipei. Sounds terrific! (And expensive…) And all of this is made possible using digital technology – it’s a fundamental enabling technology for Minerva.

Minerva Schools were able to take this approach because they were small, well resourced – and also because they were starting from scratch. It would be a huge task (probably an impossible task) for an existing university with thousands of students to change its curriculum in this way. But there might be elements of the approach that universities can adopt. It’s interesting that the Minerva project have recently opened its bespoke educational technology platform, called Forum, to partners: they claim that the platform, which was designed for use in a small-seminar format, can scale to support up to 400 students. It will be worth keeping an eye on this development. 

Image Credit: Commons Wikimedia: The Greek Goddess Minerva

Assessment online – Are we past the “hand-in” date?

Introduction

In eLearn, we have just reached the end of the exam period with our faculties intact (excuse the pun) and with very little drama (which is not normally the case). The sight of nervous students queuing up outside of Spinnaker for an exam inside a gym hall bought all those memories of dread I had experienced nearly 20 years ago flooding back.

When I think about how much has changed in the teaching landscape in terms of the integration of technology into teaching, as well as the diverse ways in which people attend university, I can’t help but feel this method of summative assessment is rather antiquated.

This could very easily turn into a blog about the nature of summative assessment, which I wrote far too many assignments about in 2004 as part of my teaching degree. I don’t want this to turn into a virtual trip down memory lane for myself but a means to highlight what is different and future possibilities.

The wonder of Turnitin

With my teacher hat firmly still on my head, I can’t be more positive about this technology when it comes to marking, having lived the late nights devoted to marking never ending piles of papers. True, it has its faults and the late nights may have merely been transferred from pen and paper to in front of a screen but it has so many facets designed to make the experience easier for both marker and student. You can’t help but feel its implementation has been a large forward step in the progression of assessment. Being able to customise and apply quickmarks across assignments prevents the numerous occasions “RTQ” would have to be written. The possibility of copy and pasting comments or highlighting text to directly link to aspects of a rubric are all seemingly small things that actually take hours when going through the work of 90 students and that is before you give personalised feedback that moves learning on.

The student gets a rich visual experience that can be accessed on any device and feedback is so easily obtainable/downloadable that it could only promote reflective practise. The hand-in process has changed dramatically with the long line outside of the faculty admin office with bound assignment in hand is a thing of the past and it can now be submitted in bed with a cuppa. Don’t get me wrong, you will still get students who will leave it till the last minute and those who perhaps have been a little too influenced by other sources within their writing but nevertheless a snapshot of this process in 2019 vastly differs from 2009 and is a world away from 1999. The same of which can’t be said for the end of year exam.

Quizzes – More than just for daytime tv

Perhaps it is slightly unfair to portray examinations at university to be solely desk based due to the increase in exams being carried out online using Moodle Quiz. The Quiz tool is far more powerful and robust than perhaps people realise. Yes you can use it to create multiple choice “pop quizzes” for the end of topic or to elicit prior conceptions at the start of something new but it can also be used to make 100 questioned essay-based behemoths which include a variety of different question types. Safe Exam Browser allows for it to be taken under true exam restrictions and the ease in which times and restrictions can be customised makes them far more accessible than its paper-based counterpart. Claro Reader software can be used to overlay colours and intuitively applies text-to-speech (dependant on how the exam has been written of course!). The possibility of including image or video within an exam assessment not only opens up a wealth of ways to question but leads me on to my next point.

The Audiovisual Essay

I was very fortunate to have witnessed a presentation from the inspiring Dr Catherine Grant who spoke about the concept The Audiovisual Essay in Film & Moving Image Studies. I would certainly recommend visiting the website, which explores the concept in great detail. There are some amazing examples and relevant research that has been undertaken about the subject. For those who are unfamiliar with this form it is essentially the expression of critical, analytical and theoretical work using the resources of audiovisuality (images/sound/video in montage) I begrudge trying to pigeon hole the genre further but it truly flies in the wind against sitting in a hall for 3 hours writing an English Literature exam. While it lends itself to creative, historical, visually rich courses and cannot be applied across the board, the premise of it being a “different” way to demonstrate understanding is valid.

Final Thoughts

This brings us back to assessment types and again perhaps explains the shift towards the greater emphasis on coursework-based assessment models. That in my eyes is a different debate, this blog is exploring whether sitting in hall to carry out an end of year assessment still has a place in modern university life. You have to question over their time in Higher Education, how many opportunities students get to sit at a desk for a considerable time and demonstrate their understanding in that way. Are we providing students with a rather unnatural medium by which to demonstrate their understanding? Does that in turn affect their ability to reach their true potential? Particularly as the end of year summative assessment the culmination of the blood, sweat and tears of their learning journey, do we not owe it to the learner to reassess the way we make this final assessment. The flip side of this is to give students more exam practice and opportunities but is this a direction where we want to go? To me that seems to be a practice that would be looking in the rear view mirror where I would argue we should have our eyes on the road ahead.

 

Featured Image:

Image by PublicDomainPictures from Pixabay

Digifest 2019

Jisc’s Digifest is my favourite educational technology conference. Spread over two days at Birmingham’s International Conference Centre it’s an ideal opportunity to meet practitioners from other institutions and this can be nearly as useful as the set programme of talks and presentations themselves. This year the focus was on practical ideas that can be taken back to the classroom or lecture theatre. To this end, I attended sessions on the use of video.

The first session, “How digital video innovates pedagogical methods”, was run by learning technologists from Havant and Southdowns College and looked at how staff and students have used video for both formative and summative assessment. Together, the technologists have worked on around 200 videos, some produced by the students themselves. It was clear that the use of video has had a significant impact on teaching and learning at the College. While claims about improved student attainment and retention lacked supporting data, feedback from both staff and students did indicate improvement in student engagement and achievement.

The importance of video in teaching and learning was also the theme of a presentation by Dominik Lukes’  (Learning Technologist, University of Oxford). In his presentation, titled “How YouTube started a revolution in learning and nobody noticed”, Dominik argued that the role of video in teaching and learning can only expand and provided this example of the power of video as a learning tool. Whilst I’m not entirely convinced by the claim that ‘video is the future’, both Dominik’s and the Havant College presentations did illustrate the usefulness and power of video if used properly. Both presentations also highlighted the ease of using YouTube as the delivery platform especially given the automatic subtitles function. It would be interesting to find out more about what students think of video as a learning tool and the extent to which benefits outweigh costs (in terms of time primarily) but from personal experience, I think a lot more use could be made of YouTube in teaching and learning.

Of the panel discussions I sat in on, the most interesting was “Listening to teachers: implications for education and digital“ The discussion was based on a piece of research by anthropologist Donna Lanclos and others the findings of which can be downloaded here. The research consisted of interviewing teachers in both the HE and FE sector with the aim “to uncover what next generation digital learning environments might look like”. Their report also came up with a series of recommendations which are well worth reading.

I was particularly interested in how terminology was being used. Many years ago when I was a student we went to lectures delivered by lecturers, in this discussion it was about “teachers” facilitating and delivering learning. This change in emphasis – teaching rather than lecturing – is something, I assume, that will become more ubiquitous given the introduction of the TEF in 2017. A number of thought provoking issues came out of the discussion not necessarily related directly to technology. For example, to provide effective holistic support to students takes time, which is one resource most staff are short of. This issue links nicely to a presentation I saw by Bolton College on the use of chatbots. But more on that in my next blog along with a look at Snatchbot!

 

Guest Blogger: James Brand – Erasmus+ Staff Training: Visiting the FernUniversität in Hagen (Part 2)

This is the second of four related blog posts about about an Erasmus+ Staff Training experience to the FernUniversität in Hagen which took place in June 2018. In this post I will be discuss the learning technologies being used at the FernUniversität and some of the innovative projects that they are undertaking.

Learning Technologies and Innovations at the University

E-Assessment

A presentation of E-assessment at the FernUniversität was very interesting and eye-opening. The FernUniversität has a number of issues unique to the institution in terms of conducting e-assessments. As the number of students is so high, it is difficult to conduct online exams. For example, one of their most popular courses has several thousand students. As a member of teaching staff highlighted, the exams cannot take place at a study center as they could not ensure that all students could attend. German data and privacy laws also mean that a lot of technical solutions could not be implemented. The University, therefore, does not conduct computer-based examinations.

Hybrid Lectures

One of the elements of teaching that I was most interested in was the use of hybrid lectures. These are delivered in a lecture hall but possible only to a handful of students. The rest of the students will be watching the lecture which is recorded and streamed online and will participate remotely using Adobe Connect software. A facilitator will then help with reading out student questions and to ensure that the session runs smoothly. One of our major challenges in the Graduate School is dealing with capacity issues of workshops and ensuring adequate provision to distance learners so the hybrid lecture looks to be a possible solution.

FEU login

One major project at the University is the introduction of implementing a single sign-on through a system being called ‘FEU login’. This will involve students signing in through a single portal to give them access to a number of online services including:

  • Classroom modules (on Moodle)
  • Student email
  • Student Calendar
  • A personal area for course and personal information
  • Library services
  • Regional Study Centre information and events
  • Learning Group Finder app

Learning Group Finder App

One interesting innovation at the FernUniversität is the development of the LernGruppen Finder, (Learning Group Finder) app. This app is designed to help the distance learning students to find others to study within their region or that are studying the same subject. This app has been introduced to support students studying and to help minimise isolation that students may experience from studying as distance learners.

Other examples of digital learning innovations at the University can be found on their E-Koo blog (German language only).

Study books

One of the most surprising revelations during the week for attendees was that paper is not dead at the FernUniversität. During a visiting to their Logistics Center, the printing, and production facility just outside of Hagen, we saw the scale of their use of non-digital learning materials. Students are provided with study books to accompany their courses. These have proved to be popular with students who like to receive something tangible to go with their studies so there are no plans to discontinue this method of producing course materials. The printing facility was very impressive on its own, with all aspects of print and design, provided in-house on large production line type setup. They also handle the production of other media including CDs with teaching material, newsletters, and stationery. 

Video Studio

One of my favourite parts of the training week was the visit to the facilities of the video studio. The media production team are responsible for the production of a range of audio-visual teaching materials including talking-head videos, video lectures, podcasts, and animated videos to name a few. They are also responsible for ensuring the accessibility of materials for all learners. The media production center features professional recording studio, with green-screen, lighting and cameras all of a professional quality. This studio was comparable to studios that might be used in news or television broadcasting. The facility had clearly had quite significant investment behind it and there was acceptance among attendees that this was needed to produce high-quality distance learning. To quote a delegate, “everyone wants distance learning until they have to pay for it!”.

In my next blog post I will be discussing some of the practices being implemented at the FernUniversität to ensure the quality of distance learning provision at University. I will also discuss instructional design and the educational theory that underpins e-learning at the FernUniversität.

 

Image credits: Photographs by James Brand

Episode 4 – eAssessment within the US – Cara North

TelTaleBanner
TelTales Podcast
Episode 4 - eAssessment within the US - Cara North
Loading
/

You can follow Cara on Twitter.

Jackie Van Nice PowerPoint Storyboard Template

Ohio State University – Ripley’s Believe it or Not “bad” assessment case study

Copyright Information:

Nowhere Land – Kevin MacLeod (incompetech.com)
Licensed under Creative Commons: By Attribution 3.0 License
http://creativecommons.org/licenses/by/3.0/

Design for Digital Learning

Back in 2005, Gráinne Conole and Karen Fill developed a learning design toolkit. Conole and Fill were concerned that, despite the increasing use in society of “Information and Communication Technologies” (as people used to refer to our connected world), educators weren’t embracing the opportunities of e-learning to enrich the student experience. Their learning design toolkit was intended to guide teachers through the process of creating “pedagogically informed learning activities which make effective use of appropriate tools and resources”.

In the 12 years since their learning design toolkit was developed, technology has continued to improve steadily – but I’d argue that learning design has failed to keep up. Perhaps a new JISC publication will go some way towards improving matters. Their new online guide to technology-enhanced curriculum design – Designing learning and assessment in a digital age – collates the most significant R&D outputs over the past decade in curriculum and learning design in a digital context. It also includes examples of good practice.

The guide is based on a model with four elements:

  • Discover – understanding of where you are now, and what you want to happen
  • Dream – your vision for what learning, teaching and assessment could be like
  • Design – understanding precisely who you are designing for and the pedagogic purposes that are appropriate to those students
  • Deliver – creating the right environment and culture for high-quality digital learning and assessment

If you are interested in learning design in a digital context then I can recommend reading the guide. Whatever your level of proficiency, you’ll be sure to find something to take away!

 

 

The discover, dream, design deliver model with improving student outcomes at the centre
©Jisc

 

 

Feature image: JISC Learning Design Toolkit by Graninne Conole and Karen Fill is licensed  under CC BY-NC-ND

UoP does the TESTA test! An introduction to the TESTA project

From January 2018 to September 2018, The University of Portsmouth will run the Transforming the Experience of Students through Assessment (TESTA) project. Initially involving 10 courses, the aim is to expand this pilot project to more courses and improve the quality of student learning through addressing programme-level assessment​ ​across the university.

What is TESTA?

TESTA, originally funded by the Higher Education Academy, is currently sustained by the University of Winchester and Southampton Solent University. TESTA aims to improve the quality of student learning through addressing programme-level assessment. Over 50 UK universities as well as universities in Australia, India and the USA have engaged with TESTA since its early project days (2009-2012). TESTA works with academics, students and managers – and for students, academics and managers – to identify study behaviour, generate assessment patterns to foster deeper learning across whole programmes, and promote assessment for learning.

Why TESTA?

Because:

  • There needs to be more consistency between modules, across programmes, and a greater emphasis on progressively developing students’ internalisation of programme-level standards, over time, rather than relying on documentation to specify criteria at the level of assignments or modules.
  • The programme view shifts perspectives from: figures/percentages  and student experience surveys (e.g. NSS)  to enhancement strategies;‘my’ unit to ‘our course’; teacher-focused on module delivery to student experience of the whole programme; from individualistic modular design to coherent team design.
  • It engenders a team approach. The process enables the researcher to get to know the team and programme;. It’s a listening process, and a valuing process​. The team make decisions based on data, knowledge and guidance.​
  • It enhances curriculum Design and Pedagogy; a.rebalancing formative and summative, b.making connections across modules, and c. ensuring sequencing and progression of assessment across the programme. Also, developing approaches to formative, including more authentic assessment, influencing curriculum design (content-load etc.) etc.

What does TESTA involve?

The process involves mixed research methods for the sake of a. exploring various dimensions of the programme and b. triangulating the data. The process for each course/programme includes: a TESTA audit; an Assessment Experience Questionnaire; and Student focus groups. The process results in a programme case study report with summary of findings, interpretations and recommendations and an interactive workshop presenting this report.

What people say:

The value was to look at what we do from a scientific perspective and look at things objectively, and that is really enabling us to re-think how we do things. Because it’s driven by the project the staff are very willing and accepting of the data. I don’t think anybody, after we had the meeting with you guys, sat there and said ‘They’re talking absolute rubbish.  What do they know?’ (Programme Leader, Nursing). ​

​‘I’ve found it useful to have a mirror held up, to give a real reflection. We talk about the ‘student voice’, but actually this has provided a mechanism, which isn’t part of the programme, which isn’t the evaluation’ (Programme Leader, Education).​

TESTA has revealed some really interesting and, I believe, accurate information about our programme/ approaches/ student experience.  The details of your report have enabled some really strong shifts. We would not have reached these conclusions otherwise and I feel that TESTA has had the desired effect of enabling us to think a little more progressively.’ (Programme Leader, Dance)

‘Our very productive TESTA meeting has stimulated much discussion about how we can develop our modules to include more formative feedback and more engagement in large lectures. Somedevelopments will be incorporated in the interim validation and others will influence our departmental policy on assessment and feedback for next year’ (Programme Leader, Psychology.)

More information about TESTA and a variety of resources can be found at http://www.port.ac.uk/departments/services/dcqe and https://www.testa.ac.uk/

Any interested programme/course leaders can send an email to amy.barlow@port.ac.uk or melita.sidiropoulou@port.ac.uk

Image credits: Photo by Andrew Neel on Unsplash

« Older posts Newer posts »

© 2024 Tel Tales

Theme by Anders NorénUp ↑