/
RSS Feed
Copyright Information:
Nowhere Land – Kevin MacLeod (incompetech.com)
Licensed under Creative Commons: By Attribution 3.0 License
http://creativecommons.org/licenses/by/3.0/
Tag: online assessment (Page 2 of 2)
For its student and other surveys, the University of Portsmouth (UoP) uses a variety of tools and research platforms including Bristol Online Surveys (BOS). In addition to these, more options are being investigated for future use across the university, such as the Qualtrics research platform – currently already used by some departments, e.g., the Department of Psychology. This article will focus on BOS, since it is already used by the university and it gives open access to all UoP staff and students for the time being.
The UoP holds a licence which allows its users to create unlimited numbers of surveys for unlimited respondents. BOS is an online survey tool designed for academic research, education, and public sector organisations. It is an easy-to-use tool for creating online surveys. Run by JISC, BOS is used by over 300 different organisations in the UK and internationally. BOS has the ability for multiple organisations to run the same survey simultaneously and form ‘Benchmarking Groups’ to get answers to common questions or issues (and common surveys). The UoP runs the following national surveys using BOS: the Postgraduate Research Experience Survey (PRES); the Postgraduate Taught Experience Survey (PTES); the JISC Digital Tracker; and the UK Engagement Survey.
BOS has a very comprehensive knowledge base at: www.onlinesurveys.ac.uk/help-support/. A brief summary of the main survey functions as described on the BOS website will be described later (below), with most of the text taken from the aforementioned website. The three main BOS functions are:
- Distributing a survey
- Analysing the survey data
- Creating a new survey
Creating and designing a new survey
There are three ways to create a new survey:
- Create a new survey from scratch.
- Create a new survey by copying an existing survey.
- Create a new survey by importing a survey structure.
- Creating a new survey from scratch
To create your new survey:
- Click + Create new at the top left of the Dashboard.
- Enter a name for your new survey (you can change it later).
- Click Create survey.
This will take you straight to the Survey Builder where you can start adding pages to your survey.
2. Creating a new survey by copying an existing survey
To copy a survey:
- On your Dashboard , find the survey that you want to copy.
- Click on the Copy survey icon. This is found at the far right of the Dashboard.
- Enter a new survey name (you can change it later).
- Click Copy survey.
The new draft survey will appear at the top of your survey list (make sure that you have selected the DRAFT tick box at the top right of the Dashboard ).
To share your survey with another user:
- Check that the person that you want to share the survey with has an active BOS user account.
- You can only share a survey with another BOS user. If the person you want to share the survey with does not have a BOS user account, they will need to request one from the BOS account administrator at their institution.
- On your Dashboard , find the survey that you want to share and click on the View/Edit survey permissions icon (or, from the Design tab of your survey, click on Survey permissions in the left-hand menu).
- Any users who already have access to the survey are listed in the Survey permissions table, alongside their permission settings. To see your own permissions, click on + Show me at the top of the first column.
- Enter a user’s email address in the search box at the top of the table, and click Add user. (Note that the user has to be registered with this email address in BOS.) The user will be added to the table.
- Tick the relevant permission(s) and click Save.
Note: Survey access control settings and survey permissions are not copied along with the survey. You may need to set these up again, if required.
Designing a new survey
To add a new question:
- Decide where you want to place your question. Adding a question into a blue area of the survey builder will add a new, independent question. Adding a question within another question (inside the brown box surrounding an existing question) will create a sub-question. Sub-questions are useful for following up a question to gather additional information and can be set up to be optional or mandatory depending on the respondent’s first answer.
- Click Add item. This will bring up a list of items that you can add to your survey.
- Select the type of question that you want to insert. The question editor will open.
- Type in the question text and format it using the tool bar, if required.
- Add links, images or embedded media to the question text, if required.
- Depending on the question type, you will also be able to add answer options and advanced options below the question text.
- Click Add question.
Your question will appear inside a box on the main survey builder page. Here you can:
- Make changes to it by clicking on the Edit question icon.
- Move, copy or delete it by using the Question actions icon.
- Preview it by clicking on the Preview icon at the top of the page that the question appears on.
An example of a question options follows (multiple line free text question)
Furthermore, there is the option to convert a question into a different question type.
Distributing a survey
The Distribute tab gives you a variety of options regarding: piloting your survey; launching your survey; distributing your survey URL; and the Survey access control.
Piloting your survey
The best way to check a survey before its official launch is to pilot a full version of it. This ‘dry run’ of your survey allows you to test all of its features, including data capture and reporting. It also means reviewers can test your survey without needing access to a BOS account.
Launching your survey
Before you launch a survey, it’s important to make sure that it works properly.
The best way to check these things is to thoroughly pilot your survey. Simple surveys should at least be proofread and tested using the Survey preview.
Certain things cannot be edited once you have launched your survey. Please ensure that you have checked your survey thoroughly before launching it.
The survey preview allows you to see what your survey will look like, navigate through the survey like a respondent and answer questions without any data being saved. The survey preview also offers the option to print your survey or to save it as a PDF. You can access the survey preview at any point while creating or running your survey.
A variety of distribution options is available and the distribution settings offer flexibility.
Distributing your survey
Once your survey has been launched, you must distribute your survey URL to your respondents so that your respondents are aware that the survey is open and know how to access it. You can:
- send out or publish the Public URL through your usual channels;
- distribute personalised URLs using the inbuilt email invitation tool;
- share the survey via social media; and/or
- embed the survey into your website.
Analysing a survey
When it is time to analyse your survey you can to the following:
- Accessing survey responses
- Filtering survey responses / Browsing and excluding individual responses
- Exporting response data
Final Remark
Any member of staff or student can ask to have access to the BOS online survey tool by sending an email to studentsurveys@port.ac.uk. More information about BOS can be found here https://www.onlinesurveys.ac.uk/about/ and help articles are available here https://www.onlinesurveys.ac.uk/help-support/ . BOS is not the only research tool the University of Portsmouth uses, and more research tools are being investigated for future use. Overall, BOS is a useful tool for qualitative and quantitative surveys.
Image credits: Photo by William Iven on Unsplash
The University of Portsmouth uses the Turnitin service to provide facilities for plagiarism detection, online marking and as a development tool for academic writing, although most users are interesting in one thing – a number.
Contained within the Originality Report is a Similarity Score out of 100, which many users wrongly believe to a be plagiarism score with a magic number, at which in can be conclusively determined whether plagiarism has or has not occurred. The problem is, this figure can be manipulated, there will also be mitigating circumstances and lastly let us not forget the system is not perfect either – there will be some margin for error.
Crudely speaking the Similarity Score number is a percentage of the words in your document which matched text from other documents that Turnitin searched against. For shorter assignments with a direct question and consequently a more concise correct answer may well therefore see higher score when compared to a longer assignment with more scope to include to include diverse material.
The number of students in your class and whether the assignment has been set in previous years (or at different institutions) may limit the scope for truly original material, that’s not to say a very high score is necessarily acceptable however it does mean that the latest content may not be unique for genuine reasons. An assignment based upon group work is also a recipe for a higher than usual Similarity Score since students are likely to be working from the same research, data and figures so will in all likelihood draw the same conclusions.
What does Turnitin check an assignment against? There are stored student papers in both a global central repository and the University of Portsmouth own repository (where we might store more sensitive documents). Turnitin also searches against material found on the internet and can check journals, periodicals and publications. Personally I would check against everything, if the service is available, use it.
Turnitin offers several filters which may be toggled, for example whether to include or exclude bibliographic references. Personally I cannot think of a reason why you want to include bibliographic references in the Similarity Score as citing sources is a requirement of good academic writing. That said if the assignment were a lab report and references were not expected then it might be safer to include bibliographic references just in case the Turnitin software incorrectly identified a bibliography and consequently excluded all of the text that followed. You can also toggle quoted material, quotes would not normally be considered within a plagiarism report although the volume of them may indicate a lack of original content from the author. Where quoted material is excluded from the Originality Report, Turnitin helpfully points out when more than 15% of the paper is quoted material. The final filter is for small matches, usually matches of 3-4 words are rather inconsequential, you may also have longer phrases that appear repeatedly throughout the assignment – you can exclude this from being repeatedly matched and skewing the Similarity Score using the ‘exclude small matches’ filter. Personally I use all the filters, excluding bibliographic references, quoted material and small matches – I can always turn them back on later when reviewing a paper if I am suspicious.
So after searching against all of the available material, excluding bibliographic references, quoted material and small matches, what is the magic number? Well, the magic number is… the number at which you become suspicious of course!
Finally, to wrap up this post, and just in case a concerned student has stumbled across this blog post, I would like to emphasise that if they know they have not deliberately plagiarised then they have nothing to worry about. If they are concerned that they have used another source and may not have referenced it properly, then guidance is available from the Academic Skills Unit (https://kb.myport.ac.uk/Article/Index/12/4?id=2747)
Email: academicskills@port.ac.uk
Telephone: +44 (0)23 9284 3462
Or, visit the Academic Skills Unit in person during our opening hours:
Third floor Reception, The Nuffield Centre
St Michael’s Road
Portsmouth
PO1 2ED
Have you ever wanted to incorporate some online activity into your session, but don’t have the facility to do so? Here, in the TEL department we have 30 Chromebooks which are available for morning and afternoon sessions or can be booked for the entire day.
So how do Chromebooks work?
The Chromebooks have two preset profiles that can be assigned through the admin panel. The first is defined as “Classroom mode”, the second is “Exam Mode”.
In Classroom mode the Chromebooks loads a Google login box where the users university details are added. Chrome OS then loads and allows the user to access their work Chrome profile, this will include access to email, drive and any other documents within their Google profile.
Exam mode is much more stringent, and automatically logs into the device and displays the exam landing page. The student would then choose the exam they are expected to take, at which point they are then asked to sign into Moodle with their credentials. They are taken to the title screen of that exam which will display start time, end time (if set) and duration of the exam.
Should there be another requirement for a different Chromebook profile then through discussion with IS it may be possible to create one that would suit the potential need. As an example: Science made a request for exam mode to be enabled with access to a shared Google Drive document that still limited any other web access. This took over a month of testing and development between Science and IS to get the framework working and in place to use. Some requests that have been made however, were not possible and subsequently implementation was not possible.
Unlike standard Chromebooks or laptops, the TEL Chromebooks require a University of Portsmouth Google account as they’re subject to authentication by Google. So if you’re thinking of borrowing the Chromebooks to use with external participants, IS will need at least 72 hours notice to give them time to create dedicated accounts. If you required a large number of external accounts you would need to contact IS directly: servicedesk@port.ac.uk
Booking the Chromebooks
If you would like to borrow the Chromebooks we would require you to complete the TEL Chromebook Booking, Enquiry Form
This form asks:
- How many Chromebooks do you require?
- Which mode do you require?
- Session Date
- Session Start Time
- Session End Time
- Session Name
- Session Location
You’ll need to complete an individual form for each session that you require the Chromebooks for. To make sure that the Chromebooks are in the correct mode we require a minimum of 72 hours from your initial booking to when you require the devices. Chromebooks are transported in wheel-able flight cases (15 Chromebooks per case), therefore it would be your responsibility to get them transferred to where you need them. We’ll make sure that they’re ready at least 30 minutes before your session starts, for you come to collect them.
If you’re interested in borrowing the Chromebooks, but not sure in what capacity and would like further explanation or demonstration then please contact elearn by either telephoning extension 3355 or email us at elearn@port.ac.uk and we can provide some advice on how they have been used before around the University.
Image credits: Photo by Helloquence on Unsplash
Moodle has successfully been upgraded to version 3.3, improving functionality and stability for all users.
New features:
- Integration of H5P – a content-authoring tool that allows you to quickly create interactive resources for your Moodle units (you can use H5P to upload or create: audio recordings; charts; drag-and-drop words or images; flashcards; interactive videos; quizzes; and timelines).
- Stealth activities – activities and resources that are accessed only through other items or links, and are not visible from the course page (previously Orphaned activities). Stealth activities and resources can now be hidden in a standard topic space but made available (under Edit, Make available) to a student / participant. This will hide resources / activities from the course page completely, but still be accessible to students through a link or item within the Moodle unit.
- Drag and drop media – when in editing mode, for quickness, you can now drag and drop media files (images, audio recordings) directly into topic spaces in your Moodle site and select whether to make them either a downloadable file or viewable resource without having to add a label, book or page file to contain them.
- Atto editor – Atto has become the default editor. Staff and students can change their preference to Tiny MCE via Editor Preferences in their profile. However, Atto does provides auto-saving functionality (useful for exams) and formats content in a much more accessible way. Tiny MCE does offer a few different elements of functionality that some staff may find useful, for example – whilst using Tiny MCE in the FireFox browser image resizing can be done by dragging the image corners, rather than having to specifying the require image dimensions.
Moodle assignment:
- The Moodle assignment tool has been dramatically improved to include inline commenting features similar in nature to Turnitin’s Feedback Studio. PDF conversion of student-submitted Word documents is handled automatically by Google Drive, making the new Moodle assignment much more robust than last year’s version.
- When setting up a Moodle assignment it is now also possible to restrict which file formats students must submit their work in. You can do this by adding in the extension name (e.g. .docx) to ‘Accepted file type’ field. However, please note that the system will then not allow uploads of any other file formats, including versions of the same software such as Word (.doc and .docx). To set multiple file formats just use a comma to separate them in the field box. If you wish to allow students to upload any file type then simply leave the ‘Accepted file type’ field blank.
- By turning on the Activity Completion option (the default is off) you can now bulk edit Activity Completion settings. An example would be that if you want all your quizzes to have a new passing grade, you are now able to set this for all activities and not manually change each instance.
Moodle theme:
New Moodle Theme
- The Moodle theme has been improved over the summer, with a full re-design planned for summer 2018/19. Staff can now upload a unit cover image (under Course Administration, Course Summary Files) to quickly and easily illustrate what a unit is about. The upload also triggers the inclusion of a new content filtering and navigation system.
Assessed Videos is a solution developed by the TEL Team to simplify the administration processes of recording a student (or group of students) for assessment. Recordings are shared privately between the assessor and the student just as a written assignment would be. The process is so simple it has been used in class whilst students have given short presentations one after the other with the recording available to the student for review before the end of the session.
Utilising our TechSmith Relay Server (formerly Camtasia Relay) and the TechSmith Fuse mobile app (available on Android, iOS and Windows devices), a video is taken by the mobile device and uploaded to the central server where metadata such as the student’s ID number and details about the recording are stored in a database and used to assign viewing rights. As a lecturer on a really basic level, all you need to do to use this service is start a recording, stop a recording, select the appropriate profile from a dropdown list when uploading the recording and enter the student’s ID number in the description field. After five minutes (longer for high definition video, longer recordings and at peak times) the recording is available for both you and your student to view at http://relay.port.ac.uk/assessed/ where you can both log in using your standard UoP details. All of your videos will be available from one simple navigation page, so no need to remember lots of URLs or save numerous emails.
Whilst working closely with early adopters of this technology/solution, it has become clear that sometimes we can save you even more time by batch processing some of the metadata for you. For example between X and Y dates you might like all of your recordings to have similar titles .e.g ‘U12345 Assessment 1 – student number’. This can be arranged for you so that all you need to do is enter the student number in the description field as described above, rather than completing the title field each time in addition. We can also ensure that all of your recordings are shared with a colleague and vice versa – particularly useful if you team teach. Have an external examiner? No problem, we can create an account for them and share either all or just a selection of your recordings with them.
For each recording, the owner (and any markers) have space to enter a numerical grade out of 100 and also complete a comments box, but that is no reason to limit yourself with the type of feedback you could be providing. Why not film yourself talking to the camera? Simply enter the ID number for the student you are providing feedback to in the description field. Or if you are a little camera shy you could use Relay on your computer to record your screen, perhaps allowing you to add an audio comment alongside a marking grid that you might be completing for the student? If you make a number of recordings throughout the year, you can even set a written reflection exercise with your students who can reference each recording with the direct URL – their recording is still private between you and them as nobody else can view that URL without permissions.
There is both a ‘quickstart’ and a more detailed user-guide available to download from http://relay.port.ac.uk/assessed/ but if you have any questions or would like a demonstration of the system please contact the TEL team at elearn@port.ac.uk for assistance.
MoodleMoot UK & Ireland is held each year; it is the main Moodle conference in the UK which focuses on collaboration and sharing best practice. This year MoodleMoot ran for three days between the 10th and the 12th April, I attended the two days that make up the main conference and missed the “hackfest” on the first day which is typically more developer-led.
The choice of sessions available to attend were diverse and interesting. I was able to visit a number of sessions that have some great ideas, which I can hopefully introduce to the team.
I’ll will try to highlight some of the most relevant or interesting information from the sessions that I attended. However I would recommend searching Twitter for the hashtag #MootIEUK17 as many people were tweeting notes, comments and observations from each session. It really enhanced the first day of the conference for me (until my phone with the Twitter account died, but that’s another story!).
The first session I saw was from the University of London. They reviewed Moodle from both staff and student perspectives, specifically looking at navigation and how to improve the user experience. An example they used was how it’s difficult getting feedback to the student when the gradebook only displays a grade and no indication of feedback. From the academic point of view they discussed how it wasn’t clear if work was marked, or whether the student could or couldn’t see it. They reviewed the new themes being developed at Moodle and the problems they may face rolling them out before they were ready.
Janice Button from Plymouth University implemented the grid format which is a Moodle plugin that hides all the topics and creates a grid view of icons. When they are clicked the relevant section of content is then displayed.
The next session I attended was presented by Lewis Carr. He demonstrated how well developed badges increase engagement, creating more gamified interactions, this was interesting at Portsmouth we have looked at creating badges within Moodle and a few in academics within the University are using them. Properly thought out achievements and goals can also improve how the student progresses through a course. It was pointed out by Janice Button that from entry courses to level 5 (second year courses) the amount of content increased dramatically, this made the progress bar more daunting and the revisiting of quizzes and similar interactions dropped off as the amount of work to be done increased. Lewis also highlighted the need of social interaction and engaging the students before, during and after the course to develop and build the learning experience.
The morning sessions were interesting and informative and gave us plenty to discuss over lunch. After we’d eaten, there were pico presentations (short and snappy when compared the the morning talks) which opened the afternoon sessions and were a great way to continue. Two sessions stood out: our own Mike Wilson spoke about Assessment Enhancement and the customisations that have been developed for our Moodle installation, and Rebecca Barrington put the “oo” back into Moodle by looking at how to keep academics on-board with the development of sites, explaining how embedding resources within Moodle is a great way to enhance and develop what you already have.
My turn to present was nearing, now, I am not normally a nervous presenter but the thought of presenting on a stage with the glare of lights and a massive screen behind me contributed to the nerves this time around! Whilst I got through it with positive feedback and managed to handle a few questions reasonably well, I felt I could have been better and should I be lucky enough to do another talk will endeavour to get those nerves in check and slow down!
Looking back, there certainly wasn’t a need to be nervous, the whole day was incredibly relaxed and just a creative hub for people who are keen educators, all looking at how we can approach our teaching, training and Moodle usage in a interesting way.
A panel discussion on getting rid of the desktop and turning to only a mobile solution ended the day. This was a very open discussion with questions from the audience and a range of views. A point that resonated with me, was the idea that we should always look at what we do for those end users on a mobile as the number of active mobile users is rising. The other side of this argument however, is that development from a mobile is not really practical. Whatever we develop should allow us to be flexible though, ensuring that our end users having the easiest and most simple experience of Moodle. The new themes and tools that are being added to the newer versions of Moodle show that this is also a concern for the ‘core’ Moodle team.
Day two of the conference saw my phone die, which meant that tweeting from the @TelPortsmouth account was curbed! The sessions I would like to draw on from the second day happened just before lunch as the second day was more sedate due to the evening festivities of the reception dinner the night before.
Mary Cooch, who is known amongst the Moodle community as the ‘Moodle Fairy’, did two sessions and I attended one entitled “10 user experience improvements since Moodle 3.1” of which I had heard of most updates. It was useful to see but essentially if you go through a versions release notes they will be highlighted there (I am sure many people don’t!)
The second session and my pick of the second day was “Competency based education a look at the features in Moodle 3.1 / 3.2”. This session was interesting, informative and above all confusing! Competencies are used to “describe the level of understanding or proficiency of a learner in certain subject-related skills”. Competencies are a new and growing feature of Moodle. TEL are researching Competencies so that members of staff in the University can possibly make use of them in the future.
MoodleMoot is a fantastic event that I would suggest to anyone involved with any aspect of Moodle. It was engaging, with a great range of subjects and everything from back end/admin level sessions to those designed to be understandable for the average user. It was also a great place to ask anyone a question, everyone was approachable and for the reception dinner, a great place to hijack the Moodle stage display and use it for photos!
#Moodle!! #MootIEUK17 😉 pic.twitter.com/knQlF9NUGW
— Paul Vaughan (@MoodleVaughany) April 11, 2017
Image credits: https://moodlemoot.org/mootieuk17/
When it comes to online exams there are a number of questions that cause headaches for support staff and academics. Where am I going to find the time to create all the questions? How do I make sense of all these settings in a Moodle Quiz? How can I keep an eye on so many students during the exam itself?
The simple answer to all these questions is normally to speak to the right people. The first port of call, if you’re interested in getting started with online assessment, it’s your friendly Faculty Online Course Developer (or the central eLearn team), who will be happy to advise or point you in the right direction.
Moodle is of course not the only tool for conducting online exams, but it is very good at handling large groups of students who are attempting many questions all at the same time. These questions generally have a right or wrong answer, most of which can be automatically marked. Essay questions can also be posed, but these will require manual grading. (Many students these days have difficulty in writing by hand for three hours, so if your exam is heavily essay-based you might want to investigate a tool such as DigiExam, which allows students to type their answers (contact the eLearn team for more information about DigiExam).
A tremendous amount of question-writing effort has already been made at UoP by staff across faculties. There are close to a million questions already in Moodle, most created directly by staff but with a significant percentage having been imported from existing Word documents, shared by colleagues in other departments or institutions, purchased from commercial suppliers or imported from older systems. You don’t always have to start from scratch, as many academics already have treasure troves of questions that can be adapted or imported.
Once you have the questions you wish to pose, your next step will be setting up the quiz that will be used to deliver the questions. This annotated pdf of typical Moodle exam settings walks you through the various quiz settings (many of which are set to the optimum setting by default). Your Faculty Online Course Developer will be able to help out here, and also assist with the important job of testing the quiz or exam.
By this point you’ll have a working, thoroughly tested Moodle quiz that you could use for a summative assessment. As a member of staff you’ll have gone through a process of familiarisation. It’s important that you allow your students the same familiarisation with the online exam process (what to expect on exam day, how the software works and so on), not to mention any administrative staff and moderators who will be involved. It’s advisable to schedule some mock exam sessions well in advance of your first exam so your students are fully prepared when it comes to the real thing. Although it’s by no means compulsory, Safe Exam Browser (SEB) can be leveraged here. SEB is a web browser, available on all student PCs, which locks students down to a single Moodle quiz and prevents them from accessing other web sites or resources. SEB will help you keep an eye on large groups of students and be certain they are concentrating on the task at hand. Take a look at this Safe Exam Browser FAQs if it’s something you might be interested in. DCQE also have a set of 30 Chromebooks which can be locked down into exam mode potentially turning any wifi enabled room into an exam room. More information along with the Chromebook booking form can be found here.
Hopefully this blog post has sparked your enthusiasm for giving online exams a go. The keys to success are (i) getting in touch with your faculty online course developer who can help you at various points along the way, and (ii) starting with non-critical familiarisation exercises which give room for finding the edges of online assessment. It’s fair to say that you will have to dedicate a bit of time to start with creating quiz questions, but the downstream benefits of online assessment can be significant.
Some useful resources
eAssessment at the University of Portsmouth
Quiz support materials for staff
Quiz questions examples and templates
Image credits: https://pixabay.com/p-1828268/?no_redirect
