Implementing online exams: notes from Norway

This year, I’ve been working with my colleagues in the CLT to look into Online Assessment, particularly online exams. In March we decided to run a University trial of a solution previously tested by the School of Management. This developed into a business case and we were granted the funding for a pilot in July. At the same time, we were invited to a seminar in Norway in September and were fortunate to succeed in our application to the TDF Travel Fund to attend.

The event was run by Inspera, the vendor responsible for the solution we are currently trialling in small capacity around the institution in 2018/19, and held at Oslo Science Park (pictured above). Scandinavia has been leading the way in this area so we were keen to see how they and UK HEIs had implemented online exams and worked with any challenges, as well as future product developments.

The experience of UK HEIs

The University of Oxford’s Medical Sciences Division introduced Computer Based Assessment (CBA) in 2003 to improve marking objectivity, the availability of questions and assessment times. Analytics were an additional bonus. In 2018, they used Inspera to run 70 summative university exams, including 100 formative assessments and 20,000 student coursework submissions, with an average of 100 questions per assessment. Their assessments were designed with many answer options to ensure students understood why they were selecting that particular answer. The Medical Sciences team wanted to ensure they were testing depth of understanding rather than knowledge recall. The key benefits they observed were that academic staff preferred Inspera to QMP; that downloading student marks in a spreadsheet has made a huge difference to processing; and that OSCE assessment would be useful.

BPP University’s Ishan Kolhatkar gave perhaps the most entertaining presentation. BPP ran their Inspera trial with the additional challenge of having campuses spread over a reasonably large geographical range. It proved to be successful and both staff and students reported back that they much preferred using an online solution. BPP are now progressing with a larger scale roll-out.

Integrations and work flow

Whilst integrations are a hot topic, Inspera acknowledges that development is expensive and time consuming. Thus, the challenge is to reduce the numbers of systems to access and have everything in one place; but in reality this is quite complex to do (as we know). Integrations are critical when it comes to this, and perhaps so is having a portal. Standardising work processes is also key but isn’t overly popular. It’s also acknowledged that integrations require maintenance and review. It isn’t a case of implementing and leaving well alone.

Technical developments

During the course of the seminar, we explored various aspects of online exams, from theoretical underpinnings through to technical developments, including:

Standards Development - the importing/exporting of questions between platforms is currently limited in terms of the standards used. Web Compliance Accessibility Guidelines (WCAG) align improvements and identify the User Interface (UI) as a top priority for immediate improvement. APIs (Application Programming Interfaces) were also discussed in terms of the five key themes: authoring, delivering and monitoring, grading and marking, reporting, and user admin.
General Data Protection Regulations (GDPR) - Inspera will be launching new tools to increase self-service for data export control. There will be an API to delete data, get user history, and manage account level configuration of personal data: a useful addition, although it will take time to implement.

…. and to finish... we had a Scantron demonstration

The Scantron scanner is a standard scanning device that holds 100 pages in one batch load, although Inspera are working with Scantron to offer a larger scanner as well. The scanner requires a special type of paper with a boundary, much like a typical exam paper, and also scans in colour.

The student code on the paper links the sketches with the associated student and exam within the system. The grader can then view the submissions with the sketches. Of course, there are instances where the number has been omitted or is incorrect, but it is possible to review these and assign to the correct student. The student can also see the total submission (exam paper and sketch) where permitted

To conclude, reports from institutions that have trialled Inspera note great feedback from academic staff and students. Any reservations tend to relate to workflow and the time needed to adapt to a new process.

I really liked that Inspera rely on their customer base to feed-in improvements and guide the road map, ensuring the solution continues to remain aligned to the business needs of those who use it.

Finally, the metro in Oslo is a lot quieter than the London tube but old habits die hard and I couldn’t help but run up the left and rush around to beat the err, crowds.