Numbas for summative assessment - checklists

[Cat Bazela]

Hi all, 

We were hoping that those of you who use Numbas for summative assessment might be able to share some of your guidance with us.  We’re starting to look at the procedures (and checklists) that may be needed as we look to implement Numbas for summative assessment at Sheffield using the LTI link into the VLE. So far we know that if Numbas is used, it will be using a lockdown browser (to negate students using the console view (F12) to look at the code behind the test, and be on campus, invigilated in a computer room.

If you are using Numbas for summative assessment would you be able to share with us what checks/checklists are in place for staff who set up the exams in the VLE  and what checklists are given invigilators to follow about setting up/running the exam?

Thanks

Cat

[Ben Brawn]

Has anyone ever been able to use F12 to work out anything that would help cheat? Wouldn't a locked-down browser be more about stopping students using wolfram alpha?

[Chris Graham]

Hi Cat,

I can share a few things from Newcastle. We are about to restart in person summative exams after a gap and some changes to our local lockdown set up, so might be able to add more later as we go through the process ourselves.

I think the first thing is the settings in the Numbas exam and LTI tool. We would use the following (obviously you may have slightly different settings):

In the Numbas Editor

• make sure "allow user to regenerate questions" is off
• make sure "Reveal answers to all questions" is set to "When entering in review mode"
• select Timing, make sure exam duration is 0 if there is to be no timer. If there is to be a timer, make sure the time is set, and untick "Allow pausing"
• select "Feedback", untick "Show current score", "Show answer state" and "Allow reveal answer"
• if solutions or expected answers are to remain hidden, even in a review mode if it is going to be available, untick them under the "Review mode" setting 
• we would normally set a password, provided in the room at the start of exam. Go for characters that are not easy to confuse (less OLI letters, more WXY)
  

As a more general point, we would always recommend that questions are checked / marked Ready to Use by someone other than the author (we try to follow our paper exam procedures as much as possible).

In the VLE/LTI tool

• Set dates in the VLE as appropriate. We use a start date only, to allow access in review mode for in course assessment, but for an exam we'd set an end date too.
• Also set from/until dates in the LTI tool. The end date is required even if you have an end date in the VLE, since the VLE end date won't kick a student out of a test at the deadline.
• Set "When to show scores to students" to "Never" (or students might see their current score when reloading a page)
• Set "When to report scores back" to "Manually, by instructor" (or students might see their score in the VLE
• The latest version of the LTI tool has an "Access changes" feature to add extra time for students as appropriate

In the exam

• This will depend on your local set up, but yes we'd definitely recommend a lockdown browser.
• We normally provide invigilators with 
  • contact details of someone they can talk to if any Numbas-related issues arise. As a minimum, someone with access to the LTI dashboard for the exam.
  • any details on starting the exam that differ from our regular digital exams (including a screenshot of the password screen)
  • screenshots of the connection warning and that it's only a problem if it persists / is still there at the end and what to do (we've previously told the invigilators to hold students back and seek support).

We also try to spread the word to students to submit as they go along and that they can re-submit as many times as they like before ending the exam.

At Newcastle, no matter how prominent Christian makes the End exam button, we still get students pressing it by accident (though don't recall that happening in an exam). Someone with access to the LTI dashboard would need to re-open the attempt if that were to happen.

I hope that helps,

Chris

[Catherine Bazela]

Thanks Chris, 

That's really helpful to know, especially the 'in the exam' scenario. 

Thanks again

Cat

--
You received this message because you are subscribed to a topic in the Google Groups "Numbas Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/numbas-users/ehd1UU-PqzM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to numbas-users...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/numbas-users/db26d4c9-c808-4dd4-8f88-fd25cbd3bc97n%40googlegroups.com.

--

Catherine Bazela (She/her) BA(Hons) MSc(Econ) FHEA

Digital Learning Adviser

Digital Learning Team

Academic Programmes & Student Engagement

University of Sheffield

New Spring House

231 Glossop Road

Sheffield

S10 2GW

Tel: 0114 222 0411 (NOTE PREFERRED CONTACT IS VIA EMAIL)

Working hours Monday - Friday 8.30am - 4.00pm

************************************

I am currently splitting my working hours between on campus work and remote working. 

My preferred method of contact is email, Google Chat and Google Meet. 

I don't expect you to respond to my email outside your working hours. 

-----------

Visit the Elevate website or sign up for our newsletter.