Some answers get deleted or changed when quiz is paused (Moodle)

[billy...@gmail.com]

Hi all,

An example question: https://numbas.mathcentre.ac.uk/question/183213/a3-q9-two-way-tables/ (SCORM generated on ~5th November)

= question 8 in the following assignment: https://numbas.mathcentre.ac.uk/exam/41792/assignment-3-probability/

Some students have reported that they completed part of this assignment, got this question correct, then closed the window or pressed pause, and when they came back later their answers have disappeared or have changed. Moodle logs give some evidence that this is happening - here is a chunk of the logs for one student:

• 18 November 2025, 7:32:02 PM SCORM package: Assignment 3: probability (assessed) SCORM package Submitted SCORM raw score The user with the id '244316' submitted the element 'cmi.score.raw' with the value of '56' for the attempt with the id '80969' for a scorm activity with the course module id '1402222'.
• 19 November 2025, 7:18:38 AM SCORM package: Assignment 3: probability (assessed) SCORM package Sco launched The user with id '244316' launched the sco with id '8684' for the scorm with course module id '1402222'.
• 19 November 2025, 7:20:03 AM SCORM package: Assignment 3: probability (assessed) SCORM package Sco launched The user with id '244316' launched the sco with id '8684' for the scorm with course module id '1402222'.
• 19 November 2025, 7:20:20 AM SCORM package: Assignment 3: probability (assessed) SCORM package Submitted SCORM raw score The user with the id '244316' submitted the element 'cmi.score.raw' with the value of '45' for the attempt with the id '80969' for a scorm activity with the course module id '1402222'.

You can see here that the student's score dropped by 11 overnight. Have I done something wrong, or is this a bug in the export process?

Thanks, and best wishes,

Billy

[Christian Lawson-Perfect]

Hmm, I'm not sure about that. Moodle's built-in SCORM player isn't very sophisticated when it comes to storing the data, so maybe not all of the attempt data had been saved when the students closed the window?

--
You received this message because you are subscribed to the Google Groups "Numbas Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to numbas-users...@googlegroups.com.
To view this discussion, visit https://groups.google.com/d/msgid/numbas-users/89fd7436-daa3-46c7-80d0-b2b5d3bd0650n%40googlegroups.com.

[billy...@gmail.com]

I'm sure it is partly a Moodle issue - Moodle's SCORM player is terrible - but this isn't happening on every quiz, or even every question on this quiz, whereas it's happening repeatedly with this question. That suggests either that my question is broken or there was a bug in the SCORM / export process.

Just out of curiosity, I downloaded a fresh SCORM package of the same quiz and ran a diff. There are very few differences, but this one jumped out at me - broken quiz on left, new exported quiz on right. Could this be the issue? (It doesn't necessarily explain why this is only affecting some questions, unless there's something subtle about how I've set up those questions that isn't coming to mind - I can send a link to an unaffected question if you like.)

[Christian Lawson-Perfect]

Ah, I see - it's just this package that you're having trouble with.

Are you saying that the newer version of the package you downloaded doesn't have this problem? The line you spotted is just a recent change to the code, so it's not a sign that your old package is broken in any way.

I'd normally suspect an erratic marking algorithm for cases like this, but your question only has number entry parts so the marking should be deterministic.

Can you email your original SCORM package to me at christia...@ncl.ac.uk, and I'll try it in my Moodle dev environment?

To view this discussion, visit https://groups.google.com/d/msgid/numbas-users/73b308a7-3db5-464e-8961-989bd6ad0affn%40googlegroups.com.

[billy...@gmail.com]

"Are you saying that the newer version of the package you downloaded doesn't have this problem?"

Oops - no - I had thought so, but I've now tested more and I was wrong. Sorry!

I've now tested more thoroughly, and here's a minimal pair of broken/unbroken questions, with my thoughts on what's broken:

1. Please download the SCORM package for this question and upload it to Moodle: https://numbas.mathcentre.ac.uk/question/183206/a3-q3-probability-formulas/
   If I complete the quiz in one sitting, or use the inbuilt Pause and Resume buttons, no problems occur. But if I leave the Moodle page and return, or use the refresh button in my browser - something that causes Moodle to relaunch the package - then the values of probsA and probsB are the same, but the values of probscap (and the dependent variable probscup) change. This causes formerly correct answers in parts (a) and (b) to be marked incorrect if the values have changed (though they are still marked correct if, coincidentally, the randomisation picked the same numbers both times).
   
   
2. Next, try this one: https://numbas.mathcentre.ac.uk/question/184981/testing-billy-s-copy-of-a3-q3-probability-formulas/ This one does not have the same problem. I can relaunch the quiz as many times as I like and it's fine.
   

The difference between the questions is that, in question 1, I have

  probscap = map(i -> random(10 .. floor(2*min(probsA[i], probsB[i])/3)), 0..3)

and in question 2, I've separated this out into

  probscap = [random(10 .. floor(2*min(probsA[0], probsB[0])/3)),
    random(10 .. floor(2*min(probsA[1], probsB[1])/3)),
    random(10 .. floor(2*min(probsA[2], probsB[2])/3)),
    random(10 .. floor(2*min(probsA[3], probsB[3])/3))]

(The other broken question in my quiz also has a random inside a map; the unbroken questions don't.)

The SCORM suspend data gives another hint in this direction. In question 1:

  "variables":{"probsa":"[ 36, 47, 41, 49 ]","probsb":"[ 38, 44, 39, 46 ]"}

In question 2:

  "variables":{"probsa":"[ 54, 54, 30, 34 ],","probsb":"[ 42, 33, 43, 52 ]","probscap":"[ 25, 22, 19, 12 ]"}

Question 1 hasn't realised that it has to store probscap! Since there's no value stored, it can't restore it when the SCORM is relaunched, so I guess it must just be generating them from scratch each time.

My guess is it must be somewhere in the Numbas.jme.isDeterministic function, but at a glance I don't fully understand how it works, so I'll stop digging here...