Recording student work in an LMS (without runestone)

[Oscar Levin]

For a bit now we have offered a SCORM output format that bundles an html build with a manifest file that instructors can upload to their LMS.  This is useful for instructors that want a document that only their students can see (using Canvas as the login manager, essentially).

But guess what: SCORM can also be used to proctor quizzes.  Interactive components on a html page can emit events that some javascript can send to an API that the LMS listens to and adds the results to the grade book.  

With the help of Claude, I've hacked together a very simple pretext generated scorm file with the additions needed to serve a multiple choice question and a webwork question.  If you answer both correctly, Canvas says I have 100%.  Only one of the questions correctly: 50%.  It's working!!!!

So besides being so excited about this that I had to tell you all right away, I do have some questions:

1. The current hack is really a hack.  The extra javascript that communicates with Canvas is listening for a console.log from some runestone javascrip that reports the status of the multiple choice question.  I don't really know what it is looking for for webwork.  What I think we need are some dispatchEvent that reports the id, response, and correct/incorrect status for each assessalbe thing.
2. A listener hook that can restore student solutions when they return to the assessment. To what extent is this already present in the Runestone javascript?
3. I'm not really sure what state the runestone javascript is in (nor what other assessment types are, relative to it.  Is this a good opportunity to clean some of that up and make it more consistent?

Happy to discuss this live next week as well.

[Jason Siefken]

This sounds neat! It also sounds like a student could open their browser console and award themselves points?

--
You received this message because you are subscribed to the Google Groups "PreTeXt development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pretext-dev...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/8e0ad866-dfa7-40b8-9a57-06d69a671756n%40googlegroups.com.

[Andrew Scholer]

Oscar - 

Brad made a push a while ago to unify the pathways for getting a score from the different runestone components and loading previous responses:

It is plausible that some hooks could be added to allow those pathways to call out to some wrapper that directs them to the SCORE api of the LMS instead of the RS server.

I'd recommend checking out:

https://github.com/RunestoneInteractive/rs/blob/main/bases/rsptx/interactives/runestone/common/js/runestonebase.js

There are some notes here:
https://github.com/RunestoneInteractive/rs/blob/main/bases/rsptx/interactives/runestone/overview.rst

Semi-related... Brad has also done recent work on the RS server side to allow other interactives to use the RS server to store state and record grades that can be part of an assignment. (i.e. for them to use RS the way you want to use the LMS). That code is based around the SPLICE protocol (https://cssplice.org/slcp/index.html). But, the RS interactives themselves don't use SPLICE to talk to the server.

To Jason's point, yes, for a (highly) knowledgeable student, the JS console can get you 100% on any Runestone interactive. The interactives are not designed to be secure enough for high stakes assessment.

Andrew

To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/1cea498f-74b7-4775-8115-d2cb9add3fb1%40gmail.com.

[Sean Fitzpatrick]

A lot of people at our university do "Moodle exams": set up as a quiz in the LMS, but configured so that it's only accessible from our Testing Centre: a big sad room full of computers where students can write their quiz on a lockdown browser while watched by proctors.

It's especially popular with people who think a test with 200 multiple choice questions is peak pedagogy. 

But I digress... This could solve problems for people who have wanted to do quizzes with WeBWorK and similar. Until now we've had the problem that the browsers don't allow navigation to URLs outside the LMS.

To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/CACm44N_u%3DPX7K9LjJ1D1vmX%3DsKga-YkxAJa%3D%3DD1L6LK9NXb5%2Bg%40mail.gmail.com.

[Oscar Levin]

This is exactly what I was hoping for Andrew.  I'm thinking that the scorm API javascript file can basically be a mock Runestone, communicating with the lms instead of Runestone.  Should make for a fun weekend!

To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/CACm44N_u%3DPX7K9LjJ1D1vmX%3DsKga-YkxAJa%3D%3DD1L6LK9NXb5%2Bg%40mail.gmail.com.

[Rob Beezer]

On 5/15/26 16:02, Oscar Levin wrote:
>   Should make for a fun weekend!

Just you and Claude. Watch your usage limits!

Seriously, sounds intriguing. Tell us all right away about progress.

Rob

[Rob Beezer]

On 5/15/26 16:02, Oscar Levin wrote:

>   Should make for a fun weekend!

[Chrissy Safranski]

This is super exciting!

Chrissy

[Oscar Levin]

I now have something that seems to be working pretty well and would love some feedback and testing.  Here is what you can try:'

1. Get the most recent version of the CLI (2.40.1)

2. Create an article and add some of your favorite types of exercises.  WeBWorK seems to be working, as do the standard runestone exercise types.  In my testing I only used single page documents, so either don't add sections or setting chunking to 0.

3. In project.ptx, add `compression="scorm"` as an attribute to your html target (or create a new target).

4. After building the target, go to your LMS and upload the scorm archive.  In Canvas, this is done by going to the SCORM section (which might need to be enabled or moved to your navigation).  Add the scorm archive as an assignment.

5. View as a student and try the assignment.  You should get scores in the grade book.  You should be able to navigate away and back and have answers restored, or continue answering to improve your score.

I'm especially interested in anything you expect to work that doesn't, and anyone who has a non Canvas LMS that can try this out.

[Oscar Levin]

I think there is a small bug with file naming (underscores and hyphens are different, apparently).  So maybe hold off a day for testing.  I'm working on a PR.

[Oscar Levin]

Released a new CLI version today that should have scorm actually working to record student work, including for Doenet activities.  Nothing has changed about how you should test this out from my previous post.  Excited to hear if it works for you.

[Oscar Levin]

If you would like to just try out the scorm assignment in your LMS, you can download this zip file: https://drive.google.com/file/d/1_qTazEspMW0lZePh6SuAtJGZbzdybDWE/view?usp=drive_link

[David Austin]

OK, this is really amazing though it's not quite working with (dumb) Blackboard.  I uploaded the scorm assignment into Blackboard, and completed the assignment.  I got 100% when I used the Doenet "Check Me" buttons, but there wasn't anyway to save my work.  When I went to the Blackboard grading section, the assignment showed up but it told me that I (the student) had saved a draft, but that the grade wasn't recorded.  It just said "Draft saved" but I don't see any way to signal the LMS that I have completed the assignment and want my (perfect) grade recorded.

David

--

You received this message because you are subscribed to the Google Groups "PreTeXt development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pretext-dev...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/8a4e62f4-0a7a-4d32-9bf0-6641cac6f3e3n%40googlegroups.com.

[Oscar Levin]

This is with the zip file I linked?  

And if you go back to the assignment, as the student, do you see your answers?

Apparently there are different ways for the lms to communicate work, and Canvas can't restore work unless it is still in progress.  I could have the js add a submit button.  Curious what that would do.

Thanks for trying this out.

To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/CANXmVMC%2BAzgZ0NzWjSvE7qFH--xdZYEU9%3DbML2e192JHjTMiPQ%40mail.gmail.com.

[David Austin]

Yes, I just took the zip file that you sent and loaded it into Blackboard.  All my answers were there when I returned later.  The assignment isn't due until midnight so maybe that will trigger the results being loaded into the gradebook.  I'll report tomorrow!

To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/CAOU9BaVOygKdv7rpdex3_jyYtTPaY7EgBPBsW-LWJXE-t2O%2B2A%40mail.gmail.com.

[David Austin]

Now that the due date has passed, I still don't have a grade in the Blackboard gradebook.  The status of my work as a student says "Draft saved", but my grading status says "Nothing to grade."  When I view my work as a student, it says that my "Maximum points" is 100 points and gives a time when my draft was saved.  I did not do any explicit action to save my work so that may have been when I exited the assignment.

[Sean Fitzpatrick]

I tried it in Moodle and it seemed to work fine.

It's hard to say for sure how the grading is working because I don't have the ability to create a dummy student account for testing, but if I act as a student and do the assignment, I get a grade.

The only thing I could see being an issue is attempts management: someone might not want students to have more than one attempt at the multiple choice. I can restrict attempts to 1 attempt, but that applies to the SCORM assignment as a whole: once I exit, I can't go back in. But while I'm in, I can try a problem as many times as I want.

To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/CANXmVMC59YP3vYt6qeWELAgjpdwv2RhQzL_Ob6JgeE6k%3DbQ8vQ%40mail.gmail.com.

[Oscar Levin]

Hi David,

I think I know what the issue is, and perhaps this version fixes the problem: https://drive.google.com/file/d/1_qTazEspMW0lZePh6SuAtJGZbzdybDWE/view?usp=sharing (I think it's the same link but the archive is new).

Sean,

Yes, just like runestone, there is no attempt management available for these.  I don't see a way to require that (I think in Doenet you can limit attempts, but certainly not for runestone components).

To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/6ea17253-f97c-4786-af04-6d786789d426%40gmail.com.

[Chrissy Safranski]

The author of the Doenet activity could limit to 1 attempt within the Doenet code, so that can happen when the instructor is also the author of the activity. 

Chrissy

You received this message because you are subscribed to a topic in the Google Groups "PreTeXt development" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/pretext-dev/8zqT8EJXMfk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to pretext-dev...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/6ea17253-f97c-4786-af04-6d786789d426%40gmail.com.

[David Austin]

Hi Oscar,

I tried loading the new scorm file and saw the same behavior.  I can do the homework assignment and my work is saved, but it doesn't transfer over to the gradebook.

David

To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/CAOU9BaU__BAjBC8-7ebwpPZrp4%2BWhuDa7s5ezqYRfzcXSpqi%2Bw%40mail.gmail.com.

[Georg Osang]

Hi Oscar,

I tried your example package on our Moodle server and it seems to be working well!

I tried to build my own package and realized I manually have to add a manifest file, after that it worked fine too. Is this intended or am I doing something wrong?

Out of curiosity, I added a STACK question and unsurprisingly no score was collected from it. Do the supported question types leverage some already existing code from the Runestone integration? I didn't see any question type specific code in your PR, so I was wondering how that works and what it'd entail to support other questions types like STACK.

Georg

[Oscar Levin]

David,

Thanks for your continued testing.  I've tried something else if you would be willing to try again.  Same link, but this time you will see "Attempt 3" before the first exercise (just to make sure we are testing with the most recent version).  If it doesn't work, then perhaps you can set the scorm assignment to load in a new page, enter `localStorage.setItem('ptxscormdebug', '1')` in the dev console, and reload.  There will then be a number of [PTX-SCORM DEBUG] messages that might give us some clues.

Georg,

Using the most recent version of the CLI (2.41.0) you can create a scorm output by adding `compression="scorm"` to the html target in project.ptx, and this will add the manifest automatically, as well as load the scorm js.  But you are right, I don't see any reason that a STACK question should work.  I entirely hijacked the runestone data for capturing and restoring student work.  But it should be reasonably easy to add support for STACK.  We just need to get the status and content of the exercises.  And have some way to identify exercises so we can add it to the total.

To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/682e0209-adaf-4dbf-96c2-9c5375a48070n%40googlegroups.com.

[David Austin]

I see the new assignment with "Attempt 3" but the same behavior persists:  work is saved as a draft but not graded.  We do have some error messages however:

[PTX-SCORM] No SCORM API found. Exercises will work normally but scores will not be reported. ptx_scorm_events.js:103:17

Error fetching SPLICE state: SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data spliceWrapper.ts:146:21

[PTX-SCORM] recordInteraction: aborting — session not initialized. _api: false _ver: null ptx_scorm_events.js:678:21

To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/CAOU9BaUZZgt%2Bqq33bM3N%3DjP8R4BDjWCeeB8DO-if1fYCWQTqQg%40mail.gmail.com.

[Oscar Levin]

Hi David,

That is very suspicious.  I think the next step will be for us to find a time to sync up in zoom to see if there is some setting in Blackboard that can be tweaked.  I don't think Blackboard uses a different scorm API, but maybe I'm wrong?  

I'll be away this next week, so maybe we can bump into each other during a dropin the following week.

[David Austin]

That sounds like a good plan, Oscar.  I'll see you online soon.

David

To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/fdfc3529-cd42-47d4-99f3-4a303bcb0b47n%40googlegroups.com.

[Oscar Levin]

Hi David,

If you would like to try again, I made one tweak that might help us debug.  Here is the new file: https://drive.google.com/file/d/1_qTazEspMW0lZePh6SuAtJGZbzdybDWE/view?usp=drive_link

Also, just to make sure, are you doing something like the following to assign the scorm file?

Open your course and navigate to the Course Content page.

Click the plus (+) icon wherever you want the content to live.

Select Create, then choose SCORM package.

Click Upload SCORM package to select the .zip file from your computer, or Browse Content Collection if it's already saved.

Once uploaded, adjust your settings in the SCORM Settings panel:

  Details & Information: Add a Title and a Due date.

  Grading & Submissions: Choose how to grade it (e.g., total SCORM score or simple completion) and define maximum points.

Click Save, and make sure to change the visibility settings to Visible to students.

To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/CANXmVMAEZsL1mjEDB6KCJP95iWHkh1AB2%2BT6ZdJKVBVfS7%2BeDw%40mail.gmail.com.

[David Austin]

Hi Oscar,

Yes, that's the process I'm following.  The only difference is that I can't choose the grading schema since I get the message "You can't change the grading schema after importing the SCORM".  It says it's graded out of 100 points though.

I just tried and had the same behavior as earlier.  There doesn't seem to be a way to submit the assignment, and it just says Draft Saved when I go to the gradebook.  I'll try to stop by the dropin this week.

David

To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/CAOU9BaWYecGxTFMpfe1mL874fS%3DPpF5K4OocTr3Uo%2BAd%2BiFhKA%40mail.gmail.com.

[Oscar Levin]

Hi David,

I have a new version of the scorm.zip file at the same link for you to try.  This has a submit button, which I really hope will successfully submit the student's work to the gradebook.  If you can try both navigating away without clicking it, and clicking it, that would be great.

Oscar.

[David Austin]

OK, progress but we're still not there.  The Submit button appears and work is saved whether I use the button or not.  When I click the button, it gives a message:  Assignment submitted. You may now close this window.  If I navigate away and come back, it tells me that "Warning! You may have this course open in another window."  I choose Open anyway, and the work is still there.  However, it's not showing up in the gradebook and only says "Draft saved."  

To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/b4ba3d8a-44d1-42fd-95b0-db44dd9bbbe4n%40googlegroups.com.

[Oscar Levin]

Good good.  Claude had some ideas what might be happening (that navigating away was setting the exit state to "suspended" even after it was submitted).  A new version is ready for testing.

[David Austin]

OK, I received a grade!  But ...

If I complete some of the assignment, and navigate away from the page without using the submit button, the assignment is submitted and graded.  When I go back to the assignment, I just see a breakdown of the grade by problem number and I can't see my responses or edit.  But I've seen different behaviors too, where I can navigate away, come back, submit and get a grade.  The behavior is not consistent.

If I start the assignment and then use the submit button, I see the old behavior:  "Draft saved."  When I try to navigate back, I get an error if a short period of time has passed.  If more time goes by, I can get back to the work, modify, and submit it.  But I don't get a grade.

To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/71a14acc-5c92-442b-a06e-ff31233bb68an%40googlegroups.com.

[Oscar Levin]

Glad we are making progress.  I new attempt is ready for testing.

[David Austin]

This was a step backwards as nothing, either submitting or just leaving the page, would get things into the gradebook.  The version before this one recorded the grade in the gradebook, but not when I used the submit button.

To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/9a9f83c6-0369-4193-9ee2-9088dc190aecn%40googlegroups.com.

[Oscar Levin]

Very well.  I have a new version for you to test.  The reasoning given by claude for this fix seems very good, but just in case, I have added a bunch of combinations of exit/completion status that can be triggered below the submit button.  If you would:

• Try using the regular submit button, and then in another tab, as the instructor, see if grades have been recorded.
  • If that works, Great!  Maybe test what happens if the student just navigates away before submitting too.  Unless you are curious, you can stop here.
  • If not, reset the student's status (in Canvas we can "Reset student" as part of the student view) and proceed to the next steps
• Repeat the above using the five testing buttons to see if any of those work.

Happy to sync up some time today or tomorrow if you want to go through this together.  I also have been reading the blackboard scorm docs and there are LOTS of advanced options available inside blackboard for the assignment that might make this work, but of course it would be best if it worked out of the box.

[David Austin]

I've been trying this again using all the buttons at the bottom, but without any success.  It would probably be easier to do this together so let's try to meet up at a dropin.  Unfortunately, that probably won't be until the week after next.

To view this discussion visit https://groups.google.com/d/msgid/pretext-dev/CAOU9BaWYecGxTFMpfe1mL874fS%3DPpF5K4OocTr3Uo%2BAd%2BiFhKA%40mail.gmail.com.