Sakai LTI in July - Fixing little things in LTI Integrations

[Charles Severance]

Hi all,

I am mostly finished with my major feature in Sakai-21 (LTI based assignments) and so I am ready to start having conversations with LTI Tool vendors about rough areas in their integrations. 

This is the way I want to go about finding and fixing these little glitches - I even want to do this if there is already a JIRA.

- Set up the LTI software on dev1.sakaicloud.org - make notes about what fails - I can look at the database on this server and deploy small fixes easily - and I don’t reset the database - so this is better for this kind of testing than a nightly or QA server

- Set up a Zoom meeting with me and a Sakai customer and someone from the LTI Tool Vendor and walk through the scenarios - I will record the Zoom session so I don’t have to write a ton of notes.

- I will review the Zoom video and create or update JIRAs with screen shots and videos.

Then I will group the JIRAs into three categories

- Just plain old bugs with a quick fix and easily back ported

- Low hanging fruit that can be fixed or implemented and back ported to 20 and 19

- Missing functionality that probably will be harder to back-port

Then I will move through these during July.  By focusing on this across all integrations at the same time - it makes it a lot easier for me and I think that we will find commonalities across what is going wrong.  And we should end up cleaning things up and meeting the Sakai-21 code freeze.

I just spent 90 minutes with Martin from TurnitIn looking through all the things he saw when testing Sakai’s LTI 1.3 support for TI.   We identified several categories of issues:

- UI Clunkiness at them moment of installation of a new LTI 1.3 tool - should be easty to fix and easy to backport

- UI clunkiness in the feature split between Lessons External Tools and Lessons Add Learning App - easy to fix - easy to back port

- Lack of support for important features of deep linking responses - mostly about setting up the gradebook columns - this is a big thing and will be hard to back port because I might need to add a column to the DB

- Little connivence / cleanup of launch data - Easy to do and easy to back port

- Make the membership service a little smarter - easy to do and easy to back port

- Handling of empty grades in LTI 1.3 - this is a bug - easy to fix and easy to back port

I will be making JIRAs, categorizing them and as I fix them circla back with TI and make sure that they test the changes.

I know there are open concerns from DigiExam and Pressbooks.  I would like to get a Zoom scheduled (this week would be great) with each so I can see where their issues overlap and /or are related to the issues that TII was so helpful in finding.

Thanks for your patience with me.  It was a heck of a race weekend for the SakaiCar.  As much time we have doing this and as much money I have spent on the race cars - you would think it would be easy by now.  But no.  Here is my racing blog post - it took two hours to write it all down.  It took 6 days of very little sleep to live it - thankfully no humans were harmed.

https://www.dr-chuck.com/csev-blog/2020/06/sakai-racing-team-24-hours-of-lemons-rustbelt-gp-2020/

Moving from fixing cars to fixing Sakai :)

/Chuck

[Pat Miller]

On this topic -- 

Here is a current issue from one of our 3rd parties -- Gradescope--it seems Sakai 19 is missing the grade sync endpoint??

From Gradescope support:

I reached out on Apereo's Slack to figure out what the alternative API endpoint would be to get the list of courses/gradebooks that an instructor has access to, and it seems that GBNG (Sakai 19's version of Gradebook) doesn't have that endpoint yet. And looking deeper into the upgrade, the endpoint to post grades ("/direct/grades/gradeitem/siteId/assignmentName.json") no longer exists in Sakai 19 either.

This is strange since we were using GBNG in Sakai 12 and the Gradescope Grade sync was working fine.  What happened in Sakai 19 such that Gradescope can't find the GBNG API??

Pat Miller

[Charles Severance]

This does not sound like an LTI issue - it is not something that LTI expects an LMS to provide.

/Chuck

[Pat Miller]

Right, Sam Ottenhoff indicates that Gradescope should be using the LTI outcomes service.  Is that a service in Sakai 1.1 as well as 1.3 or is it new to 1.3.   

Thanks,

Pat

[Charles Severance]

The LTI Outcomes service does not have an end point that can read all of the courses/gradebooks that an instructor can access.  The LTI Outcomes serves does not even give tools access to gradebook columns in a course that the tool did not create.  So LTI Advantage does not help here.

There are ways in some LMS’s to get all that data with their proprietary APIs - but *my opinion* has always been that giving that broad access to a tool (when launched form one course) to see all courses, all grade columns and all students grades in any course is not a good idea when FERPA, GDPR, and the other privacy laws are considered.

Do other LMS’s do it?  Yes.  Should they do it?  It makes faculty happy - so it seems like a great idea. Will they do it when they look at really complying with GDPR?  Probably not.

Of course none of that matters - because it is a very nice API for a tool and lets the tool muck around in the LMS data at will - allowing tools to do some very cool things.

If we want to walk away from the privacy high ground - we should just make a new API that in effect lets them spider all the grade data for an instructor in your LMS.  I would suggest that this is off by default so schools are eyes wide open when they give this access to a third party.

/Chuck

[Pat Miller]

So how is a 3rd party Vendor like Gradescope 'supposed' to get grades that faculty input into their tool to the LMS?  I assume there is some part of LTI 1.1 and/or 1.3 that supports that capability.  Maybe I (or Sam) is using the wrong terminology for how this is 'supposed' to take place.  

Pat

[Laura Gekeler]

Are you saying that using an instructor's login via the now defunct GB1 API gets access not only to the grades in the current course site from which the LTI was invoked, but to all grades in all courses the instructor has access to? That seems excessive and stupid(?) if I'm understanding this correctly.

   ~LauraG

Laura Gekeler
LMS Administrator, Learning Management Solutions

University of Notre Dame

[Charles Severance]

On Jul 1, 2020, at 4:07 PM, Pat Miller <pmi...@nd.edu> wrote:

So how is a 3rd party Vendor like Gradescope 'supposed' to get grades that faculty input into their tool to the LMS?  I assume there is some part of LTI 1.1 and/or 1.3 that supports that capability.  Maybe I (or Sam) is using the wrong terminology for how this is 'supposed' to take place.  

This is / was a thread about little glitches / rough edges in the LTI and LTI Advantage implementations in Sakai.  Sakai fully implements both specs.  But there are little things where it would “a lot nicer if..” - I am not talking about expanding the scope of these specs - just cleaning up little bits and bobs like "you change the title or max points one place and it does not change other places”.  

Sakai supports these LTI grade flows:

(1) Using LTI 1.1 a tool can send one grade back to Sakai - we all know that that works 

(2) LTI Advantage allows a tool to get the whole roster of a course from which it was launched

(3) LTI Advantage allows a tool to create as many grade book columns in Sakai that it likes and “owns” those columns and can do CRUD operations on those owned columns.  These columns show up like any other columns in our grade book.

So using LTI Advantage a tool like Gradescope could at its very first launch in a course - even by an instructor, load the entire roster and then go make 10 grade columns and then start reading, updating, and deleting those grades *in a course*.   

But (always a but) - there is no way in LTI Advantage to:

(4) Read all the courses owned by an instructor

(5) Read columns in a grade book that were not created by the tool (LTI Advantage) or created specifically for the tool (LTI 1.1) 

With that said, this is what you asked for:

From Gradescope support:

I reached out on Apereo's Slack to figure out what the alternative API endpoint would be to get the list of courses/gradebooks that an instructor has access to, and it seems that GBNG (Sakai 19's version of Gradebook) doesn't have that endpoint yet. And looking deeper into the upgrade, the endpoint to post grades ("/direct/grades/gradeitem/siteId/assignmentName.json") no longer exists in Sakai 19 either.

The question from Gradescope is not about LTI - it is about one of Sakai’s several Sakai-specific APIs (direct).  I said in my earlier responses to your message that the use case of "courses/gradebooks that an instructor has access to” is not part of LTI and unlikely to ever be part of LTI because of privacy concerns.

I don’t use /direct and actually know very little about it.  This could be a bug / feature request / regression in /direct (Laura’s note sounds like it *might* be a regression or removal of a feature) - and Sam and others know (infinity) percent more about /direct than I do because I know so little.  :)

We should separate the threads - I was looking more to pick up work on your needs w.r.t. DigiExam and LTI - which if I understand correctly *are* exactly about rough edges in our LTI support - which I do want to investigate and fix in the next few weeks.

I found a lot of the glitches from my interaction with Martin at TurnitIn - Martin wrote the PHP reference implementation of LTI Advantage from which I borrowed liberally to make the Tsugi LTI advantage and Martin has been involved in LTI Advantage and knows a lot about Sakai - so he found a lot of weak spots where it is working - but should work a little better :)   I just want to find more while I am at it :)

/Chuck

[Pat Miller]

Thanks much, Dr. Chuck, for this great clarification on LTI vs. API integration.   We are hopeful that Gradescope/Turnitin will soon complete their work on LTI 1.3 and we can get beyond their custom integration.  

We appreciate your efforts in assisting vendors like Digiexam to get their tools working  with Sakai LTI.  I will follow up with Digiexam and get back to you.

Sincerely,

Pat

[Charles Severance]

On Jul 2, 2020, at 8:13 AM, Pat Miller <pmi...@nd.edu> wrote:

Thanks much, Dr. Chuck, for this great clarification on LTI vs. API integration.   We are hopeful that Gradescope/Turnitin will soon complete their work on LTI 1.3 and we can get beyond their custom integration.  

I agree completely.  That was why when *the* top LTI Advantage developer from TurnitIn got ahold of me and said “We found a few LTI Advantage glitches in Sakai” - I was *all ears* and cleared my schedule. :)

I actually think that Turnitin found about 80% of the things that need improving - but I think DigiExam found several use cases that are related but not the same - and so it is easier for me to fix them all at one time - especially since some of them are a little tricky to fix and might require new data model items or something new from the gradebook APIs within Sakai.  And that is where I need some help from folks like Earle and Sam so I don’t want to do it twice.

Another cool thing this discussion triggered (but lets table for now) is that it made me think about adding non-standard extensions (like we did with LTI 1.1) that use the LTI Advantage security model.  And while I don’t want to open that can of worms until -21 is in good shape - I mentally took a quick peek into the can and it was not as scary as I expected.

/Chuck

[Michael Greene]

We'd love to help here too, particularly with the deeplinking workflow, I'll ask our LTI1.3 vendors to do this.

MG

[Charles Severance]

On Jul 2, 2020, at 1:24 PM, Michael Greene <michael...@duke.edu> wrote:

We'd love to help here too, particularly with the deeplinking workflow, I'll ask our LTI1.3 vendors to do this.

MG

Yeah - I need to follow up on that - that is more of a “within Sakai” thing.  When talking to your LTI 1.1 or 1.3 vendors - I will bet a lot of the issues are the fact that the link between a gradebook column and an LTI launch is based on the name of the column - which made things simple when we built it - but then little things don’t work as expected.

Also more and more of them are building increasingly complex Deep Link workflow and adding stuff that we had never seen and Sakai ignores it.

This is a natural evolution of a spec’s usage.  You have to build it and use it to see how certain features get used and then adjust.

/Chuck