LTI ContentItem, sakai.session as a URL parameter, and clustering

[Stephen Marquard]

Hi all,

 

We have an issue adding LTI ContentItem items from non-local LTI providers like h5p.com. We currently have SameSite=lax (the default). On latest Chrome, h5p issues a POST to

 

"POST /portal/tool/TOOLID/sakai.basiclti.admin.helper.helper?eventSubmit_doSingleContentItemResponse=Save&flow=lessons&sakai.session=SESSIONID&returnUrl=https%3A%2F%2Fvula.uct.ac.za%2Fportal%2Fsite%2F79f7e453-662b-4465-94bd-606506a3b250%2Ftool%2F3d70fb1b-670f-46e4-b91c-30820033bedc%2FBltiPicker%3F83977%26itemId%3D-1%26addBefore%3D&panel=PostContentItem&tool_id=195&sakai_csrf_token=TOKEN

 

The SESSIONID is what would normally be in the JSESSIONID cookie, except that the POST request does not include this as a cookie. We use mod_jk as load-balancer which does not pay attention to the sakai.session request parameter (it uses the JSESSIONID cookie for app server affinity) and can send this request to another app server on which the specified sakai.session is not valid, and so the request fails because it’s not authenticated.

 

It seems SameSite=none might work around this but is undesirable. Is the requirement that load-balancers need to also pay attention to the sakai.session parameter for app server affinity? If so, does anyone have a working configuration for apache + mod_jk (or any other working load-balancer configuration for this, e.g. apache + mod_proxy_ajp).

 

Cheers

Stephen

 

---

Stephen Marquard, Acting Deputy Director

Centre for Innovation in Learning and Teaching (CILT)
University of Cape Town
http://www.cilt.uct.ac.za
stephen....@uct.ac.za
Phone: +27-21-650-5037 Cell: +27-83-500-5290

 

Disclaimer - University of Cape Town This email is subject to UCT policies and email disclaimer published on our website at http://www.uct.ac.za/main/email-disclaimer or obtainable from +27 21 650 9111. If this email is not related to the business of UCT, it is sent by the sender in an individual capacity. Please report security incidents or abuse via https://csirt.uct.ac.za/page/report-an-incident.php.

[Sam Ottenhoff]

Hi Stephen,

Yes, to support this POST back to the LMS, I think you can either revert back to SameSite=None (old browser behavior) or add the sakai.session GET parameter as a holder of the stickysession value.

I don't use Apache HTTPD much anymore, but it looks like ProxyPass supports this kind of setup:

ProxyPass / balancer://sakaicluster/ stickysession=JSESSIONID|sakai.session

Balancer sticky session name. The value is usually set to something like JSESSIONID or PHPSESSIONID, and it depends on the backend application server that support sessions. If the backend application server uses different name for cookies and url encoded id (like servlet containers) use | to separate them. The first part is for the cookie the second for the path.

Available in Apache HTTP Server 2.4.4 and later.

--
You received this message because you are subscribed to the Google Groups "Sakai Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sakai-dev+...@apereo.org.
To view this discussion on the web visit https://groups.google.com/a/apereo.org/d/msgid/sakai-dev/AM6PR03MB5847566429BBD61411B4A2BED3BC0%40AM6PR03MB5847.eurprd03.prod.outlook.com.

[Stephen Marquard]

Hi everyone

 

We ended up going with this configuration in our apache 2.4 + mod_jk 1.2 config:

 

<Location /portal/tool/>

         SetHandler jakarta-servlet

         SetEnvIfExpr "%{QUERY_STRING} =~ /sakai.session=[0-9a-f-]{36}.(vula[0-9ab]{4})/" JK_ROUTE=$1

</Location>

 

which routes the request to the right node. There may be a simpler or more general configuration for this, but the above does seem to work. Note the regexp matches our worker names explicitly so this is site-specific.

 

This seems like something of a configuration “gotcha” for using LTI ContentItem perhaps should be documented more explicitly somewhere, or we should site SameSite=none by default (though less secure).

 

Regards

Stephen

[Charles Severance]

Stephen, 

I am glad you got this figured out.  We could add this to the ever-growing internal documentation for LTI at basiclti/docs folder in github.   It would be great if someone who understood the problem directly wrote the first cut of the LOADBALANCER.md document - I would add some of my experience running LTI tools behind a proxy like CloudFlare.

Perhaps we should file a JIRA to write this documentation to make it easier to find in the future as well.

If you wrote a few paragraphs - I would take care of making the file and adding my CloudFlare bits to it.

I would also add that that as browsers get more and more grumpy about cookies in iframes, two things are happening:

(1) I make these back-and-forth flows increasingly resilient to things that go wrong - so keeping your Sakai up to date is a good hedge against cookie/session related gotchas

(2) The LTI working group is increasingly looking at converting some of these flows from redirect to windows.postmessage() flows and webservice callbacks with one-time tokens.  This is early days - but it allows us to never share a session at all, and does not depend on cookies at all, and works flawlessly in an iframe.   The new automatic LTI Advantage provisioning (not released by IMS - with an early version shipping in Sakai 21) is an example of this new - no session - no cookie flow needed.

So until we have new protocols - Sakai will remain ever vigilant in keeping these flows working - but in the long run a better solution is on the horizon.  Some of this was talked about in the Sakai LTI briefing last week:

https://www.youtube.com/watch?v=cwOURotVrYg&feature=youtu.be

Hope this helps.

/Chuck

To view this discussion on the web visit https://groups.google.com/a/apereo.org/d/msgid/sakai-dev/AM6PR03MB584744FB501F5BCC32B16A62D3BB0%40AM6PR03MB5847.eurprd03.prod.outlook.com.

[Ashley Willis]

This is a really timely thread thanks!

Since upgrading to Sakai 20 in early December we have really had a time with external LTI tools. We use quite a few and most of them seem to have issues posting back to Sakai. In debugging these, I can clearly see that the cookie is getting dropped and a new cookie is being set on post back to Sakai. This results in creating a new session on a different host and the user gets presented with a login screen and sometimes kicked to the new host with the new session. If I force sameSite = none in the RequestFilter it seems to work but I'm not sure that is the right approach.

It looks that you are trying to workaround this by pulling the sakai.session from the query string and telling the Load Balancer to treat it as the jsessionid for routing? Sorry if I missed something, Apache/Tomcat/Load Balancer config is handled by a different team, trying to get my head around the changes so we can work with them to address it.

FWIW, this all works fine in Sakai 12 so I am curious what changed in 20 to make this a requirement?

Thanks!

Ashley

UNC Chapel Hill

[Charles Severance]

Ashley,

TL;DR - It is a much better strategy to adjust your load balancer than to set sameSite=none

It is surprising that it seems that Sakai 12 works “better" than Sakai 20.  I would like to understand that better.  

Starting with Sakai-19 there are several trends that affect these “in-iframe redirects”:

- LTI Advantage has 3 redirects without user interaction (i.e. a “continue button” button) in the frame while LTI 1.1 had only one redirect in the frame

- As Sakai added the sameSite setting and set it to lax, three years ago I started evolving LTI anticipating that sameSite=strict might be forced on us by browsers - I did some tests with sameSite=strict - I won’t say that it is 100% perfect under sameSite=strict so stick with sameSite=lax - but some progress was made - if in the future we are forced to use sameSite=strict - we have a leg up on that

- Current browsers are more and more simply refusing to send cookies inside iframes (i.e. sliding into sameSite=strict regardless of the sameSite setting) - different browsers are taking cookies away under different scenarios but the writing is on the wall - sooner or later we won’t be able to set or depend on cookies inside iframes.  Just as an example - I was doing some testing with LTI Advantage in Sakai-21 between Sakai and Tsugi and cookies worked on the first request - but then once a few redirect’s happened the browser decided the site in question was “acting like a tracker” and shut the cookies off - literally they stopped working in the *middle* of a sequence of redirects.

There are a few ways to “keep going” …

- Use GET parameters and take steps to re-establish the cookie when they use comes back (the way Sakai does it for now) - what we have works and there are ways it can and should be improved.  This way tools work around their session issues and Sakai works around its session issues.

- Open a new window and futz with cookies - this is really jarring for the user - this means that we pause the launch and say “hang on while we pause the launch and pop up a window” - once you do something in that window, close the window, come back to this window and press a button to continue - if the cookie did not make it - we pop up another window.  The browsers assume that if things are happening without user interaction, you must be a tracker.  Moving something that currently requires zero user interaction to something that takes 2-3 clicks will make LTI not very convenient at all.

- Ask the user to relax the browser security for a site - this is a non-starter because the browsers inevitably name the field you set something like “Enable evil tracking to be done to you on nightly.sakaiproject.org” - the browsers want to kill cookies in iframes and that is that.  Users tweaking their browsers is clearly a non-starter.

- Future LTI specs are being designed not to demand cookies but instead to use windows.postMessage() instead.   This works great - it is being used in the “Auto provision LTI Advnatage spec” and I love it - but old specs like LTI 1.1 ContentItem and even the core LTI Advantage and Deep Links - are stuck with cookies or a workaround for a long time.  Even if new practices are put in place - existing tools will in effect never change their protocol - they always blame the LMS rather than change their code.  Even if IMS deprecates a spec and does not allow future certifications - the tool vendors just say  they comply and it is the LMS fault.

- There are other tricks that some tools are using like local storage and a frame within a frame - but these are complex and feel brittle to me and are not long term solutions.

Sooo… TL;DR is get your load balancer configured and then we will move forward keeping this stuff working the best we can as browsers make it increasingly difficult.  Lets not “give up” and go sameSite=none

This is part of the reason for our new LTI sessions every month or so.

/Chuck

[Hans Rubio]

Hello Ashley, I was seeing an email in the sakai community about a problem that you indicated on January 29, 2021 about cookies being dropped and that causes you to have a problem when creating a new session on a different host and when user is presented with a login screen and sometimes sent to the new host with the new session. If I force sameSite = none on RequestFilter it seems to work, but I'm not sure it's the correct approach.

Here in Chile, at the university where we are working, we have the same problem that I mentioned, my query is two:

1.- the problem could be solved with sameSite leaving the value in RequestFilter ?.

2.- in which part is the sameSite option configured in sakai?

I would appreciate the information very much because I did not find clear information about that problem, we are using version 20.2 of Sakai.

Thanks and regards.

[Ashley Willis]

Hi Hans,

FWW, I wouldn't recommend forcing sameSite = none on a production system, we only used that for troubleshooting in our non-prods to prove it was a cookie issue. The cookie is dropped when you go to a 3rd party site. On callback, the value is preserved in the query string as sakai.session. Sakai needs that cookie to re-establish the user's session on the correct host. To solve the issue, we created a rule similar to Stephen's in PoxyPass that pulls the cookie from the query string (sakai.session)and uses that value to reestablish the session.

 

Our solution ended up looking like this:

ProxyPass / balancer://sakaiCluster/ stickysession=JSESSIONID|jsessionid scolonpathdelim=On
ProxyPass /portal/tool/ balancer://sakaiCluster/ stickysession=JSESSIONID|sakai.session scolonpathdelim=On

I agree this solution should be documented, it is a pretty significant change IMHO and impacted our 20 upgrade quite a bit. We have been running with that fix in production for roughly 4 months now with no issues.

Cheers,

Ashley

You received this message because you are subscribed to a topic in the Google Groups "Sakai Development" group.
To unsubscribe from this topic, visit https://groups.google.com/a/apereo.org/d/topic/sakai-dev/GSUHzMGcpl0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sakai-dev+...@apereo.org.
To view this discussion on the web visit https://groups.google.com/a/apereo.org/d/msgid/sakai-dev/a1977ce0-76b3-499d-82f6-85c18cb4249bn%40apereo.org.

[bernardo.garcia]

Hello,

We have a similar problem and we believe it might be related to this. Has anyone faced this issue using an AWS load balancer? And if so, which solution was applied?

Does anyone remember what kind of errors were the LTI attempts throwing when failing? So we can at least verify it's the same reason...

Thanks everyone and regards.

[Earle Nietzel]

Hi Bernardo,

You could look at https://sakaiproject.atlassian.net/browse/SAK-48484

-earle

To view this discussion on the web visit https://groups.google.com/a/apereo.org/d/msgid/sakai-dev/7ff14d7d-2f04-4064-b10e-7fd1a40a5436n%40apereo.org.

[bernardo.garcia]

Hi Earle,

Thanks, but we have that one already applied so it has to be a different thing.

Regards.