external_auth, CAS and UserProfile

[Eugene Medvedev]

So I'm here again with another involved question...

There's that plan in my organization of implementing a central authentication server that would handle logging into edX and numerous other side jobs, including managing user metadata, and right now, the options I'm considering are an OAuth2 server and a CAS protocol server. (using a Django implementation thereof, but that's beside the point.) CAS is currently looking the most enticing, since I should be able to have absolutely seamless login this way without any extra clicking, and also because CAS will work in Studio, while OAuth2 will not.

The code that logs people into edX through external auth has a comment that doesn't inspire, though: "We assume that user details are maintained by the central service, and thus an empty user profile is appropriate." The actual code confirms this, as the only thing that does get into the UserProfile is the user's name. And even that is actually the Django username, not even assembled from the first/last names returned by the server.

Well, the user details will be maintained by the central service indeed, but I need edX to be aware of at least some of them. I don't see any practical way to get everything I need in there, though -- the external authentication wiki page  describes how one would set the fields of the User object from whatever CAS returns, but that whatever the custom attribute handler returns is going to be fields of User, and these are kind of limited -- getting the name through seems easy, but there's quite a bit more metadata I need to be there.

Unless I, in my inexperience, am missing something blindingly obvious, I'd have to make an extra request for metadata to create a proper UserProfile object, which seems silly. Any advice?

[Carson Gee]

I haven't fully experimented with it yet, but I think the attribute mapper can import other modules such as the UserProfile model and populate all the attributes there if you return them from your CAS server (like language, etc).

Part of the problem here is CAS, which generally doesn't return anything at all except the username and that the authentication succeeded.  The feature of adding and mapping additional attributes is an extension that has varying support among CAS servers (the Django based one we use, as well as the Jasig one support them though).  The other part of the problem is that the CAS integration was done minimally and isn't fully integrated into the profile creation process like shibboleth and oauth2.  This largely is because for our use case we don't generally need that information for our users in the platform and thus try to make the user experience better by skipping profile creation.  If you wanted to add that part back in, I don't think it would be too hard since other providers do this  as well, we would just want to be able to skip it as it is done now.

It is somewhat unfortunate that there isn't better external authentication integration into studio, but I think since the course creation process usually has so few users in comparison to the LMS that it isn't heavily prioritized.

[Eugene Medvedev]

On Saturday, 13 September 2014 04:08:02 UTC+4, Carson Gee wrote:

I haven't fully experimented with it yet, but I think the attribute mapper can import other modules such as the UserProfile model and populate all the attributes there if you return them from your CAS server (like language, etc).

Thanks, I didn't think of that. By the time it gets called, User should already exist, so it should be able to just manage the associated UserProfile, and edX only creates an empty UserProfile object if none exists after the CAS login is done, which would happen after the attribute mapper returns.

That's certainly smoother than hacking up the cas_login function that I had in mind. :)

Part of the problem here is CAS, which generally doesn't return anything at all except the username and that the authentication succeeded.  The feature of adding and mapping additional attributes is an extension that has varying support among CAS servers (the Django based one we use, as well as the Jasig one support them though).

I've been thinking of using https://github.com/jbittel/django-mama-cas/ which does support responding with arbitrary attributes as far as I can see, so this, at least, doesn't sound like it would be a problem.

 

The other part of the problem is that the CAS integration was done minimally and isn't fully integrated into the profile creation process like shibboleth and oauth2.  This largely is because for our use case we don't generally need that information for our users in the platform and thus try to make the user experience better by skipping profile creation.  If you wanted to add that part back in, I don't think it would be too hard since other providers do this  as well, we would just want to be able to skip it as it is done now.

I try to stay as far away from altering edX core code as possible, because this entire project started for me with discovering that my predecessors left me with a snowball of unmanageable merge conflicts, and we've ended up starting from scratch. It'll be a long, long while before I work up to submitting a PR.

[Eugene Medvedev]

On Saturday, 13 September 2014 11:25:25 UTC+4, Eugene Medvedev wrote:

Thanks, I didn't think of that. By the time it gets called, User should already exist, so it should be able to just manage the associated UserProfile, and edX only creates an empty UserProfile object if none exists after the CAS login is done, which would happen after the attribute mapper returns.

Just a followup for whoever googles for this later:

Attribute mapper function can import any model objects, but it can't necessarily do that at file level. This is because it is being loaded by django settings to get a function into a variable so that django_cas can call it later during login. At the time of import some settings do not exist yet, which causes the import to fail.

The solution I came up with turned out to be rather inelegant: Import UserProfile within the function body, rather than at the top of the file, which makes it only fire when the mapper function is actually invoked. As long as you don't have millions of users up at the same time it shouldn't matter, but if you do, you might want to look for other methods.

[Nilesh Londhe]

Hi Eugene:

I followed your guidance on this thread but am running into issue. CAS Server logs me in and sends me back to birch.rc3 and I see 500 error. 

What may be causing this? I posted on the logs ops mailing list https://groups.google.com/forum/#!topic/openedx-ops/kKLPUp3E_m4

Appreciate your help.

Thanks.

-

Nilesh

[Eugene Medvedev]

On Sunday, 22 February 2015 09:21:51 UTC+3, Nilesh Londhe wrote:

 

I followed your guidance on this thread but am running into issue. CAS Server logs me in and sends me back to birch.rc3 and I see 500 error. 

What may be causing this? I posted on the logs ops mailing list https://groups.google.com/forum/#!topic/openedx-ops/kKLPUp3E_m4

Let's start with listing what I know you definitely did wrong:

    CAS_URI = 'https://testing.cloudgeni.us/wp-cas/'

I did that too initially and to my chagrin later found out that 'http://www.yale.edu/tp/cas' is what it's supposed to be according to CAS protocol and it is not meant to refer to your particular server/clients environments, it's part of XML namespace identifier.

The next thing is, your logs say:

IOError: [Errno socket error] [Errno 111] Connection refused

Well, I expect this is exactly what's happening. CAS protocol is a bit more complex than that, but it kind of goes like this:

1. Browser requests a page from the CAS client.
   
2. CAS client finds out browser isn't logged in. In the http response, it redirects the browser to an URL on the CAS server. In GET parameters to this URL it tells the CAS server who the client is and where to send the browser back if there's a legit user behind them.
   
3. Browser takes this data and goes to the CAS server. Assuming the browser is logged into the CAS server, what they get is another redirect, now back to CAS client, with more parameters - the most important of which is the ticket, a unique string.
   
4. Browser carries over the request to the CAS client. CAS client receives it
5. Then CAS client tries to make a request directly of the CAS server, presenting the ticket, all the while keeping the browser hanging..
6. If the CAS server recognises the ticket as one it had issued, it responds with an XML packet, containing 'authenticationSuccess' and other interesting metadata you'll be parsing in the mapper.
   

What it means for setting this up is that the following must be true:

• Users can http-request both CAS server and CAS client with no basic auth or other access restrictions.
  
• CAS client can http-request CAS server directly with no basic auth or other access restrictions.
  

In your case it appears the second of these is not true and step five of the above described process fails.

[Nilesh Londhe]

Hi Eugene:

Thank you for an awesomely helpful response. What tools would be best to drill down and debug why step 5 fails on my setup?

Nilesh Londhe

http://be.a.cloudgeni.us

[Eugene Medvedev]

On Tuesday, 24 February 2015 00:49:34 UTC+3, Nilesh Londhe wrote:

Hi Eugene:

Thank you for an awesomely helpful response. What tools would be best to drill down and debug why step 5 fails on my setup?

Start with pinging the CAS server from CAS client, then running a wget/curl/console browser on the CAS client to see if there's a connection at all and whether the server responds with anything interesting when you try to do it manually -- the actual request should be going out to somewhere like "https://testing.cloudgeni.us/wp-cas/validate/". If you did not specifically lock up your CAS server against unauthorised access, there's most likely a routing problem of some sort which you need to find. If you did, you need to figure out a method to permit CAS clients to directly access the CAS server without presenting credentials, and which one will do depends very much on your setup.

[Nilesh Londhe]

I am in a test environment and no specific lock downs in place.

Terminal on CAS client (openedX) when pointed to CAS server say no which means openedX is reaching CAS server and getting a no response because I am not passing the expected service ticket via this curl.

root@domain:/edx/var/log/lms# curl https://testing.cloudgeni.us/wp-cas/validate 

no

Browser in my workstation shows

Forbidden
Login failed.

and I see a service ticket in the browser URL.

http://openedX.domain/cas-auth/login/?next=%2Flogout&ticket=ST-TmlsZXNofGh0dHA6Ly9vbmxpbmUuY2xvdWRnZW5pLnVzL2Nhcy1hdXRoL2xvZ2luLz9uZXh0PSUyRmxvZ291dHwxNDI0NzUzNTMxLjAwODd8ODZmOGNiMGIyYTk4ZWI5MjcxOTkzNGZlNTg2NTEzZWM4ZWFiNGJhOQ=

This ticket=ST-TmlsZXNofGh0dHA6Ly9vbmxpbmUuY2xvdWRnZW5pLnVzL2Nhcy1hdXRoL2xvZ2luLz9uZXh0PSUyRmxvZ291dHwxNDI0NzUzNTMxLjAwODd8ODZmOGNiMGIyYTk4ZWI5MjcxOTkzNGZlNTg2NTEzZWM4ZWFiNGJhOQ= looks like a service ticket CAS server sent to openedX CAS client.

Is there a way I can use this service ticket from browser URL and pass through curl in hopes to receive a different response other than no ?

Nilesh Londhe

http://be.a.cloudgeni.us

​

[Nilesh Londhe]

I tried this on CAS client. CAS server says it needs to receive a ticket. Looks like CAS client is able to reach CAS server. 

curl https://testing.cloudgeni.us/wp-cas/serviceValidate  

<?xml version="1.0" encoding="UTF-8"?> 
<cas:serviceResponse xmlns:cas="http://www.yale.edu/tp/cas"><cas:authenticationFailure code="INVALID_REQUEST">Ticket is required.</cas:authenticationFailure></cas:serviceResponse>

​

Nilesh Londhe

http://be.a.cloudgeni.us

[Eugene Medvedev]

On Tuesday, 24 February 2015 08:12:07 UTC+3, Nilesh Londhe wrote:

Is there a way I can use this service ticket from browser URL and pass through curl in hopes to receive a different response other than no ?

In theory yes, though I've never had the occasion to try myself. You get something like 60 seconds at most to do it before the ticket is expired, depending on server settings. The request should be like the one that fails for you on this exact line: https://github.com/mitocw/django-cas/blob/master/django_cas/backends.py#L54 - i.e. https://<domain>/wp-cas/proxyValidate/?ticket=<ticket>&service=[http|https]://<edx domain>/cas-auth/login - and I the service url needs to be urlencoded.

I tried this on CAS client. CAS server says it needs to receive a ticket. Looks like CAS client is able to reach CAS server. 

This is certainly unusual. Either this only happens part the time and it should work at least once eventually or there's something particularly special about the request that actually goes out of edX.

Is there anything interesting in CAS server logs?

[Nilesh Londhe]

My CAS server seems to work with a different CAS client I setup. Do you have success with birch as CAS client? Which git tag are you running CAS with?

Nilesh Londhe

http://be.a.cloudgeni.us

[Eugene Medvedev]

On Wednesday, 25 February 2015 08:51:05 UTC+3, Nilesh Londhe wrote:

My CAS server seems to work with a different CAS client I setup. Do you have success with birch as CAS client? Which git tag are you running CAS with?

 

We don't use named releases, we track the upstream release branch in a fork that contains local customizations and try to keep it up to date. Right now I have the dev server which is current up to release-2015-02-19 and two production servers (long story) which are two months out of date or three (whichever it was when TNL-726 happened, I forget) and I had a version forever stuck somewhere in spring of 2014 before, and they all worked fine with the same CAS server, although the mapper code had to be different between them because of changes to CourseEnrollment objects. 

The version of edX you're running is very unlikely to make a difference for CAS login itself, since edX relies on a particular version of django-cas library, patched to work with django 1.4, which hasn't changed for quite a while: https://github.com/edx/edx-platform/commit/189284ace6c8525dc739eff2f8a99216c735d83c and basically doesn't do anything about CAS itself other than feed the library configuration parameters.

[Stuart O'Day]

Hi,

   I ran into the same error while configuring devstack, running named-release/birch in vagrant,  IOError: [Errno socket error] [Errno 111] Connection refused, and it turned out to be a connectivity issue during the back-channel validation of the Ticket from my Open edX Server (running in the vagrant VM) to the CAS Server.  (See step 6 in this diagram).  My Open edX Server was incorrectly resolving the DNS Name in the CAS Server URL. Once that was fixed, everything started working.  

Not sure if this situation is related to yours, but I thought I would post anyways.

Stuart

[Eduardo Cuomo]

This is still happening...

[Stuart O'Day]

Not sure what 'This is still happening...' means.  

This discussion from early 2015 was related to (at least on my part) a mis-configuration of my CAS Server DNS Name in my devstack environment, which broke the connectivity between my edX application and the CAS server. 

Since the Dogwood release (including the Eucalyptus release) there have been other issues with CAS. See the discussion and my post here: 

https://groups.google.com/forum/#!topic/openedx-ops/XU0zcGH4S2o