Strange mod_auth_cas behavior (no cookies created in CASCookiePath)

[Neil Sabol]

Hello CAS Community,

I hope this message finds you all well.

As time permits, I am hoping to pick your brains about a mysterious issue we experienced recently with mod_auth_cas (suspect it was not mod_auth_cas itself but something related).

We have been running mod_auth_cas (version 1.1) in production for a long time without incident. Yesterday, we began to experience a strange behavior on one of our production servers:

• mod_auth_cas stopped creating cookies in the defined CASCookiePath (no users were able to login to the application - all requests for CAS-protected resources resulted in a redirect back to the CAS login page and a 401 error upon return to the application)
  
  
• Debug logs did not reveal anything interesting - the only related entries I noticed were the following
  
  [debug] mod_auth_cas.c(930): [client X.X.X.X] Cache entry 'ae0aa61bf431d62b9e4be00089e87df8' could not be opened, referer: http://something.unm.edu
  [debug] mod_auth_cas.c(1676): [client X.X.X.X] Cookie 'ae0aa61bf431d62b9e4be00089e87df8' is corrupt or invalid, referer: http://something.unm.edu
  
  
• Permissions, file system status, etc. were all good - from all appearances, mod_auth_cas was not attempting to create cookies in the CASCookiePath (confirmed apache could write to the path, etc.)
  
  
• The CASCookiePath directory contained only a .metadata file about 2-3 hours after this issue started occurring

We ended up using the IT hammer to restore the affected VM from snapshot, so I no longer have the specific logs or state of the system available. The restore did the trick (mod_auth_cas resumed normal operation and began creating cookies in the CASCookiePath), but I am concerned this issue may recur.

The only possible explanation for this that I can think of (in hindsight) is time drift between the application server/clients/cas server. Does that sound possible? If yes, would something like that be logged with debug logging enabled?

If you have any insight or guidance into what could cause this sort of situation with mod_auth_cas, please let me know.

Thank you in advance for your time and expertise!

-Neil

[Matt Smith]

Hi Neil,

Without the logs, it is difficult to tell.  It /could/ be related to time drift, but I'd find it unlikely that that would prevent writing to disk.

More likely, I'd investigate number of open file handles.  Did some httpd sub-process (e.g., a CGI or PHP) possibly create an egregious number of handles?  This would likely show in error messages printed to the logs.  lsof could also be your friend here.

Matt

--
- CAS gitter chatroom: https://gitter.im/apereo/cas
- CAS mailing list guidelines: https://apereo.github.io/cas/Mailing-Lists.html
- CAS documentation website: https://apereo.github.io/cas
- CAS project website: https://github.com/apereo/cas
---
You received this message because you are subscribed to the Google Groups "CAS Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cas-user+unsubscribe@apereo.org.
To view this discussion on the web visit https://groups.google.com/a/apereo.org/d/msgid/cas-user/9b0635b6-657c-4b2e-a091-3acd4b0fec1c%40apereo.org.

[Neil Sabol]

Hi Matt,

Thank you for the quick reply and information - I had not considered file handles as a culprit. If the issue recurs, I will dig into that.

We did find some anomalies on the virtual machine where the mod_auth_cas behavior manifested (outdated VMware tools and really old VM hardware version). Not sure if those were related but we've since updated both. Will keep this list posted with our findings and experience going forward.

Thank you again for your time, suggestion, and expertise - it is appreciated!

-Neil

To unsubscribe from this group and stop receiving emails from it, send an email to cas-user+u...@apereo.org.