User limit of inotify instances reached or too many open files

[Robert]

Hi all,

We ware running CAS 6.2.8 on Ubuntu 20.04.2 LTS with OpenJDK 11.0.10.

After an uptime of about 2 weeks we've hit the following error resulting that scripted attributes became just empty:

"User limit of inotify instances reached or too many open files"

Increasing the inotify max_user_instances from 128 to 256 via "echo 256 > /proc/sys/fs/inotify/max_user_instances" solved that probem just temporaly, because this limit was reached again within 1 day.

Ive tried to find out what was causing this issue and used "lsof -p $(pgrep java) | grep notify" to see a lot (253) anonymous inodes. Ive created a Java Heapdump to found 263 instances of a FileDescriptor.

Now im out of ideas to find the root cause, but I guess, that there is some kind of leak.

Someone else experienced that issue already?

[Jephte Clain]

hello,

we noticed that CAS doesn't release inotify handles when watching changes on groovy scripts

as our scripts doesn't change (CAS is run inside a docker container), we ended up disabling watching changes (thanks to Jerome Leleu for his help on this)

I attach the modified class. I won't be able to help you more than that, sorry

hope this helps. regards,

Jephté Clain

--
- Website: https://apereo.github.io/cas
- Gitter Chatroom: https://gitter.im/apereo/cas
- List Guidelines: https://goo.gl/1VRrw7
- Contributions: https://goo.gl/mh7qDG
---
You received this message because you are subscribed to the Google Groups "CAS Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cas-user+u...@apereo.org.
To view this discussion on the web visit https://groups.google.com/a/apereo.org/d/msgid/cas-user/db4d659f-9cdd-4523-8f6e-f572388c83d8n%40apereo.org.

--

Jephté Clain

Développeur, intégrateur d'applications

Direction des Systèmes d'Information

Université de La Réunion

Interne : 2107 | Externe : +262 262 93 86 31 | Mobile : +262 692 29 58 24

[Robert]

Thanks for the great hint. That helps a lot.

The core question for the CAS devs stays: Where does that leak come from? I mean, FileWatcherService and PathWatcherService seem to be straight forward.