Disable caching of offline user and client session

[Peter Flintholm]

Hi all,

I'm considering implementing support to disable the infinispan cache completely for offline user and client sessions, and instead operate directly on the database.

My usecases involves large numbers (>3mill) of long lived offline sessions that are only used infrequently.

It seems to me that a lot of complexity can be removed for syncronization/tx across clusters etc. So the purpose is both to reduce memory requirements, but also to improve stability in the cluster.

My plan is currently to make an alternative implementation of UserSessionProvider to InfinispanUserSessionProvider, but there are quite a lot of explicit references to the InfinispanUserSessionProvider in e.g. UserSessionAdapter.

Does anyone have any suggestions/caveats for this project ?

-Peter

[Schuster Sebastian (IOC/PAU1)]

Have you seen Thomas’ work to support lazy loading of offline session that keeps them (mostly) in the DB and is therefore targeted at infrequently accessed offline sessions?

https://github.com/keycloak/keycloak/pull/7722

 

Best regards,

Sebastian

 

 

Mit freundlichen Grüßen / Best regards

Dr.-Ing. Sebastian Schuster

Project Delivery Berlin 22 (IOC/PDL22)
Bosch.IO GmbH | Ullsteinstr. 128 | 12109 Berlin | GERMANY | www.bosch.io
Tel. +49 30 726112-485 | Mobil +49 152 02177668 | Telefax +49 30 726112-100 | Threema / Threema Work: MF9VMEAE | Sebastian...@bosch.io

Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten Lücke; Geschäftsführung: Dr. Stefan Ferber, Dr. Aleksandar Mitrovic, Yvonne Reckling

--
You received this message because you are subscribed to the Google Groups "Keycloak Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keycloak-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/2b5ec69e-acc6-42e1-b115-5e0a748d25d0n%40googlegroups.com.

[Peter Flintholm]

Yes indeed. While this is a great feature, it doesn't quite solve my problem. I expect most users to perform a refresh at least once in between restarts of the cluster, so the cache size and fragility of syncronization of caches between instances will be the same.

-Peter

[Thomas Darimont]

Hello Peter,

Adding to what Sebastian wrote: if you configure the infinispan UserSessionProvider with "preloadOfflineSessionsFromDatabase=false"  

AND set a low capacity (e.g. <heap-memory size="1000"/>) for the offlineSession and offlineClientSession caches, you could effectively achieve what you want without having to write your own UserSessionProvider.

The current implementation of the InfinispanUserSessionProvider (since 14.0.0?) will lookup offline-sessions from the database if they are not found in memory.

But I agree with you here: Having a purely database backed UserSessionProvider for offline sessions handling, which would not load offline sessions in memory at all, would be a better option long term. 

I also tried this when I touched that code the first time, but had to abort the implementation after some discussions with the Keycloak team because the required changes were considered too large.

That's why I only added the fallback database lookup for offline sessions when I added the support for lazy-loading.

Cheers,

Thomas

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/f1f2bcbf-51b3-4726-af5b-321800bc6cabn%40googlegroups.com.

[Peter Flintholm]

Hi Thomas,

 

Oh yes, that is a very good point! This would of course make it possible to control memory usage.

 

I actually tried this back when I was optimizing preload, and if I remember correctly it failed due to some cache eviction issues. I think the eviction strategy was being changed around version 12 so I might give it another try.

I guess a setup like this will first have the overhead of distributing the cache objects, and very soon thereafter have additionally overhead for distributing the eviction. It is precisely these async distributed tasks I want to avoid.

 

Would it perhaps be possible to configure the offline user session and client session caches to be local infinispan caches instead of distributed ?

 

Could you perhaps share some of the considerations from your initial discussion with the Keycloak team regarding a pure database backed version ?

For now, I’m still optimistic about writing a pure database version 😊

 

Kind regards,

Peter