Multiple realms performance

[Sagun Tumkar]

Hi There,

We are currently evaluating keycloak and trying to figure out if it is ok to create one realm per customer. In our case we will end up creating more than 500 realms if we take this route. I have couple of questions.

1) Is it ok to do this or would there be any performance implications?

2) If there are performance issues, is it possible to create single realm for all customers and divide them by creating groups (one group per customer)? Or is there a better way?

I would really appreciate if someone who already solved this use case can share their thoughts. Thanks in advance.

~Sagun

[Pedro Igor Craveiro e Silva]

Hi,

I would not recommend doing so, for now. The reasons are:

* Admin console might get unusable as the number of realms grows. And this might impact your availability.

* There are some operations that rely on looping over realms

* We lack optimizations to DB queries used to load realms

* There are places that perform constantly operations to discover the roles users have that involves going over all realms + processing composite roles

* Time to migrate

We did some research on this area some time ago but, although we started some changes to make it happen, it was discontinued due to other priorities and our plans around the new store.

It is not that you can't use multiple realms though ... I know some people are following a realm per-tenant approach. Don't have numbers to share but perhaps you can get more feedback here. But we know there are limitations like those aforementioned that might impact any deployment.

A single realm + groups should work. Based on groups you should be able to manage users on a per-customer basis, mainly in regards to permissions.

Regards.

Pedro Igor

--
You received this message because you are subscribed to the Google Groups "Keycloak Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keycloak-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/05a1468d-c8ef-42c3-9b61-1a0efe016d6cn%40googlegroups.com.

[Sagun Tumkar]

Thanks Pedro. Hoping to get more replies to see if anyone is doing this differently as this is very basic use case. I am worried if we take the approach of  single realm and multiple groups, would we end up in any unforeseen issues.

--

Sagun Tumkar | Sr. Software Engineer, Application Platform | M: (650-863-4922)

[Pedro Igor Craveiro e Silva]

Yeah and I think some of the issues are related to not having a clear logical boundary between tenants as when you have with realms. And definitely, a blocker if you have some specific realm settings that change between tenants.

From an operation perspective, you also have issues due to tenants sharing the resources. Where in a multi-realm approach you could theoretically segregate nodes on a per-realm basis (although they are still members of a single cluster).

[Thibaud Benoist]

Hi Sagun,

We are trying to use several optimisations. The main one was proposed by Pedro on keycloak 13.0 but not merged in the master.

At this time it works for us with 2600 realms. We can use the admin and most API are OK. The last performance problem we should have to solve is the time to POST a new realm (several minutes when we have more then 2000 realms).

Best regards,

Thibaud

[Sagun Tumkar]

Hi Thibaud,

Sorry which option are you referring which works for 2600 realms? Is it one realm per tenant? I am still very new to keycloak so dont know much terminologies. Does it mean one realm per docker container or one realm per host in cluster?

Could you please share any findings you have for performance issue with creating realm?

Thanks in advance.

~Sagun

You received this message because you are subscribed to a topic in the Google Groups "Keycloak Dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/keycloak-dev/ilWkQqJJsRY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to keycloak-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/64cb1c74-9cdd-461f-b218-770a7fa0e5e0n%40googlegroups.com.

[Thibaud Benoist]

I have 2600 realms on one docker. For the moment I did not used cluster. I ´ll share with pleasure. 

--

Thibaud BENOIST

06 59 25 53 09

[Willem Noort]

Hi Thibaud,

 

We’d love to see the fixes that enabled you to scale to 2600 realms! In our use case, we use a realm per customer too. 2600 realms would really help us out.

 

Kind regards

 

 

Willem Noort

Error! Filename not specified.

--
You received this message because you are subscribed to a topic in the Google Groups "Keycloak Dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/keycloak-dev/ilWkQqJJsRY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to keycloak-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/64cb1c74-9cdd-461f-b218-770a7fa0e5e0n%40googlegroups.com.

--

Sagun Tumkar | Sr. Software Engineer, Application Platform | M: (650-863-4922)

Error! Filename not specified.

--

Thibaud BENOIST

06 59 25 53 09

--

You received this message because you are subscribed to the Google Groups "Keycloak Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keycloak-dev...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/CAPSH85u6opjdJ%3DU84ahdC6rz-F26mo_1Q52ONv4B5EAwn_DXNQ%40mail.gmail.com.

[Nikolas Laskaris]

We have submitted a pull request with a fix for the first bullet

"* Admin console might get unusable as the number of realms grows. And this might impact your availability."

a little while ago, which seems to give a huge boost to the Admin UI. 
Our problem was that we had around 20 realms with more than 500 IdPs each. The UI was getting unusuable, so we had to create this fix (along with a series of other) with which we now have a lot lighter keycloak which seems to run smoothly. 
More details for the fix for the realms' UI, here:
https://issues.redhat.com/browse/KEYCLOAK-18288
https://github.com/keycloak/keycloak/pull/8096

You will probably be interested also for the other pull requests, which make keycloak's Admin UI smoother.
For instance, one of these, adds paging to the IdPs - we have 100s, but this is a nice enhancement even if you have more than 20-30. 

[Pedro Igor]

Yeah, that is one of the things that PR did not include. As you get more data in DB more time it takes when doing write operations. How far this is just a matter of using DB capabilities like partitioning (not saying they are the best fit for solving the problem as it should have other consequences) or reviewing database configuration/DDL (even if for a given database vendor), I don't know.

The fact is that we need to create quite a few data when creating a new realm and ideally, we should review this approach to provisioning, avoid creating more data when provisioning, and perhaps rely on some async processing when (de)provisioning realms.

Some operations are also inherently costly like those that rely on crypto such as storing user passwords or generating keys.

The new store should help us with that. Hynek can give you a much better insight into what is being planned.

[Sagun Tumkar]

Hi Willem,

How many realms you had? Are you guys still using one realm per customer? Do you suggest going on this route or you suggest going with single realm and multiple groups where each group represents one customer?\

Thanks,

Sagun