Custom CRs for Keycloak Operator
226 views
Skip to first unread message
Kevin Reis
Apr 11, 2023, 2:02:34 AM4/11/23
to Keycloak Dev
Hello everyone,
this thread is a continuation of "Contribution to Keycloak.X Operator" [3] as Google Groups keeps deleting my responses on that thread.
Our use case: We have multiple Kubernetes clusters which run isolated from each other. Each Kubernetes cluster contains a Keycloak instance with a preconfigured realm. The preconfiguration always happens with an import of an exported realm JSON via CLI if the associated realm does not exist and it happens before actual Keycloak startup. We need this setup across all Kubernetes clusters so we can assure that the Keycloak configuration is the same across all clusters. You can think of Kubernetes clusters which serve a suite of softwares, with the different clusters representing different deployment environments (Test, staging, ...).
Maintaining, reviewing and versioning the exported JSONs in Git is difficult as we need to handle huge realm JSON files. Things like changes in IDs of the entity in the export JSON (which are irrelevant for the import step) bloat the Git diff. It makes working with these JSONs in Git inconvenient. But having these realm settings split in separate resources makes it easier to maintain the configs (=> More CRDs?).
That's why using an operator crossed our minds. We can declaratively define how a Keycloak should be configured without going through the hassle of exporting and re-importing configurations. Changes are kept in sync in one-way (k8s custom resources -> Keycloak), and these custom resources can be shipped as e. g. Helm Chart. This enables deployment approaches such as "GitOps for Keycloak" or Infrastructure-as-a-Code. In a happy day scenario, we just need to deploy a Keycloak instance, the operator and the configs as custom resources. Then, the configuration happens automatically. If there are any updates to the config, the operator simply applies them to the Keycloak entities with nearly no downtime unlike the strategy of exporting the realm as JSON, deleting Keycloak and re-importing it.
I understand why you might refrain from any CRD changes, especially if the new storage system comes around the corner sooner or later and because of the Keycloak configuration method via Admin REST API being prone to breaking changes. I have read in the architecture design documentation that there is a concept of having Keycloak configuration in GitOps style [1]. But I assume it will take some time until the new storage is production-ready. The waiting time stalls the development of Keycloak configuration logic in the new Keycloak.X operator. Please correct me if I get something wrong. In the meantime, we probably have to keep relying on the old operator.
The old operator (Keycloak Realm Operator) can manage clients and users in a one-way-sync-manner. But the way the Realm Operator reconciles realms is very basic: If the realm does not exist, create it. If it does exist, don't do anything, regardless if there are any changes in the custom resources. I wish it would handle realms like clients and users. I bring this up in this GitHub discussion: [2].
I'm open to suggestions for handling such auto-configurations across multiple Kubernetes clusters.
[1] https://github.com/keycloak/keycloak-community/blob/93a7b6f2185decf5e65a2dce72f80f1d06aae9d4/design/keycloak.x/operator.md#configuring-keycloak-business-objects-using-kubernetes-resources
[2] https://github.com/keycloak/keycloak/discussions/17245#discussion-488926
Maintaining, reviewing and versioning the exported JSONs in Git is difficult as we need to handle huge realm JSON files. Things like changes in IDs of the entity in the export JSON (which are irrelevant for the import step) bloat the Git diff. It makes working with these JSONs in Git inconvenient. But having these realm settings split in separate resources makes it easier to maintain the configs (=> More CRDs?).
That's why using an operator crossed our minds. We can declaratively define how a Keycloak should be configured without going through the hassle of exporting and re-importing configurations. Changes are kept in sync in one-way (k8s custom resources -> Keycloak), and these custom resources can be shipped as e. g. Helm Chart. This enables deployment approaches such as "GitOps for Keycloak" or Infrastructure-as-a-Code. In a happy day scenario, we just need to deploy a Keycloak instance, the operator and the configs as custom resources. Then, the configuration happens automatically. If there are any updates to the config, the operator simply applies them to the Keycloak entities with nearly no downtime unlike the strategy of exporting the realm as JSON, deleting Keycloak and re-importing it.
I understand why you might refrain from any CRD changes, especially if the new storage system comes around the corner sooner or later and because of the Keycloak configuration method via Admin REST API being prone to breaking changes. I have read in the architecture design documentation that there is a concept of having Keycloak configuration in GitOps style [1]. But I assume it will take some time until the new storage is production-ready. The waiting time stalls the development of Keycloak configuration logic in the new Keycloak.X operator. Please correct me if I get something wrong. In the meantime, we probably have to keep relying on the old operator.
The old operator (Keycloak Realm Operator) can manage clients and users in a one-way-sync-manner. But the way the Realm Operator reconciles realms is very basic: If the realm does not exist, create it. If it does exist, don't do anything, regardless if there are any changes in the custom resources. I wish it would handle realms like clients and users. I bring this up in this GitHub discussion: [2].
I'm open to suggestions for handling such auto-configurations across multiple Kubernetes clusters.
[1] https://github.com/keycloak/keycloak-community/blob/93a7b6f2185decf5e65a2dce72f80f1d06aae9d4/design/keycloak.x/operator.md#configuring-keycloak-business-objects-using-kubernetes-resources
[2] https://github.com/keycloak/keycloak/discussions/17245#discussion-488926
Alexander Schwartz
Apr 13, 2023, 3:25:52 AM4/13/23
to Kevin Reis, Keycloak Dev
Hi Kevin,
the changing IDs you describe here could improve when using the new map file store. Join the discussion here to read more on how to use it and ask more questions: https://github.com/keycloak/keycloak/discussions/16768
Even if it is not officially production ready, it might offer the things you need for your setup already.
Best,
Alexander
Alexander
--
You received this message because you are subscribed to the Google Groups "Keycloak Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keycloak-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/26ea9bab-24a6-4ef2-ad27-345f4ef33b5fn%40googlegroups.com.
Alexander Schwartz, RHCE
He/Him
Principal Software Engineer, Keycloak
Red Hat - Germany remote
 |
Red Hat GmbH, Registered seat: Werner von Siemens Ring 12, D-85630 Grasbrunn, Germany
Commercial register: Amtsgericht Muenchen/Munich, HRB 153243,
Managing Directors: Ryan Barnhart, Charles Cachera, Michael O'Neill, Amy Ross
Commercial register: Amtsgericht Muenchen/Munich, HRB 153243,
Managing Directors: Ryan Barnhart, Charles Cachera, Michael O'Neill, Amy Ross
Václav Muzikář
May 2, 2023, 4:49:03 AM5/2/23
to Kevin Reis, Keycloak Dev
Hello Kevin,
thanks for providing useful insights on your use case. Adding more CRs and/or GitOps for fully declarative config is definitely high on our list but I don't have any ETA to share at the moment.
You are mentioning that you need to create/import a Realm if it does not exist. I wonder, what additional functionality in this aspect the old Operator provides for you that the current RealmImport CR from the new one does not have? Or do you leverage Client and User CRs as well?
On Tue, Apr 11, 2023 at 8:02 AM Kevin Reis <kere...@gmail.com> wrote:
--
You received this message because you are subscribed to the Google Groups "Keycloak Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keycloak-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/26ea9bab-24a6-4ef2-ad27-345f4ef33b5fn%40googlegroups.com.
Václav Muzikář
Principal Software Engineer, Keycloak
|
Kevin Reis
May 12, 2023, 12:54:46 AM5/12/23
to Keycloak Dev
Hi Václav,
the RealmImport CR in the old and new operator serve the same purpose, with the only differences that the properties of the old RealmImport CR are not up-to-date, and the new operator runs the realm import once as a Kubernetes job and then "never ever again" unless the job is deleted (that was at least at the time I tested the new operator). No changes in the RealmImport CR are reconciled by the operators. The old operator may recreate the realm if it doesn't exist (caused by deletion for instance). I know that the CR name implies that the realm should be only created/imported if it doesn't exist, but it'd be great if the operators were also able to reconcile existing realms with the changes to the RealmImport CR.
We leverage clients as CRs as well. The old operator handles Client CRs, the new one currently doesn't and probably won't handle that e.g. due to the file map storage and possible plan on following GitOps approach.
Reply all
Reply to author
Forward
0 new messages