Re: [keycloak-dev] KEYCLOAK-15476: Support external Keycloak in Operator

[Sebastian Łaskawiec]

Thanks for the proposal.

I'm not in favor of this. Keycloak Operator is an opinionated way for deploying and managing Keycloaks in your Kubernetes/OpenShift cluster. Managing a cluster that has been deployed manually is something that goes in an opposite direction.

On Mon, 14 Sep 2020 at 11:58, 'Jochen Niebuhr' via Keycloak Dev <keyclo...@googlegroups.com> wrote:

Hello,

I've opened this issue because we would like to use the operator with an external keycloak instance and I think other people could have this use case as well.

Is this an addition you guys would want and what would be necessary to get it in the operator? I'm willing to submit a PR for this issue.

Greetings, 

Jochen Niebuhr.

--
You received this message because you are subscribed to the Google Groups "Keycloak Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keycloak-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/56464f56-0d77-458d-ab4b-82d99a0d63b4n%40googlegroups.com.

--

Sebastian Łaskawiec

[Jochen Niebuhr]

In our case the Keycloak instance is on another Kubernetes cluster and also deployed with the operator. But to create clients from the application cluster the operator would need this to reference the Keycloak instance.

[Stian Thorgersen]

I can see the need for an Operator in one cluster to be able to register/manage clients in a different cluster (or even not deployed to K8s at all).

"Support external Keycloak in Operator" is definitively to broad a scope, but perhaps this would actually make sense, and we could at least initially limit it to only KeycloakCLient CRD.

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/18b43154-5621-429a-8e4a-484fe01c5191n%40googlegroups.com.

[Jochen Niebuhr]

Sure, only clients would be enough from us. From what I've tried in the code it seems pretty easy to return both Keycloak and "ExternalKeycloak" CRDs from the discovery method and extend the rest client creation to talk to an external keycloak. All the other logic would then be the same I guess.

[Sebastian Łaskawiec]

I'm not convinced Operators should play any role with this kind of architecture. Perhaps it's worth reaching the Operator SDK Team out?

I see this as a problem of syncing resources (CRs) between two Kubernetes clusters. There are tools designed to play this role. Personally I haven't used neither of them, but maybe it's worth to give them a go (before diving straight into the code):

- Kubernetes Federation (as far as I know, there's v1 and v2 version of it)

- Portworx: https://docs.portworx.com/portworx-install-with-kubernetes/disaster-recovery/px-metro/3-schedule-migrations/

- IBM Sync Operator: https://www.ibm.com/cloud/blog/secret-sync-operator

Those are just a few from the top search in Google. I believe if you spend a bit more time on it, you will find something better/more suitable.

Personally, I would prefer to keep such code away from the Operator. The implementation doesn't seem to be hard now but in the future we will be adding more features that may collide with it, such as using Service Accounts [1]. With that feature on, you will need to also sync Secrets and possibly Realms between the clusters. It might become messy very quickly...

[1] https://github.com/keycloak/keycloak-operator/pull/244

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/b569a12a-3fca-4ed9-a24c-672d26c78620n%40googlegroups.com.

[Jochen Niebuhr]

Hi,

I agree on what Kevin said. 

About Sebastians points:

Yes, I've found that operator and know about the other federation ways. The operator you've mentioned only does secrets. I couldn't find one that performs general resource syncs. 

Setting up federation just for keycloak CRDs seems a bit too much for me.

For the implementation part:

I did a little test on replacing the "Keycloak" resource everywhere with an interface and adding another "ExternalKeycloak" resource and all existing logic works just fine in the Unit/E2E tests.

If we want to use the existing resources, there will be a little field duplication because as of right now, the internal url is taken from status. That field would need to be duplicated into spec. Both methods work I guess.

Am Mo., 14. Sept. 2020 um 19:21 Uhr schrieb Fox, Kevin M <Kevi...@pnnl.gov>:

Yeah. KeycloakClient at minimum would be very very useful. User might be useful. Realm might be useful.

API wise though, just setting some kind of external flag and url in the  Keycloak CR would then let KeycloakClient/Realm/etc to work unmodified. The deployment part of the operator could just ignore it and just the part running the Keycloak admin api would still do its thing.

Thanks,
Kevin

________________________________________
From: keyclo...@googlegroups.com <keyclo...@googlegroups.com> on behalf of Stian Thorgersen <stho...@redhat.com>
Sent: Monday, September 14, 2020 3:42 AM
To: Jochen Niebuhr
Cc: Keycloak Dev
Subject: Re: [keycloak-dev] KEYCLOAK-15476: Support external Keycloak in Operator

I can see the need for an Operator in one cluster to be able to register/manage clients in a different cluster (or even not deployed to K8s at all).

"Support external Keycloak in Operator" is definitively to broad a scope, but perhaps this would actually make sense, and we could at least initially limit it to only KeycloakCLient CRD.



On Mon, 14 Sep 2020 at 12:12, 'Jochen Niebuhr' via Keycloak Dev <keyclo...@googlegroups.com<mailto:keyclo...@googlegroups.com>> wrote:
In our case the Keycloak instance is on another Kubernetes cluster and also deployed with the operator. But to create clients from the application cluster the operator would need this to reference the Keycloak instance.

sebastian...@gmail.com<mailto:sebastian...@gmail.com> schrieb am Montag, 14. September 2020 um 12:09:37 UTC+2:
Thanks for the proposal.

I'm not in favor of this. Keycloak Operator is an opinionated way for deploying and managing Keycloaks in your Kubernetes/OpenShift cluster. Managing a cluster that has been deployed manually is something that goes in an opposite direction.

On Mon, 14 Sep 2020 at 11:58, 'Jochen Niebuhr' via Keycloak Dev <keyclo...@googlegroups.com> wrote:
Hello,

I've opened this issue because we would like to use the operator with an external keycloak instance and I think other people could have this use case as well.
Is this an addition you guys would want and what would be necessary to get it in the operator? I'm willing to submit a PR for this issue.

Greetings,
Jochen Niebuhr.

--
You received this message because you are subscribed to the Google Groups "Keycloak Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keycloak-dev...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/56464f56-0d77-458d-ab4b-82d99a0d63b4n%40googlegroups.com<https://groups.google.com/d/msgid/keycloak-dev/56464f56-0d77-458d-ab4b-82d99a0d63b4n%40googlegroups.com?utm_medium=email&utm_source=footer>.

--
Sebastian Łaskawiec

--
You received this message because you are subscribed to the Google Groups "Keycloak Dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to keycloak-dev...@googlegroups.com<mailto:keycloak-dev...@googlegroups.com>.
To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/18b43154-5621-429a-8e4a-484fe01c5191n%40googlegroups.com<https://groups.google.com/d/msgid/keycloak-dev/18b43154-5621-429a-8e4a-484fe01c5191n%40googlegroups.com?utm_medium=email&utm_source=footer>.

--
You received this message because you are subscribed to the Google Groups "Keycloak Dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to keycloak-dev...@googlegroups.com<mailto:keycloak-dev...@googlegroups.com>.
To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/CAJgngAcRpWYcsLf047ZuXKpkNgLv%2B_Q2n%3D1P3vc1NbEJ_jLZNg%40mail.gmail.com<https://groups.google.com/d/msgid/keycloak-dev/CAJgngAcRpWYcsLf047ZuXKpkNgLv%2B_Q2n%3D1P3vc1NbEJ_jLZNg%40mail.gmail.com?utm_medium=email&utm_source=footer>.

[Stian Thorgersen]

End of the day the current Keycloak Operator actually does two distinctly separate tasks:

1. Install and manage Keycloak

2. Register and manage clients

The two separate tasks could just as well have been done by two separate operators, and it will be more than common that clients should be registered in a Keycloak instance that is not controlled by the operator. 

[Sebastian Łaskawiec]

I wonder if creating Clients is the only use case here? What about creating Users, Realms? 

Currently, Keycloak Operator uses admin username and password to create Clients. Probably a slightly more preferred way is to do it via OIDC Client Registration Endpoint but so far we didn't have enough capacity to implement it. But the current situation enables us to sync other resources as well.

As for the implementation, Kevin - may I ask you to create a Pull Request? I'd like to compare it with my recent work around Service Accounts.

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/CAJgngAf%2BLBiQbzPm4y-E11NP3Xyz0k55GwMovS%2Bhr7a%2B55n9_Q%40mail.gmail.com.

--

Sebastian Łaskawiec

[Jochen Niebuhr]

It’s pretty easy to just add the change for all resources at the same time.

I think you mean me with the PR? Yes, I have a draft that’s missing unit tests but I’ll submit it and send you the link.

Am 15.09.2020 um 13:24 schrieb Sebastian Łaskawiec <sebastian...@gmail.com>:



[Jochen Niebuhr]

I have created two suggestions as pull requests. Both are missing documentation and unit/e2e tests for now. I'll add those once we can settle on an implementation style.

https://github.com/keycloak/keycloak-operator/pull/248
https://github.com/keycloak/keycloak-operator/pull/247

[Stian Thorgersen]

On Tue, 15 Sep 2020 at 13:24, Sebastian Łaskawiec <sebastian...@gmail.com> wrote:

I wonder if creating Clients is the only use case here? What about creating Users, Realms? 

I'm really not convinced about Users being a CR at all.

For Realms I'd say that is something that should be managed close to the instance itself. Also, not really sure if a Realm should be managed as a CR, or if that is just a bootstrap thing.

[Stian Thorgersen]

Can you summarize the approaches here?

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/9286eff4-f99f-4f48-9a53-fc0ed7a9ba1fn%40googlegroups.com.

[Jochen Niebuhr]

Variant A is the one I've suggested in my issue. In that one I've created an ExternalKeycloak and ExternalKeycloakRealm CR to be targeted by the instanceSelector and realmSelector fields, much like you can define an ExternalName service in K8S. I created interfaces to be used by the controller code that's implemented by Keycloak and ExternalKeycloak (for KeycloakReference) and KeycloakRealm and ExternalKeycloakRealm (for KeycloakRealmReference). Then it was just a matter of switching the references to the CR to the interfaces and adjusting the GetMatchingRealms and GetMatchingKeycloaks functions.

Variant B was suggested in here. For now I've added just an external Flag to Keycloak and KeycloakRealm which means they will not be managed but can be referenced by other resources. For the Keycloak CR this also means adding the Endpoint and CredentialSecret fields from the status into the spec. The only thing that needed to be done in the controller code was checking the external flag before reconciling any resource and adjusting rest client creation to switch between the different endpoint fields. There were already suggestions in the PR to move the external flag to "external.enabled" or use an enum, so that could be done as well.

[Stian Thorgersen]

On Wed, 16 Sep 2020, 10:08 'Jochen Niebuhr' via Keycloak Dev, <keyclo...@googlegroups.com> wrote:

Variant A is the one I've suggested in my issue. In that one I've created an ExternalKeycloak and ExternalKeycloakRealm CR to be targeted by the instanceSelector and realmSelector fields, much like you can define an ExternalName service in K8S. I created interfaces to be used by the controller code that's implemented by Keycloak and ExternalKeycloak (for KeycloakReference) and KeycloakRealm and ExternalKeycloakRealm (for KeycloakRealmReference). Then it was just a matter of switching the references to the CR to the interfaces and adjusting the GetMatchingRealms and GetMatchingKeycloaks functions.

This approach makes sense to me, but is there really a need to have ExternalKeycloak? Only ExternalKeycloakRealm makes more sense to me.

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/31e35b21-2896-40b8-8057-0412c97e0bcen%40googlegroups.com.

[Sebastian Łaskawiec]

On Wed, 16 Sep 2020 at 14:12, Stian Thorgersen <stho...@redhat.com> wrote:

On Wed, 16 Sep 2020, 10:08 'Jochen Niebuhr' via Keycloak Dev, <keyclo...@googlegroups.com> wrote:

Variant A is the one I've suggested in my issue. In that one I've created an ExternalKeycloak and ExternalKeycloakRealm CR to be targeted by the instanceSelector and realmSelector fields, much like you can define an ExternalName service in K8S. I created interfaces to be used by the controller code that's implemented by Keycloak and ExternalKeycloak (for KeycloakReference) and KeycloakRealm and ExternalKeycloakRealm (for KeycloakRealmReference). Then it was just a matter of switching the references to the CR to the interfaces and adjusting the GetMatchingRealms and GetMatchingKeycloaks functions.

This approach makes sense to me, but is there really a need to have ExternalKeycloak? Only ExternalKeycloakRealm makes more sense to me.

When the Operator creates a Realm, Client or User, it needs to get the Keycloak URL. The URL depends on the Keycloak CR settings (it might be a Service URL, a Route or an Ingress) and its being written back by the Operator into KeycloakCR.status.internalURL field.

 

Variant B was suggested in here. For now I've added just an external Flag to Keycloak and KeycloakRealm which means they will not be managed but can be referenced by other resources. For the Keycloak CR this also means adding the Endpoint and CredentialSecret fields from the status into the spec. The only thing that needed to be done in the controller code was checking the external flag before reconciling any resource and adjusting rest client creation to switch between the different endpoint fields. There were already suggestions in the PR to move the external flag to "external.enabled" or use an enum, so that could be done as well.

Thanks for the both proposals Jochen!

It goes without saying that the Proposal A (the one with separate CRDs) is much larger. This approach requires quite a bit of code duplication. The Proposal B (with spec.external flag) is much simpler. Other Operators seems to handle this scenarios with flags as well:

- CAO has a spec.managementState flag [1]

- Cluster Logging Operator does exactly the same [2]

Maybe we could use a similar naming (so that others could find this option quicker). Something like spec.management.enabled?

I'm leaning towards Proposal B as it's simpler, does all it needs to do and it's similar to what other OpenShift Operators do.

Let me also ask David and Peter for input.

[1] https://github.com/openshift/cluster-authentication-operator/blob/9cb7c09c06878c2776bab8abed26a311035dc53b/manifests/02_config.cr.yaml#L8

[2] https://github.com/openshift/cluster-logging-operator/blob/5971ee038d4bb87df76ab5b168975970f8c3a8cd/hack/collector-only.yaml

 

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/CAJgngAeFV%2B7XtpQr_Nrcby6d6DjWwE78iVkGKkip%2BgXz9re8oA%40mail.gmail.com.

--

Sebastian Łaskawiec

[Sebastian Łaskawiec]

The only problem with Proposal B that I can think of is how to specify a URL to the external Keycloak? Putting it into the status doesn't seem like a good fit. I propose to create an additional field in the spec. However, then using spec.management seems a bit awkward.

So how about this:

- spec.external.enabled

- spec.external.externalURL (for the symmetry with the status field)

So similarly to what's in the Pull Request.

[Jochen Niebuhr]

I wanted to keep the possibility in both PRs to also create realms for those keycloaks, that's why I had the two separate CRs in Variant A.

If we want to use the "managementState" field, we could have a separate field called "access" containing "endpoint" and "credentialsSecret" fields which is required when "managementState" is "Unmanaged".

[Sebastian Łaskawiec]

Honestly, I think spec.external is a better name. So the question is what we will need there. I can see:

- spec.external.enabled

- spec.external.externalURL

There's probably no need to specify credential secret here because the Operator figures it out from the CR name. Later on, we can introduce a feature that will enable you to override this behaviour (but there's no need for this right now).

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/fa24d159-f1ea-4995-ad71-091652df6a55n%40googlegroups.com.

--

Sebastian Łaskawiec

[Stian Thorgersen]

I don't think we should support creating realms in an external KC instance, only clients at this point.

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/CADP5VTXS6rFq1d%2B8iy2w7Yiu-pcZSK6xwdRy8NJfRdGYP55mzQ%40mail.gmail.com.

[Jochen Niebuhr]

Alright. So I guess I'll change the PR accordingly and start adding tests and documentation.

[Sebastian Łaskawiec]

Great! Thank you!

Please have in mind that Keycloak Operator's documentation lives in keycloak-documentation repo.

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/41933c61-0966-4cf3-9a86-819364f78f81n%40googlegroups.com.

--

Sebastian Łaskawiec

[Sebastian Łaskawiec]

I like this idea!

On Tue, 22 Sep 2020 at 18:17, Fox, Kevin M <Kevi...@pnnl.gov> wrote:

+1 to client only.

I'd recommend making a "unmanaged" field of KeycloakRealm and must be set to true if Keycloak CR is external, and if false or missing, throw an error.

This would allow the option for a managed one to be added later without breaking any existing systems if desired. It also could allow which default, true or false to be decided later without breaking existing deployments.

Thanks,
Kevin

________________________________________
From: keyclo...@googlegroups.com <keyclo...@googlegroups.com> on behalf of Stian Thorgersen <stho...@redhat.com>

Sent: Thursday, September 17, 2020 1:03 AM
To: Sebastian Łaskawiec
Cc: Jochen Niebuhr; Keycloak Dev

Subject: Re: [keycloak-dev] KEYCLOAK-15476: Support external Keycloak in Operator

I don't think we should support creating realms in an external KC instance, only clients at this point.

On Thu, 17 Sep 2020 at 09:28, Sebastian Łaskawiec <sebastian...@gmail.com<mailto:sebastian...@gmail.com>> wrote:
Honestly, I think spec.external is a better name. So the question is what we will need there. I can see:
- spec.external.enabled
- spec.external.externalURL

There's probably no need to specify credential secret here because the Operator figures it out from the CR name. Later on, we can introduce a feature that will enable you to override this behaviour (but there's no need for this right now).

On Wed, 16 Sep 2020 at 14:54, 'Jochen Niebuhr' via Keycloak Dev <keyclo...@googlegroups.com<mailto:keyclo...@googlegroups.com>> wrote:
I wanted to keep the possibility in both PRs to also create realms for those keycloaks, that's why I had the two separate CRs in Variant A.

If we want to use the "managementState" field, we could have a separate field called "access" containing "endpoint" and "credentialsSecret" fields which is required when "managementState" is "Unmanaged".

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/CAJgngAf%2BLBiQbzPm4y-E11NP3Xyz0k55GwMovS%2Bhr7a%2B55n9_Q%40mail.gmail.com<https://groups.google.com/d/msgid/keycloak-dev/CAJgngAf%2BLBiQbzPm4y-E11NP3Xyz0k55GwMovS%2Bhr7a%2B55n9_Q%40mail.gmail.com?utm_medium=email&utm_source=footer>.

--
Sebastian Łaskawiec

--
You received this message because you are subscribed to the Google Groups "Keycloak Dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to keycloak-dev...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/9286eff4-f99f-4f48-9a53-fc0ed7a9ba1fn%40googlegroups.com<https://groups.google.com/d/msgid/keycloak-dev/9286eff4-f99f-4f48-9a53-fc0ed7a9ba1fn%40googlegroups.com?utm_medium=email&utm_source=footer>.

--
You received this message because you are subscribed to the Google Groups "Keycloak Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keycloak-dev...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/31e35b21-2896-40b8-8057-0412c97e0bcen%40googlegroups.com<https://groups.google.com/d/msgid/keycloak-dev/31e35b21-2896-40b8-8057-0412c97e0bcen%40googlegroups.com?utm_medium=email&utm_source=footer>.

--
You received this message because you are subscribed to the Google Groups "Keycloak Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keycloak-dev...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/CAJgngAeFV%2B7XtpQr_Nrcby6d6DjWwE78iVkGKkip%2BgXz9re8oA%40mail.gmail.com<https://groups.google.com/d/msgid/keycloak-dev/CAJgngAeFV%2B7XtpQr_Nrcby6d6DjWwE78iVkGKkip%2BgXz9re8oA%40mail.gmail.com?utm_medium=email&utm_source=footer>.

--
Sebastian Łaskawiec



--
You received this message because you are subscribed to the Google Groups "Keycloak Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keycloak-dev...@googlegroups.com<mailto:keycloak-dev...@googlegroups.com>.

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/fa24d159-f1ea-4995-ad71-091652df6a55n%40googlegroups.com<https://groups.google.com/d/msgid/keycloak-dev/fa24d159-f1ea-4995-ad71-091652df6a55n%40googlegroups.com?utm_medium=email&utm_source=footer>.

--
Sebastian Łaskawiec

--
You received this message because you are subscribed to the Google Groups "Keycloak Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keycloak-dev...@googlegroups.com<mailto:keycloak-dev...@googlegroups.com>.

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/CADP5VTXS6rFq1d%2B8iy2w7Yiu-pcZSK6xwdRy8NJfRdGYP55mzQ%40mail.gmail.com<https://groups.google.com/d/msgid/keycloak-dev/CADP5VTXS6rFq1d%2B8iy2w7Yiu-pcZSK6xwdRy8NJfRdGYP55mzQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.

--
You received this message because you are subscribed to the Google Groups "Keycloak Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keycloak-dev...@googlegroups.com<mailto:keycloak-dev...@googlegroups.com>.

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-dev/CAJgngAc-HOTwJv8A7Ct9EZCPQd_-KvPFQhYg%2BZgRkfNj0X-bPg%40mail.gmail.com<https://groups.google.com/d/msgid/keycloak-dev/CAJgngAc-HOTwJv8A7Ct9EZCPQd_-KvPFQhYg%2BZgRkfNj0X-bPg%40mail.gmail.com?utm_medium=email&utm_source=footer>.

--

Sebastian Łaskawiec

[Jochen Niebuhr]

Hello,

I finished the PR and it's ready to be reviewed. Some test is failing with a weird error which isn't happening locally. Can someone assist here?

Greetings,

Jochen Niebuhr.

[Peter Macherey]

Hey, there I am new here and tried to follow the discussion as best as possible. I found this PR (https://github.com/keycloak/keycloak-operator/pull/247) in the repo which enables us to add external KCs. The final implementation of the PR  includes multiple checks that throw errors if you try to add an external managed KC, realms, clients, etc. What is the purpose of an external KC resource if we cannot manage it. I would really like it to be able to install keycloak in a cluster via a helm chart and still be able to manage the resource via code (realms, clients etc). Can someone help me here?