Need help on cache stack ec2

[Arulaln A R]

Hi Team,

We are trying to run Keycloak version 18, based on the EKS cluster. We are trying to build a HA cluster. I have gone through the configuration, where the "KC_CACHE_STACK" value should be defined as "ec2" since our EKS cluster worker nodes run on EC2 instances.

I have come across the page "https://infinispan.org/docs/dev/titles/embedding/embedding.html#jgroups-cloud-discovery-protocols_cluster-transport" based on the articles.

Can see, need to update the pom.xml details and also need to include the aws region, bucket details. 

But not sure what are the configurations i have to carry out under /opt/keycloak/conf/cache-ispn.xml.

If any of the members have carried out this configuration, do let me know. 

Really appreciate if you have any working documents handy which describes all the steps to include "ec2" cache stack

--

Regards,

Arulaln A R

+91-8754438576

[Johannes Reppin]

Hi,

From the documentaion: 

>Note that none of these stacks are Kubernetes / OpenShift stacks, so no need exists to enable the "google" stack if you want to run Keycloak on top of the Google Kubernetes engine. In that case, use the kubernetes stack. 

So go with `kubernetes` instead of `ec2`.

cheers

[Arulaln A R]

HI Johannes,

Below is the complete sentence of the document. For AWS EC2 instances, we can't use the kubernetes stack. Only ec2 should be used.

The following table shows transport stacks that are supported by Keycloak, but need some extra steps to work. Note that none of these stacks are Kubernetes / OpenShift stacks, so no need exists to enable the "google" stack if you want to run Keycloak on top of the Google Kubernetes engine. In that case, use the kubernetes stack. Instead, when you have a distributed cache setup running on AWS EC2 instances, you would need to set the stack to ec2, because ec2 does not support a default discovery mechanism such as UDP.

Stack name	Transport protocol	Discovery
ec2	TCP	NATIVE_S3_PING
google	TCP	GOOGLE_PING2
azure	TCP	AZURE_PING

Cloud vendor specific stacks have additional dependencies for Keycloak. For more information and links to repositories with these dependencies, see the Infinispan documentation.

--
You received this message because you are subscribed to the Google Groups "Keycloak User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to keycloak-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-user/1cdb372d-2486-408b-b080-54e0a9984ca9n%40googlegroups.com.

[Bruno Ribeiro]

Hey Arulaln,

Even if you're running on an EC2 environment, you can use other ways for discovery, such as JDBC or DNS. But the default config for ec2 is NATIVE_S3_PING. I'm assuming you're going with NATIVE_S3_PING as you talked about AWS configurations, so this might help you: https://github.com/jgroups-extras/jgroups-aws

And the default ec2 configuration provided by Infinispan: https://github.com/infinispan/infinispan/blob/main/core/src/main/resources/default-configs/default-jgroups-ec2.xml

So maybe setting the JAVA_OPTS_APPEND environment variable to something like the below works (assuming you're using the quarkus distribution and AWS credentials are correctly set):

JAVA_OPTS_APPEND=-Djgroups.s3.region_name={put-the-region-here} -Djgroups.s3.bucket_name={put-your-bucket-name-here}

Best,

Bruno Ribeiro
http://brunocesar.com

[Arulaln A R]

Thanks Bruno, when I run on EKS cluster I can still use the kubernetes stack?

Only when I run keycloak as the local to EC2 instances should I go for ec2 stack?

please confirm this point

To view this discussion on the web visit https://groups.google.com/d/msgid/keycloak-user/e9543716-5fe5-447f-9310-7a93bc1180e7n%40googlegroups.com.

[Bruno César]

Hi Arulaln,

AFAIK, you can set the cache stack config to Kubernetes when running on EKS. I never run it on EKS, but on GKE, OpenShift and local cluster I only need something like the below for JAVA_OPTS_APPEND.

-Djgroups.dns.query={{ your service name here }}.{{ .Release.Namespace }}.svc.cluster.local

KC_CACHE_STACK is set to "kubernetes", of course.

Best,

Bruno Ribeiro
http://brunocesar.com

[Arulaln A R]

Hi Bruno,

Thanks, yes. cache stack should be supplied during the build stage itself. JAVA_OPTS_APPEND environment variable let me give it during starting keycloak as a pod

[Arulaln A R]

HI Bruno,

I have set up according to the comments below, but don't see any cluster nodes formed. I think, according to the article the service which keycloak is exposed to should be headless service (clusterip is none), in our case it is not. Maybe that might be the reason? I don't see any error. I ran it with two instances both have started up and running without any issues. Only think it wasn't recognised as a cluster.

Example Kubernetes Yaml env details :

- name: JAVA_OPTS_APPEND

  value: "-Djgroups.dns.query=servicename.namespace.svc.cluster.local"

During build stage itself i have set the environment varible under Dockerfile "ENV KC_CACHE_STACK=kubernetes"

You mentioned in your earlier email that you have setup in GKE, kubernetes. What type of service your keycloak is exposed? is it LB/clusterip/NodePort

[Bruno César]

Hi Arulaln,

For the discovery service, I'm using ClusterIP and setting clusterIP to None, basically, like the below:

apiVersion: v1
kind: Service
metadata:
  name: keycloak-discovery
  labels:
    {{ include "labels" . | nindent 4 }}
spec:
  type: ClusterIP
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 8080
  clusterIP: None
  sessionAffinity: None
  selector:
    app: {{ template "selector.name" . }}
    release: {{ .Release.Name }}

It is pretty much like the one in the Keycloak Operator.

Best,

Bruno Ribeiro
http://brunocesar.com

[Arulaln A R]

Thanks Bruno, we also followed a similar setup. Where we have two services (headless and normal service) pointing to the same keycloak applications. And used the headless service for dns ping (kubernetes stack). Works smoothly.